2024-09-21T05:51:21.413 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T05:51:21.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:51:21.536 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913520' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPm6F6VVzQX02K9ONp8wvVaKqSkK0SYSMYWSb1xDgp2JmTHjzxFbtwQ7/JfniOGTpffiVyvBJLxSbd0ZGx6rlD0= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI553tNzeFZFPwdIJ5YyGRqT9XG/R1eRzNF+bxkvgTob6x9wxerBxtlVVCBrDJprZ5wEJc3yjyBBo91yN/d4GQY= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBmIlU/JspkdvDM6TkuqCTQ9e0nJ7a8XrYELsH+sWLRkdshPs7/+s+vaaDEYwRrooioGnPejYkua/xRkqSO5eKU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T05:51:21.536 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T05:51:21.537 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T05:51:21.537 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T05:51:21.539 INFO:teuthology.task.internal:Checking packages... 2024-09-21T05:51:21.559 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T05:51:21.559 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T05:51:21.560 INFO:teuthology.packaging:ref: None 2024-09-21T05:51:21.560 INFO:teuthology.packaging:tag: None 2024-09-21T05:51:21.560 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:51:21.560 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:51:21.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:51:21.701 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T05:51:21.703 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T05:51:21.710 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T05:51:21.710 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T05:51:21.749 INFO:teuthology.task.internal:Saving configuration 2024-09-21T05:51:21.771 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T05:51:21.776 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T05:51:21.799 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:46:17.700956', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPm6F6VVzQX02K9ONp8wvVaKqSkK0SYSMYWSb1xDgp2JmTHjzxFbtwQ7/JfniOGTpffiVyvBJLxSbd0ZGx6rlD0='} 2024-09-21T05:51:21.817 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:46:17.704223', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI553tNzeFZFPwdIJ5YyGRqT9XG/R1eRzNF+bxkvgTob6x9wxerBxtlVVCBrDJprZ5wEJc3yjyBBo91yN/d4GQY='} 2024-09-21T05:51:21.836 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:46:17.702938', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBmIlU/JspkdvDM6TkuqCTQ9e0nJ7a8XrYELsH+sWLRkdshPs7/+s+vaaDEYwRrooioGnPejYkua/xRkqSO5eKU='} 2024-09-21T05:51:21.836 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T05:51:21.841 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T05:51:21.841 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T05:51:21.842 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T05:51:21.842 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T05:51:21.922 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f27734db640>, signals=[15]) 2024-09-21T05:51:21.923 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T05:51:21.931 INFO:teuthology.task.internal:Opening connections... 2024-09-21T05:51:21.931 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T05:51:21.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:51:22.008 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T05:51:22.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:51:22.081 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T05:51:22.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:51:22.154 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T05:51:22.160 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-21T05:51:22.178 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-21T05:51:22.179 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-21T05:51:22.236 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-21T05:51:22.236 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:51:22.237 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:51:22.238 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-21T05:51:22.260 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-21T05:51:22.278 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-21T05:51:22.278 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-21T05:51:22.336 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-21T05:51:22.336 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-21T05:51:22.336 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-21T05:51:22.336 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:51:22.336 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:51:22.337 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:51:22.337 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-21T05:51:22.360 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-21T05:51:22.379 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-21T05:51:22.379 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:51:22.436 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-21T05:51:22.437 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:51:22.437 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:51:22.437 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:51:22.437 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:51:22.437 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:51:22.437 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:51:22.437 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-21T05:51:22.460 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T05:51:22.469 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T05:51:22.474 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T05:51:22.474 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:51:22.476 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:51:22.479 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:51:22.495 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T05:51:22.501 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T05:51:22.501 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:51:22.533 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:51:22.536 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:51:22.634 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T05:51:22.708 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T05:51:22.720 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T05:51:22.720 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T05:51:22.721 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T05:51:22.721 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:51:22.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:51:22.738 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-21T05:51:22.795 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-21T05:51:22.796 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-21T05:51:22.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-21T05:51:25.657 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.3 MB 00:01 2024-09-21T05:51:27.467 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-21T05:51:31.918 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T05:51:33.660 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-21T05:51:34.431 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T05:51:40.044 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-09-21T05:51:42.009 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T05:51:42.009 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T05:51:42.103 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T05:51:42.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-21T05:51:42.120 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-21T05:51:42.254 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 344 kB/s | 36 kB 00:00 2024-09-21T05:51:42.963 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-21T05:51:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-21T05:51:43.813 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-21T05:51:43.813 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:51:43.813 INFO:teuthology.orchestra.run.smithi031.stdout:Total 47 MB/s | 79 MB 00:01 2024-09-21T05:51:44.522 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-21T05:51:44.899 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-21T05:51:44.899 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-21T05:51:45.882 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-21T05:51:45.882 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-21T05:51:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-21T05:51:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:51:49.224 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:51:51.967 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:51:52.094 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:51:57.209 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:51:57.253 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:52:02.399 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:52:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:52:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:52:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:52:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:52:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:52:41.825 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:52:42.638 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:52:42.638 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:52:42.639 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-21T05:52:42.987 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:52:43.086 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:52:43.087 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:52:43.087 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T05:52:43.087 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T05:52:43.087 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T05:52:43.087 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T05:52:43.088 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T05:52:43.088 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:52:43.146 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:52:43.147 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:52:43.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T05:52:43.185 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-21T05:52:43.186 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:52:43.187 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T05:52:43.187 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T05:52:43.187 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-508.el9.x86_64 2024-09-21T05:52:43.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T05:52:43.879 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-21T05:52:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T05:52:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-21T05:52:45.204 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-21T05:52:45.219 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.bmMyHIfoI2 2024-09-21T05:52:45.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bmMyHIfoI2 2024-09-21T05:52:45.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.bmMyHIfoI2 2024-09-21T05:52:45.438 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.bmMyHIfoI2 is 6KB 2024-09-21T05:52:45.486 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.bmMyHIfoI2 2024-09-21T05:52:45.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T05:52:45.563 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T05:52:45.563 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T05:52:45.563 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T05:52:45.563 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T05:52:45.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T05:52:45.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-21T05:53:15.724 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T05:53:15.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T05:53:15.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:53:34.191 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-21T05:53:43.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T05:53:43.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:54:01.587 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-21T05:54:13.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T05:54:13.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:54:14.007 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-21T05:54:14.626 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T05:54:14.626 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T05:54:14.626 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-21T05:54:14.642 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-508.el9.x86_64 2024-09-21T05:54:14.642 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T05:54:14.642 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:54:14.642 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T05:54:15.643 DEBUG:teuthology.parallel:result is None 2024-09-21T05:54:15.643 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T05:54:15.650 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T05:54:15.651 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:54:15.654 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:54:15.656 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:54:15.675 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T05:54:15.681 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T05:54:15.692 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T05:54:15.692 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:54:15.710 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:54:15.715 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:54:15.821 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T05:54:15.828 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T05:54:15.829 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:54:15.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:54:15.846 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:54:15.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:54:15.864 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:54:15.866 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:54:15.889 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:54:15.949 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:54:15.951 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:54:15.969 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:54:15.970 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:54:15.974 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:54:16.025 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:54:16.027 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T05:54:16.035 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T05:54:16.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:54:16.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:54:16.040 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:54:16.069 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T05:54:16.077 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T05:54:16.077 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:54:16.103 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:54:16.108 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:54:16.126 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:54:16.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:54:16.266 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:54:16.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:54:16.350 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T05:54:16.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:54:16.444 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:54:16.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:54:16.538 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:54:16.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:54:16.667 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T05:54:16.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:54:16.735 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:54:16.778 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:54:16.845 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:54:16.889 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:54:16.932 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T05:54:16.932 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:54:17.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-21T05:54:17.054 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-21T05:54:17.057 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-21T05:54:17.097 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:54:17.100 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:54:17.101 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:54:17.584 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T05:54:17.598 INFO:teuthology.task.internal:Starting timer... 2024-09-21T05:54:17.598 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T05:54:17.643 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T05:54:17.658 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T05:54:17.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-21T05:54:17.697 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-21T05:54:17.700 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-21T05:54:17.764 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-21T05:54:17.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-21T05:54:17.820 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-21T05:54:17.823 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T05:54:17.823 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-21T05:54:17.843 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-21T05:54:17.843 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-21T05:54:17.863 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-21T05:54:17.864 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-21T05:54:17.895 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-21T05:54:17.896 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive'} 2024-09-21T05:54:17.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:54:17.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:54:17.923 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:54:17.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:54:17.951 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:54:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:54:17.979 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T05:54:17.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:54:18.015 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:54:18.050 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:54:18.083 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T05:54:18.092 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T05:54:18.103 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T05:54:18.103 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi107.front.sepia.ceph.com,smithi110.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T05:58:20.293 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com')] 2024-09-21T05:58:20.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T05:58:20.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:58:20.376 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-21T05:58:20.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-21T05:58:20.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T05:58:20.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:58:20.531 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-21T05:58:20.565 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-21T05:58:20.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T05:58:20.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:58:20.641 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-21T05:58:20.718 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-21T05:58:20.719 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T05:58:20.729 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T05:58:20.729 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:58:20.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:58:20.732 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:58:20.732 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:58:20.735 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:58:20.735 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:58:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:58:20.762 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:58:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:58:20.777 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:58:20.789 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:58:20.799 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-21T05:58:20.803 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:58:20.804 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-21T05:58:20.810 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-21T05:58:20.815 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-21T05:58:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:58:20.828 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-21T05:58:20.831 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:58:20.838 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:58:20.840 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-21T05:58:20.845 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:58:20.854 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:58:20.868 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:58:20.893 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:58:20.896 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:58:20.896 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-21T05:58:20.896 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.896 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.896 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.896 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.901 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:58:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:58:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-21T05:58:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.905 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:58:20.927 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:58:20.927 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-21T05:58:20.927 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.927 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.928 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.928 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:58:20.928 INFO:teuthology.run_tasks:Running task install... 2024-09-21T05:58:20.937 DEBUG:teuthology.task.install:project ceph 2024-09-21T05:58:20.938 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T05:58:20.938 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T05:58:20.938 INFO:teuthology.task.install:Using flavor: default 2024-09-21T05:58:20.944 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T05:58:20.944 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T05:58:20.944 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:58:20.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:58:20.947 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:58:20.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:58:20.948 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:58:20.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:58:21.098 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:58:21.099 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:58:21.105 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:58:21.105 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:58:21.109 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:58:21.109 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:58:21.208 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:58:21.208 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T05:58:21.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:58:21.214 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:58:21.215 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T05:58:21.215 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:58:21.221 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:58:21.221 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T05:58:21.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:58:21.235 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:58:21.236 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:58:21.241 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:58:21.241 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:58:21.247 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:58:21.247 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:58:21.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:58:21.307 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:58:21.312 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:58:21.379 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:58:21.382 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:58:21.387 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:58:21.444 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-21T05:58:21.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-21T05:58:21.447 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-09-21T05:58:21.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-09-21T05:58:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-21T05:58:21.452 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-21T05:58:21.671 INFO:teuthology.orchestra.run.smithi110.stdout:45 files removed 2024-09-21T05:58:21.676 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-21T05:58:21.692 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-21T05:58:21.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:58:21.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:58:21.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:58:22.234 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-21T05:58:22.247 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-21T05:58:22.333 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 268 kB/s | 84 kB 00:00 2024-09-21T05:58:22.518 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-21T05:58:22.547 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T05:58:22.634 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T05:58:22.794 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T05:58:22.823 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T05:58:22.918 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T05:58:23.505 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-21T05:58:23.712 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T05:58:23.712 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-21T05:58:25.524 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-21T05:58:29.077 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-21T05:58:29.637 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 20 MB 00:05 2024-09-21T05:58:31.118 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-21T05:58:32.827 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-21T05:58:33.581 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-21T05:58:34.244 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.5 MB 00:01 2024-09-21T05:58:35.213 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T05:58:35.737 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-21T05:58:36.516 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T05:58:36.914 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-21T05:58:37.760 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-21T05:58:38.961 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T05:58:41.063 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:58:41.063 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:58:41.074 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:58:41.075 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:58:41.076 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:58:41.077 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:58:41.078 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:58:41.081 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:58:41.082 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:58:41.143 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:58:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:58:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:58:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:58:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:58:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:58:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:58:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:58:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:58:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:58:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:58:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:58:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:58:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout:Install 149 Packages 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 8 Packages 2024-09-21T05:58:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:58:41.181 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 228 M 2024-09-21T05:58:41.181 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-21T05:58:41.713 INFO:teuthology.orchestra.run.smithi107.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-21T05:58:41.963 INFO:teuthology.orchestra.run.smithi107.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.4 MB/s | 873 kB 00:00 2024-09-21T05:58:42.063 INFO:teuthology.orchestra.run.smithi107.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.4 MB/s | 148 kB 00:00 2024-09-21T05:58:42.155 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T05:58:42.614 INFO:teuthology.orchestra.run.smithi107.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 4.2 MB/s | 2.3 MB 00:00 2024-09-21T05:58:42.689 INFO:teuthology.orchestra.run.smithi107.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 5.0 MB/s | 5.5 MB 00:01 2024-09-21T05:58:42.864 INFO:teuthology.orchestra.run.smithi107.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.1 MB/s | 1.5 MB 00:00 2024-09-21T05:58:43.341 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T05:58:43.397 INFO:teuthology.orchestra.run.smithi107.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.0 MB/s | 5.0 MB 00:00 2024-09-21T05:58:44.431 INFO:teuthology.orchestra.run.smithi107.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 7.0 MB/s | 20 MB 00:02 2024-09-21T05:58:44.433 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:58:44.434 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:58:44.445 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:58:44.446 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:58:44.447 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:58:44.448 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:58:44.450 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:58:44.452 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:58:44.453 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:58:44.519 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-21T05:58:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-21T05:58:44.529 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-21T05:58:44.529 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:58:44.530 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:58:44.531 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:58:44.532 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:58:44.533 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:58:44.534 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:58:44.535 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:58:44.536 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:58:44.537 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:58:44.538 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:58:44.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:58:44.540 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:58:44.541 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:58:44.542 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:Install 149 Packages 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 8 Packages 2024-09-21T05:58:44.543 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:58:44.544 INFO:teuthology.orchestra.run.smithi107.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 232 kB/s | 25 kB 00:00 2024-09-21T05:58:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 228 M 2024-09-21T05:58:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-21T05:58:44.932 INFO:teuthology.orchestra.run.smithi107.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 6.8 MB/s | 10 MB 00:01 2024-09-21T05:58:45.007 INFO:teuthology.orchestra.run.smithi107.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.0 MB/s | 226 kB 00:00 2024-09-21T05:58:45.099 INFO:teuthology.orchestra.run.smithi107.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 295 kB/s | 27 kB 00:00 2024-09-21T05:58:45.174 INFO:teuthology.orchestra.run.smithi107.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T05:58:45.324 INFO:teuthology.orchestra.run.smithi107.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 5.0 MB/s | 766 kB 00:00 2024-09-21T05:58:45.375 INFO:teuthology.orchestra.run.smithi107.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T05:58:45.425 INFO:teuthology.orchestra.run.smithi107.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T05:58:45.447 INFO:teuthology.orchestra.run.smithi031.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 57 kB/s | 6.6 kB 00:00 2024-09-21T05:58:45.478 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:58:45.479 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:58:45.492 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:58:45.492 INFO:teuthology.orchestra.run.smithi107.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 7.4 MB/s | 507 kB 00:00 2024-09-21T05:58:45.493 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:58:45.494 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:58:45.495 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:58:45.496 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:58:45.499 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:58:45.500 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:58:45.562 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-21T05:58:45.587 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-21T05:58:45.587 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-21T05:58:45.587 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-21T05:58:45.587 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-21T05:58:45.587 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:58:45.588 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:58:45.589 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-09-21T05:58:45.590 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:58:45.591 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:58:45.592 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:58:45.593 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:58:45.594 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:58:45.595 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:58:45.596 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:58:45.597 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:58:45.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:58:45.599 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout:Install 149 Packages 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 8 Packages 2024-09-21T05:58:45.600 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:58:45.601 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 228 M 2024-09-21T05:58:45.601 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-21T05:58:45.990 INFO:teuthology.orchestra.run.smithi031.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.6 MB/s | 873 kB 00:00 2024-09-21T05:58:46.148 INFO:teuthology.orchestra.run.smithi031.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 938 kB/s | 148 kB 00:00 2024-09-21T05:58:46.192 INFO:teuthology.orchestra.run.smithi107.stdout:(18/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 5.4 MB/s | 18 MB 00:03 2024-09-21T05:58:46.273 INFO:teuthology.orchestra.run.smithi031.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 5.9 MB/s | 5.5 MB 00:00 2024-09-21T05:58:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 492 kB/s | 45 kB 00:00 2024-09-21T05:58:46.351 INFO:teuthology.orchestra.run.smithi110.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-09-21T05:58:46.665 INFO:teuthology.orchestra.run.smithi031.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-09-21T05:58:46.826 INFO:teuthology.orchestra.run.smithi110.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.8 MB/s | 873 kB 00:00 2024-09-21T05:58:46.876 INFO:teuthology.orchestra.run.smithi107.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 3.8 MB/s | 5.3 MB 00:01 2024-09-21T05:58:46.893 INFO:teuthology.orchestra.run.smithi110.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T05:58:46.909 INFO:teuthology.orchestra.run.smithi107.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 237 kB/s | 148 kB 00:00 2024-09-21T05:58:46.960 INFO:teuthology.orchestra.run.smithi107.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T05:58:47.093 INFO:teuthology.orchestra.run.smithi107.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 1.7 MB/s | 321 kB 00:00 2024-09-21T05:58:47.194 INFO:teuthology.orchestra.run.smithi107.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 994 kB/s | 99 kB 00:00 2024-09-21T05:58:47.202 INFO:teuthology.orchestra.run.smithi110.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 7.4 MB/s | 2.3 MB 00:00 2024-09-21T05:58:47.207 INFO:teuthology.orchestra.run.smithi031.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 2.3 MB 00:01 2024-09-21T05:58:47.252 INFO:teuthology.orchestra.run.smithi107.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T05:58:47.311 INFO:teuthology.orchestra.run.smithi107.stdout:(26/157): python3-rbd-19.3.0-5120.g18432218.el9 874 kB/s | 307 kB 00:00 2024-09-21T05:58:47.411 INFO:teuthology.orchestra.run.smithi107.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 170 kB 00:00 2024-09-21T05:58:47.427 INFO:teuthology.orchestra.run.smithi110.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-09-21T05:58:47.470 INFO:teuthology.orchestra.run.smithi107.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 588 kB/s | 34 kB 00:00 2024-09-21T05:58:47.474 INFO:teuthology.orchestra.run.smithi031.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 6.1 MB/s | 5.0 MB 00:00 2024-09-21T05:58:47.537 INFO:teuthology.orchestra.run.smithi107.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.3 MB/s | 154 kB 00:00 2024-09-21T05:58:47.833 INFO:teuthology.orchestra.run.smithi031.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 8.0 MB/s | 20 MB 00:02 2024-09-21T05:58:47.877 INFO:teuthology.orchestra.run.smithi110.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-21T05:58:47.908 INFO:teuthology.orchestra.run.smithi031.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 336 kB/s | 25 kB 00:00 2024-09-21T05:58:48.329 INFO:teuthology.orchestra.run.smithi107.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 2.9 MB/s | 3.1 MB 00:01 2024-09-21T05:58:49.536 INFO:teuthology.orchestra.run.smithi110.stdout:(7/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 11 MB/s | 18 MB 00:01 2024-09-21T05:58:49.675 INFO:teuthology.orchestra.run.smithi031.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 4.7 MB/s | 10 MB 00:02 2024-09-21T05:58:49.750 INFO:teuthology.orchestra.run.smithi031.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.9 MB/s | 226 kB 00:00 2024-09-21T05:58:49.817 INFO:teuthology.orchestra.run.smithi031.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 405 kB/s | 27 kB 00:00 2024-09-21T05:58:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 475 kB/s | 32 kB 00:00 2024-09-21T05:58:50.067 INFO:teuthology.orchestra.run.smithi031.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 4.1 MB/s | 766 kB 00:00 2024-09-21T05:58:50.134 INFO:teuthology.orchestra.run.smithi031.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T05:58:50.211 INFO:teuthology.orchestra.run.smithi110.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 15 MB/s | 10 MB 00:00 2024-09-21T05:58:50.261 INFO:teuthology.orchestra.run.smithi110.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 507 kB/s | 25 kB 00:00 2024-09-21T05:58:50.343 INFO:teuthology.orchestra.run.smithi031.stdout:(16/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 5.7 MB/s | 18 MB 00:03 2024-09-21T05:58:50.368 INFO:teuthology.orchestra.run.smithi031.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 542 kB/s | 127 kB 00:00 2024-09-21T05:58:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 3.3 MB/s | 507 kB 00:00 2024-09-21T05:58:50.560 INFO:teuthology.orchestra.run.smithi031.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 676 kB/s | 45 kB 00:00 2024-09-21T05:58:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.2 MB/s | 148 kB 00:00 2024-09-21T05:58:50.670 INFO:teuthology.orchestra.run.smithi107.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 3.2 MB/s | 7.4 MB 00:02 2024-09-21T05:58:50.703 INFO:teuthology.orchestra.run.smithi031.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T05:58:50.778 INFO:teuthology.orchestra.run.smithi031.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 4.2 MB/s | 321 kB 00:00 2024-09-21T05:58:50.795 INFO:teuthology.orchestra.run.smithi107.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.3 MB/s | 292 kB 00:00 2024-09-21T05:58:50.845 INFO:teuthology.orchestra.run.smithi031.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T05:58:50.871 INFO:teuthology.orchestra.run.smithi107.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 658 kB/s | 49 kB 00:00 2024-09-21T05:58:50.912 INFO:teuthology.orchestra.run.smithi031.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T05:58:50.938 INFO:teuthology.orchestra.run.smithi107.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T05:58:50.978 INFO:teuthology.orchestra.run.smithi031.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T05:58:51.071 INFO:teuthology.orchestra.run.smithi107.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 2.1 MB/s | 285 kB 00:00 2024-09-21T05:58:51.228 INFO:teuthology.orchestra.run.smithi110.stdout:(10/157): ceph-base-19.3.0-5120.g18432218.el9.x 1.1 MB/s | 5.5 MB 00:05 2024-09-21T05:58:51.279 INFO:teuthology.orchestra.run.smithi031.stdout:(26/157): librgw2-19.3.0-5120.g18432218.el9.x86 5.8 MB/s | 5.3 MB 00:00 2024-09-21T05:58:51.313 INFO:teuthology.orchestra.run.smithi107.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.2 MB/s | 784 kB 00:00 2024-09-21T05:58:51.354 INFO:teuthology.orchestra.run.smithi031.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 170 kB 00:00 2024-09-21T05:58:51.378 INFO:teuthology.orchestra.run.smithi110.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.5 MB/s | 226 kB 00:00 2024-09-21T05:58:51.380 INFO:teuthology.orchestra.run.smithi107.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T05:58:51.429 INFO:teuthology.orchestra.run.smithi031.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 6.9 MB/s | 3.1 MB 00:00 2024-09-21T05:58:51.437 INFO:teuthology.orchestra.run.smithi110.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 464 kB/s | 27 kB 00:00 2024-09-21T05:58:51.463 INFO:teuthology.orchestra.run.smithi031.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 315 kB/s | 34 kB 00:00 2024-09-21T05:58:51.488 INFO:teuthology.orchestra.run.smithi031.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T05:58:51.489 INFO:teuthology.orchestra.run.smithi107.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-21T05:58:51.504 INFO:teuthology.orchestra.run.smithi110.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 475 kB/s | 32 kB 00:00 2024-09-21T05:58:51.614 INFO:teuthology.orchestra.run.smithi107.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 6.3 MB/s | 801 kB 00:00 2024-09-21T05:58:51.656 INFO:teuthology.orchestra.run.smithi107.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T05:58:51.698 INFO:teuthology.orchestra.run.smithi107.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-09-21T05:58:51.770 INFO:teuthology.orchestra.run.smithi107.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.2 MB 00:00 2024-09-21T05:58:51.812 INFO:teuthology.orchestra.run.smithi107.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T05:58:51.845 INFO:teuthology.orchestra.run.smithi107.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-09-21T05:58:51.887 INFO:teuthology.orchestra.run.smithi107.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-21T05:58:51.912 INFO:teuthology.orchestra.run.smithi110.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 1.8 MB/s | 766 kB 00:00 2024-09-21T05:58:51.929 INFO:teuthology.orchestra.run.smithi107.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-09-21T05:58:52.005 INFO:teuthology.orchestra.run.smithi110.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.7 MB/s | 158 kB 00:00 2024-09-21T05:58:52.088 INFO:teuthology.orchestra.run.smithi110.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T05:58:52.171 INFO:teuthology.orchestra.run.smithi107.stdout:(47/157): ceph-mgr-dashboard-19.3.0-5120.g18432 3.1 MB/s | 14 MB 00:04 2024-09-21T05:58:52.205 INFO:teuthology.orchestra.run.smithi107.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-21T05:58:52.289 INFO:teuthology.orchestra.run.smithi031.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 9.1 MB/s | 40 MB 00:04 2024-09-21T05:58:52.356 INFO:teuthology.orchestra.run.smithi031.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T05:58:52.389 INFO:teuthology.orchestra.run.smithi031.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 1.5 MB/s | 49 kB 00:00 2024-09-21T05:58:52.463 INFO:teuthology.orchestra.run.smithi110.stdout:(17/157): ceph-test-19.3.0-5120.g18432218.el9.x 18 MB/s | 40 MB 00:02 2024-09-21T05:58:52.481 INFO:teuthology.orchestra.run.smithi031.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 7.4 MB/s | 7.4 MB 00:00 2024-09-21T05:58:52.497 INFO:teuthology.orchestra.run.smithi110.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 1.2 MB/s | 507 kB 00:00 2024-09-21T05:58:52.498 INFO:teuthology.orchestra.run.smithi031.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 151 kB/s | 16 kB 00:00 2024-09-21T05:58:52.532 INFO:teuthology.orchestra.run.smithi031.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.5 MB/s | 285 kB 00:00 2024-09-21T05:58:52.539 INFO:teuthology.orchestra.run.smithi110.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 1.1 MB/s | 45 kB 00:00 2024-09-21T05:58:52.572 INFO:teuthology.orchestra.run.smithi107.stdout:(49/157): ceph-test-19.3.0-5120.g18432218.el9.x 5.0 MB/s | 40 MB 00:08 2024-09-21T05:58:52.589 INFO:teuthology.orchestra.run.smithi107.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 278 kB/s | 107 kB 00:00 2024-09-21T05:58:52.599 INFO:teuthology.orchestra.run.smithi031.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.6 MB/s | 784 kB 00:00 2024-09-21T05:58:52.614 INFO:teuthology.orchestra.run.smithi110.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T05:58:52.624 INFO:teuthology.orchestra.run.smithi031.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 201 kB/s | 18 kB 00:00 2024-09-21T05:58:52.648 INFO:teuthology.orchestra.run.smithi107.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-09-21T05:58:52.665 INFO:teuthology.orchestra.run.smithi107.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 235 kB/s | 22 kB 00:00 2024-09-21T05:58:52.682 INFO:teuthology.orchestra.run.smithi107.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 968 kB/s | 33 kB 00:00 2024-09-21T05:58:52.724 INFO:teuthology.orchestra.run.smithi031.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 639 kB/s | 80 kB 00:00 2024-09-21T05:58:52.731 INFO:teuthology.orchestra.run.smithi110.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.3 MB 00:00 2024-09-21T05:58:52.732 INFO:teuthology.orchestra.run.smithi107.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 1.6 MB/s | 943 kB 00:00 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi107.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi110.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.1 MB/s | 161 kB 00:00 2024-09-21T05:58:52.766 INFO:teuthology.orchestra.run.smithi107.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-09-21T05:58:52.790 INFO:teuthology.orchestra.run.smithi110.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T05:58:52.791 INFO:teuthology.orchestra.run.smithi031.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-21T05:58:52.799 INFO:teuthology.orchestra.run.smithi107.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T05:58:52.817 INFO:teuthology.orchestra.run.smithi031.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-21T05:58:52.833 INFO:teuthology.orchestra.run.smithi107.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-21T05:58:52.840 INFO:teuthology.orchestra.run.smithi110.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T05:58:52.850 INFO:teuthology.orchestra.run.smithi107.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-21T05:58:52.874 INFO:teuthology.orchestra.run.smithi110.stdout:(25/157): python3-rbd-19.3.0-5120.g18432218.el9 2.6 MB/s | 307 kB 00:00 2024-09-21T05:58:52.899 INFO:teuthology.orchestra.run.smithi110.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T05:58:52.900 INFO:teuthology.orchestra.run.smithi031.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-21T05:58:52.900 INFO:teuthology.orchestra.run.smithi107.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-21T05:58:52.934 INFO:teuthology.orchestra.run.smithi107.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 733 kB/s | 24 kB 00:00 2024-09-21T05:58:52.949 INFO:teuthology.orchestra.run.smithi110.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T05:58:52.959 INFO:teuthology.orchestra.run.smithi031.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T05:58:52.967 INFO:teuthology.orchestra.run.smithi107.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T05:58:53.000 INFO:teuthology.orchestra.run.smithi110.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 686 kB/s | 34 kB 00:00 2024-09-21T05:58:53.026 INFO:teuthology.orchestra.run.smithi031.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-21T05:58:53.034 INFO:teuthology.orchestra.run.smithi107.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-21T05:58:53.050 INFO:teuthology.orchestra.run.smithi110.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T05:58:53.068 INFO:teuthology.orchestra.run.smithi031.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-21T05:58:53.068 INFO:teuthology.orchestra.run.smithi107.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-21T05:58:53.093 INFO:teuthology.orchestra.run.smithi031.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-21T05:58:53.102 INFO:teuthology.orchestra.run.smithi107.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-09-21T05:58:53.127 INFO:teuthology.orchestra.run.smithi107.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-21T05:58:53.160 INFO:teuthology.orchestra.run.smithi031.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-21T05:58:53.302 INFO:teuthology.orchestra.run.smithi031.stdout:(48/157): ceph-mgr-dashboard-19.3.0-5120.g18432 7.7 MB/s | 14 MB 00:01 2024-09-21T05:58:53.360 INFO:teuthology.orchestra.run.smithi031.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-21T05:58:53.386 INFO:teuthology.orchestra.run.smithi107.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-09-21T05:58:53.402 INFO:teuthology.orchestra.run.smithi031.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-21T05:58:53.428 INFO:teuthology.orchestra.run.smithi031.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 853 kB/s | 107 kB 00:00 2024-09-21T05:58:53.453 INFO:teuthology.orchestra.run.smithi031.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T05:58:53.487 INFO:teuthology.orchestra.run.smithi031.stdout:(53/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T05:58:53.512 INFO:teuthology.orchestra.run.smithi031.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-21T05:58:53.525 INFO:teuthology.orchestra.run.smithi110.stdout:(30/157): ceph-mgr-dashboard-19.3.0-5120.g18432 30 MB/s | 14 MB 00:00 2024-09-21T05:58:53.537 INFO:teuthology.orchestra.run.smithi031.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-21T05:58:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T05:58:53.587 INFO:teuthology.orchestra.run.smithi031.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T05:58:53.617 INFO:teuthology.orchestra.run.smithi110.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 4.2 MB/s | 3.1 MB 00:00 2024-09-21T05:58:53.621 INFO:teuthology.orchestra.run.smithi031.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-21T05:58:53.646 INFO:teuthology.orchestra.run.smithi031.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-21T05:58:53.705 INFO:teuthology.orchestra.run.smithi031.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-21T05:58:53.710 INFO:teuthology.orchestra.run.smithi110.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T05:58:53.730 INFO:teuthology.orchestra.run.smithi031.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-21T05:58:53.755 INFO:teuthology.orchestra.run.smithi031.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-21T05:58:53.768 INFO:teuthology.orchestra.run.smithi110.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 847 kB/s | 49 kB 00:00 2024-09-21T05:58:53.781 INFO:teuthology.orchestra.run.smithi031.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T05:58:53.819 INFO:teuthology.orchestra.run.smithi107.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-21T05:58:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-21T05:58:53.848 INFO:teuthology.orchestra.run.smithi031.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T05:58:53.852 INFO:teuthology.orchestra.run.smithi110.stdout:(34/157): ceph-mgr-diskprediction-local-19.3.0- 23 MB/s | 7.4 MB 00:00 2024-09-21T05:58:53.873 INFO:teuthology.orchestra.run.smithi031.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 761 kB/s | 38 kB 00:00 2024-09-21T05:58:53.877 INFO:teuthology.orchestra.run.smithi110.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 151 kB/s | 16 kB 00:00 2024-09-21T05:58:53.898 INFO:teuthology.orchestra.run.smithi031.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T05:58:53.903 INFO:teuthology.orchestra.run.smithi110.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T05:58:53.936 INFO:teuthology.orchestra.run.smithi110.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 555 kB/s | 18 kB 00:00 2024-09-21T05:58:53.994 INFO:teuthology.orchestra.run.smithi107.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-21T05:58:54.016 INFO:teuthology.orchestra.run.smithi031.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-21T05:58:54.038 INFO:teuthology.orchestra.run.smithi110.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.8 MB/s | 784 kB 00:00 2024-09-21T05:58:54.041 INFO:teuthology.orchestra.run.smithi031.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-21T05:58:54.062 INFO:teuthology.orchestra.run.smithi107.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T05:58:54.075 INFO:teuthology.orchestra.run.smithi031.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T05:58:54.108 INFO:teuthology.orchestra.run.smithi031.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T05:58:54.175 INFO:teuthology.orchestra.run.smithi031.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-21T05:58:54.217 INFO:teuthology.orchestra.run.smithi031.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-21T05:58:54.242 INFO:teuthology.orchestra.run.smithi031.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T05:58:54.334 INFO:teuthology.orchestra.run.smithi031.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-21T05:58:54.359 INFO:teuthology.orchestra.run.smithi031.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-21T05:58:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T05:58:54.410 INFO:teuthology.orchestra.run.smithi031.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T05:58:54.435 INFO:teuthology.orchestra.run.smithi031.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T05:58:54.460 INFO:teuthology.orchestra.run.smithi031.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-21T05:58:54.461 INFO:teuthology.orchestra.run.smithi107.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-21T05:58:54.486 INFO:teuthology.orchestra.run.smithi031.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T05:58:54.511 INFO:teuthology.orchestra.run.smithi031.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T05:58:54.528 INFO:teuthology.orchestra.run.smithi107.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-09-21T05:58:54.587 INFO:teuthology.orchestra.run.smithi107.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T05:58:54.611 INFO:teuthology.orchestra.run.smithi031.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T05:58:54.637 INFO:teuthology.orchestra.run.smithi031.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-21T05:58:54.670 INFO:teuthology.orchestra.run.smithi031.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T05:58:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:(39/157): ceph-common-19.3.0-5120.g18432218.el9 2.3 MB/s | 20 MB 00:08 2024-09-21T05:58:54.695 INFO:teuthology.orchestra.run.smithi031.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T05:58:54.712 INFO:teuthology.orchestra.run.smithi110.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 103 kB/s | 80 kB 00:00 2024-09-21T05:58:54.721 INFO:teuthology.orchestra.run.smithi031.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-21T05:58:54.746 INFO:teuthology.orchestra.run.smithi031.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T05:58:54.771 INFO:teuthology.orchestra.run.smithi031.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T05:58:54.796 INFO:teuthology.orchestra.run.smithi031.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T05:58:54.820 INFO:teuthology.orchestra.run.smithi107.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-21T05:58:54.821 INFO:teuthology.orchestra.run.smithi031.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T05:58:54.847 INFO:teuthology.orchestra.run.smithi031.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T05:58:54.862 INFO:teuthology.orchestra.run.smithi107.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T05:58:54.914 INFO:teuthology.orchestra.run.smithi031.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T05:58:54.939 INFO:teuthology.orchestra.run.smithi031.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-09-21T05:58:54.963 INFO:teuthology.orchestra.run.smithi107.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T05:58:54.964 INFO:teuthology.orchestra.run.smithi031.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-09-21T05:58:54.990 INFO:teuthology.orchestra.run.smithi031.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T05:58:55.015 INFO:teuthology.orchestra.run.smithi031.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-21T05:58:55.041 INFO:teuthology.orchestra.run.smithi031.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T05:58:55.045 INFO:teuthology.orchestra.run.smithi110.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 794 kB/s | 801 kB 00:01 2024-09-21T05:58:55.063 INFO:teuthology.orchestra.run.smithi107.stdout:(77/157): postgresql-server-13.16-1.el9.x86_64. 5.8 MB/s | 5.8 MB 00:01 2024-09-21T05:58:55.066 INFO:teuthology.orchestra.run.smithi031.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T05:58:55.091 INFO:teuthology.orchestra.run.smithi031.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T05:58:55.105 INFO:teuthology.orchestra.run.smithi107.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-21T05:58:55.116 INFO:teuthology.orchestra.run.smithi031.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T05:58:55.130 INFO:teuthology.orchestra.run.smithi107.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-09-21T05:58:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T05:58:55.156 INFO:teuthology.orchestra.run.smithi107.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-21T05:58:55.167 INFO:teuthology.orchestra.run.smithi031.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-21T05:58:55.181 INFO:teuthology.orchestra.run.smithi107.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T05:58:55.206 INFO:teuthology.orchestra.run.smithi107.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-21T05:58:55.250 INFO:teuthology.orchestra.run.smithi031.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-21T05:58:55.309 INFO:teuthology.orchestra.run.smithi031.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T05:58:55.323 INFO:teuthology.orchestra.run.smithi107.stdout:(83/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-21T05:58:55.342 INFO:teuthology.orchestra.run.smithi031.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T05:58:55.374 INFO:teuthology.orchestra.run.smithi107.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-09-21T05:58:55.376 INFO:teuthology.orchestra.run.smithi031.stdout:(107/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-21T05:58:55.399 INFO:teuthology.orchestra.run.smithi107.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-21T05:58:55.401 INFO:teuthology.orchestra.run.smithi031.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-21T05:58:55.416 INFO:teuthology.orchestra.run.smithi110.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 262 kB/s | 191 kB 00:00 2024-09-21T05:58:55.424 INFO:teuthology.orchestra.run.smithi107.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T05:58:55.435 INFO:teuthology.orchestra.run.smithi031.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T05:58:55.441 INFO:teuthology.orchestra.run.smithi107.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T05:58:55.460 INFO:teuthology.orchestra.run.smithi031.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-21T05:58:55.466 INFO:teuthology.orchestra.run.smithi110.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 336 kB/s | 253 kB 00:00 2024-09-21T05:58:55.467 INFO:teuthology.orchestra.run.smithi107.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-21T05:58:55.486 INFO:teuthology.orchestra.run.smithi031.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T05:58:55.492 INFO:teuthology.orchestra.run.smithi107.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T05:58:55.519 INFO:teuthology.orchestra.run.smithi031.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-21T05:58:55.525 INFO:teuthology.orchestra.run.smithi107.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-21T05:58:55.559 INFO:teuthology.orchestra.run.smithi107.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T05:58:55.570 INFO:teuthology.orchestra.run.smithi031.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-21T05:58:55.584 INFO:teuthology.orchestra.run.smithi107.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T05:58:55.626 INFO:teuthology.orchestra.run.smithi107.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T05:58:55.668 INFO:teuthology.orchestra.run.smithi107.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T05:58:55.737 INFO:teuthology.orchestra.run.smithi031.stdout:(114/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-21T05:58:55.787 INFO:teuthology.orchestra.run.smithi031.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-09-21T05:58:55.812 INFO:teuthology.orchestra.run.smithi031.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-21T05:58:55.829 INFO:teuthology.orchestra.run.smithi031.stdout:(117/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-09-21T05:58:55.854 INFO:teuthology.orchestra.run.smithi031.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 338 kB/s | 14 kB 00:00 2024-09-21T05:58:55.888 INFO:teuthology.orchestra.run.smithi031.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T05:58:55.902 INFO:teuthology.orchestra.run.smithi107.stdout:(95/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-09-21T05:58:55.913 INFO:teuthology.orchestra.run.smithi031.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T05:58:55.927 INFO:teuthology.orchestra.run.smithi107.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 853 kB/s | 221 kB 00:00 2024-09-21T05:58:55.946 INFO:teuthology.orchestra.run.smithi031.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-21T05:58:55.952 INFO:teuthology.orchestra.run.smithi107.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-21T05:58:55.978 INFO:teuthology.orchestra.run.smithi107.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T05:58:56.003 INFO:teuthology.orchestra.run.smithi107.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T05:58:56.013 INFO:teuthology.orchestra.run.smithi031.stdout:(122/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-21T05:58:56.028 INFO:teuthology.orchestra.run.smithi107.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T05:58:56.039 INFO:teuthology.orchestra.run.smithi031.stdout:(123/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-09-21T05:58:56.054 INFO:teuthology.orchestra.run.smithi107.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T05:58:56.064 INFO:teuthology.orchestra.run.smithi031.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T05:58:56.067 INFO:teuthology.orchestra.run.smithi110.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-21T05:58:56.079 INFO:teuthology.orchestra.run.smithi107.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T05:58:56.081 INFO:teuthology.orchestra.run.smithi031.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-09-21T05:58:56.092 INFO:teuthology.orchestra.run.smithi110.stdout:(45/157): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-21T05:58:56.097 INFO:teuthology.orchestra.run.smithi031.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T05:58:56.104 INFO:teuthology.orchestra.run.smithi107.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-21T05:58:56.114 INFO:teuthology.orchestra.run.smithi031.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 694 kB/s | 23 kB 00:00 2024-09-21T05:58:56.132 INFO:teuthology.orchestra.run.smithi031.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-21T05:58:56.149 INFO:teuthology.orchestra.run.smithi031.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-09-21T05:58:56.159 INFO:teuthology.orchestra.run.smithi110.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 195 kB/s | 135 kB 00:00 2024-09-21T05:58:56.166 INFO:teuthology.orchestra.run.smithi031.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-21T05:58:56.171 INFO:teuthology.orchestra.run.smithi107.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-21T05:58:56.191 INFO:teuthology.orchestra.run.smithi031.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:58:56.208 INFO:teuthology.orchestra.run.smithi031.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-21T05:58:56.230 INFO:teuthology.orchestra.run.smithi107.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T05:58:56.249 INFO:teuthology.orchestra.run.smithi031.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-09-21T05:58:56.255 INFO:teuthology.orchestra.run.smithi107.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T05:58:56.274 INFO:teuthology.orchestra.run.smithi031.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T05:58:56.280 INFO:teuthology.orchestra.run.smithi107.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-09-21T05:58:56.299 INFO:teuthology.orchestra.run.smithi031.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-21T05:58:56.306 INFO:teuthology.orchestra.run.smithi107.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-09-21T05:58:56.316 INFO:teuthology.orchestra.run.smithi031.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.6 MB/s | 28 kB 00:00 2024-09-21T05:58:56.342 INFO:teuthology.orchestra.run.smithi031.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-21T05:58:56.342 INFO:teuthology.orchestra.run.smithi110.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-21T05:58:56.348 INFO:teuthology.orchestra.run.smithi107.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-09-21T05:58:56.359 INFO:teuthology.orchestra.run.smithi031.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 732 kB/s | 31 kB 00:00 2024-09-21T05:58:56.373 INFO:teuthology.orchestra.run.smithi107.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T05:58:56.375 INFO:teuthology.orchestra.run.smithi031.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-21T05:58:56.401 INFO:teuthology.orchestra.run.smithi031.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-21T05:58:56.407 INFO:teuthology.orchestra.run.smithi107.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T05:58:56.426 INFO:teuthology.orchestra.run.smithi110.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 13 MB/s | 943 kB 00:00 2024-09-21T05:58:56.427 INFO:teuthology.orchestra.run.smithi031.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-21T05:58:56.452 INFO:teuthology.orchestra.run.smithi031.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T05:58:56.468 INFO:teuthology.orchestra.run.smithi110.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-09-21T05:58:56.491 INFO:teuthology.orchestra.run.smithi107.stdout:(112/157): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-21T05:58:56.502 INFO:teuthology.orchestra.run.smithi031.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-21T05:58:56.510 INFO:teuthology.orchestra.run.smithi110.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 521 kB/s | 22 kB 00:00 2024-09-21T05:58:56.516 INFO:teuthology.orchestra.run.smithi107.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-21T05:58:56.527 INFO:teuthology.orchestra.run.smithi031.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-21T05:58:56.550 INFO:teuthology.orchestra.run.smithi107.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T05:58:56.552 INFO:teuthology.orchestra.run.smithi110.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-09-21T05:58:56.561 INFO:teuthology.orchestra.run.smithi031.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-21T05:58:56.575 INFO:teuthology.orchestra.run.smithi107.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T05:58:56.586 INFO:teuthology.orchestra.run.smithi031.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 343 kB/s | 20 kB 00:00 2024-09-21T05:58:56.600 INFO:teuthology.orchestra.run.smithi107.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:58:56.611 INFO:teuthology.orchestra.run.smithi031.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T05:58:56.619 INFO:teuthology.orchestra.run.smithi110.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-21T05:58:56.625 INFO:teuthology.orchestra.run.smithi107.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 569 kB/s | 14 kB 00:00 2024-09-21T05:58:56.645 INFO:teuthology.orchestra.run.smithi031.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-21T05:58:56.697 INFO:teuthology.orchestra.run.smithi031.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T05:58:56.701 INFO:teuthology.orchestra.run.smithi107.stdout:(118/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-09-21T05:58:56.702 INFO:teuthology.orchestra.run.smithi110.stdout:(53/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 36 MB/s | 3.0 MB 00:00 2024-09-21T05:58:56.735 INFO:teuthology.orchestra.run.smithi107.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-09-21T05:58:56.745 INFO:teuthology.orchestra.run.smithi110.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-21T05:58:56.768 INFO:teuthology.orchestra.run.smithi107.stdout:(120/157): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-09-21T05:58:56.787 INFO:teuthology.orchestra.run.smithi110.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-21T05:58:56.793 INFO:teuthology.orchestra.run.smithi107.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.2 MB/s | 303 kB 00:00 2024-09-21T05:58:56.819 INFO:teuthology.orchestra.run.smithi107.stdout:(122/157): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-21T05:58:56.828 INFO:teuthology.orchestra.run.smithi110.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 4.0 MB/s | 171 kB 00:00 2024-09-21T05:58:56.844 INFO:teuthology.orchestra.run.smithi107.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-21T05:58:56.854 INFO:teuthology.orchestra.run.smithi110.stdout:(57/157): python3-requests-2.25.1-8.el9.noarch. 159 kB/s | 125 kB 00:00 2024-09-21T05:58:56.869 INFO:teuthology.orchestra.run.smithi107.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T05:58:56.879 INFO:teuthology.orchestra.run.smithi110.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T05:58:56.894 INFO:teuthology.orchestra.run.smithi107.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-21T05:58:56.920 INFO:teuthology.orchestra.run.smithi107.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-21T05:58:56.929 INFO:teuthology.orchestra.run.smithi110.stdout:(59/157): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-09-21T05:58:56.937 INFO:teuthology.orchestra.run.smithi031.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 10 MB/s | 3.4 MB 00:00 2024-09-21T05:58:56.945 INFO:teuthology.orchestra.run.smithi107.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-21T05:58:56.955 INFO:teuthology.orchestra.run.smithi110.stdout:(60/157): python3-urllib3-1.26.5-6.el9.noarch.r 249 kB/s | 215 kB 00:00 2024-09-21T05:58:56.970 INFO:teuthology.orchestra.run.smithi107.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-09-21T05:58:56.980 INFO:teuthology.orchestra.run.smithi110.stdout:(61/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 360 kB/s | 45 kB 00:00 2024-09-21T05:58:56.995 INFO:teuthology.orchestra.run.smithi107.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-21T05:58:57.005 INFO:teuthology.orchestra.run.smithi110.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-21T05:58:57.021 INFO:teuthology.orchestra.run.smithi107.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T05:58:57.029 INFO:teuthology.orchestra.run.smithi031.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.3 MB/s | 3.2 MB 00:00 2024-09-21T05:58:57.031 INFO:teuthology.orchestra.run.smithi110.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-21T05:58:57.046 INFO:teuthology.orchestra.run.smithi107.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:58:57.056 INFO:teuthology.orchestra.run.smithi110.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T05:58:57.071 INFO:teuthology.orchestra.run.smithi107.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T05:58:57.081 INFO:teuthology.orchestra.run.smithi110.stdout:(65/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-21T05:58:57.106 INFO:teuthology.orchestra.run.smithi107.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-21T05:58:57.106 INFO:teuthology.orchestra.run.smithi110.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-21T05:58:57.121 INFO:teuthology.orchestra.run.smithi031.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 446 kB/s | 41 kB 00:00 2024-09-21T05:58:57.130 INFO:teuthology.orchestra.run.smithi107.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-21T05:58:57.131 INFO:teuthology.orchestra.run.smithi110.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-21T05:58:57.155 INFO:teuthology.orchestra.run.smithi107.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-21T05:58:57.181 INFO:teuthology.orchestra.run.smithi107.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T05:58:57.204 INFO:teuthology.orchestra.run.smithi031.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.9 MB/s | 249 kB 00:00 2024-09-21T05:58:57.206 INFO:teuthology.orchestra.run.smithi107.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T05:58:57.215 INFO:teuthology.orchestra.run.smithi110.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T05:58:57.232 INFO:teuthology.orchestra.run.smithi107.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-21T05:58:57.248 INFO:teuthology.orchestra.run.smithi107.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-09-21T05:58:57.266 INFO:teuthology.orchestra.run.smithi110.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 18 MB/s | 910 kB 00:00 2024-09-21T05:58:57.274 INFO:teuthology.orchestra.run.smithi107.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-21T05:58:57.299 INFO:teuthology.orchestra.run.smithi107.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-21T05:58:57.324 INFO:teuthology.orchestra.run.smithi107.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T05:58:57.329 INFO:teuthology.orchestra.run.smithi031.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 3.7 MB/s | 2.3 MB 00:00 2024-09-21T05:58:57.358 INFO:teuthology.orchestra.run.smithi110.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-21T05:58:57.366 INFO:teuthology.orchestra.run.smithi107.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-21T05:58:57.383 INFO:teuthology.orchestra.run.smithi110.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T05:58:57.392 INFO:teuthology.orchestra.run.smithi107.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T05:58:57.417 INFO:teuthology.orchestra.run.smithi107.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-21T05:58:57.434 INFO:teuthology.orchestra.run.smithi107.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-21T05:58:57.450 INFO:teuthology.orchestra.run.smithi110.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-21T05:58:57.459 INFO:teuthology.orchestra.run.smithi107.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T05:58:57.480 INFO:teuthology.orchestra.run.smithi031.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-09-21T05:58:57.483 INFO:teuthology.orchestra.run.smithi110.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-21T05:58:57.509 INFO:teuthology.orchestra.run.smithi110.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T05:58:57.530 INFO:teuthology.orchestra.run.smithi031.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 4.2 MB/s | 1.4 MB 00:00 2024-09-21T05:58:57.543 INFO:teuthology.orchestra.run.smithi110.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-21T05:58:57.568 INFO:teuthology.orchestra.run.smithi107.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-09-21T05:58:57.618 INFO:teuthology.orchestra.run.smithi110.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-21T05:58:57.618 INFO:teuthology.orchestra.run.smithi107.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-21T05:58:57.643 INFO:teuthology.orchestra.run.smithi110.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T05:58:57.668 INFO:teuthology.orchestra.run.smithi110.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T05:58:57.693 INFO:teuthology.orchestra.run.smithi110.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T05:58:57.727 INFO:teuthology.orchestra.run.smithi110.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-09-21T05:58:57.753 INFO:teuthology.orchestra.run.smithi110.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T05:58:57.778 INFO:teuthology.orchestra.run.smithi110.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-21T05:58:57.810 INFO:teuthology.orchestra.run.smithi107.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-21T05:58:57.853 INFO:teuthology.orchestra.run.smithi110.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-21T05:58:57.869 INFO:teuthology.orchestra.run.smithi107.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 31 MB/s | 1.8 MB 00:00 2024-09-21T05:58:57.879 INFO:teuthology.orchestra.run.smithi110.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T05:58:57.903 INFO:teuthology.orchestra.run.smithi107.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-09-21T05:58:57.904 INFO:teuthology.orchestra.run.smithi110.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T05:58:57.929 INFO:teuthology.orchestra.run.smithi110.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T05:58:57.945 INFO:teuthology.orchestra.run.smithi107.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.9 MB/s | 249 kB 00:00 2024-09-21T05:58:57.954 INFO:teuthology.orchestra.run.smithi110.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T05:58:57.979 INFO:teuthology.orchestra.run.smithi110.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T05:58:58.003 INFO:teuthology.orchestra.run.smithi107.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-21T05:58:58.005 INFO:teuthology.orchestra.run.smithi110.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T05:58:58.030 INFO:teuthology.orchestra.run.smithi110.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T05:58:58.055 INFO:teuthology.orchestra.run.smithi031.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 2.4 MB 00:00 2024-09-21T05:58:58.056 INFO:teuthology.orchestra.run.smithi110.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T05:58:58.062 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:58:58.062 INFO:teuthology.orchestra.run.smithi031.stdout:Total 17 MB/s | 228 MB 00:13 2024-09-21T05:58:58.081 INFO:teuthology.orchestra.run.smithi110.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T05:58:58.087 INFO:teuthology.orchestra.run.smithi107.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-21T05:58:58.145 INFO:teuthology.orchestra.run.smithi107.stdout:(156/157): librados2-19.3.0-5120.g18432218.el9. 4.9 MB/s | 3.4 MB 00:00 2024-09-21T05:58:58.181 INFO:teuthology.orchestra.run.smithi110.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T05:58:58.206 INFO:teuthology.orchestra.run.smithi110.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 42 kB 00:00 2024-09-21T05:58:58.229 INFO:teuthology.orchestra.run.smithi107.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 4.8 MB/s | 3.2 MB 00:00 2024-09-21T05:58:58.231 INFO:teuthology.orchestra.run.smithi110.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-21T05:58:58.235 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:58:58.235 INFO:teuthology.orchestra.run.smithi107.stdout:Total 13 MB/s | 228 MB 00:17 2024-09-21T05:58:58.257 INFO:teuthology.orchestra.run.smithi110.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T05:58:58.282 INFO:teuthology.orchestra.run.smithi110.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-21T05:58:58.307 INFO:teuthology.orchestra.run.smithi110.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-21T05:58:58.333 INFO:teuthology.orchestra.run.smithi110.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T05:58:58.358 INFO:teuthology.orchestra.run.smithi110.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T05:58:58.375 INFO:teuthology.orchestra.run.smithi110.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-21T05:58:58.400 INFO:teuthology.orchestra.run.smithi110.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T05:58:58.426 INFO:teuthology.orchestra.run.smithi110.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-21T05:58:58.451 INFO:teuthology.orchestra.run.smithi110.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-21T05:58:58.493 INFO:teuthology.orchestra.run.smithi110.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-21T05:58:58.560 INFO:teuthology.orchestra.run.smithi110.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.7 MB/s | 308 kB 00:00 2024-09-21T05:58:58.593 INFO:teuthology.orchestra.run.smithi110.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.8 MB/s | 548 kB 00:00 2024-09-21T05:58:58.619 INFO:teuthology.orchestra.run.smithi110.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-21T05:58:58.644 INFO:teuthology.orchestra.run.smithi110.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T05:58:58.669 INFO:teuthology.orchestra.run.smithi110.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T05:58:58.695 INFO:teuthology.orchestra.run.smithi110.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T05:58:58.720 INFO:teuthology.orchestra.run.smithi110.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T05:58:58.887 INFO:teuthology.orchestra.run.smithi110.stdout:(113/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-21T05:58:58.929 INFO:teuthology.orchestra.run.smithi110.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-21T05:58:58.963 INFO:teuthology.orchestra.run.smithi110.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.4 MB/s | 43 kB 00:00 2024-09-21T05:58:58.988 INFO:teuthology.orchestra.run.smithi110.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:58:59.005 INFO:teuthology.orchestra.run.smithi110.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 851 kB/s | 14 kB 00:00 2024-09-21T05:58:59.030 INFO:teuthology.orchestra.run.smithi110.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T05:58:59.064 INFO:teuthology.orchestra.run.smithi110.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-09-21T05:58:59.098 INFO:teuthology.orchestra.run.smithi110.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T05:58:59.127 INFO:teuthology.orchestra.run.smithi110.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T05:58:59.157 INFO:teuthology.orchestra.run.smithi110.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T05:58:59.190 INFO:teuthology.orchestra.run.smithi110.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T05:58:59.215 INFO:teuthology.orchestra.run.smithi110.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-21T05:58:59.283 INFO:teuthology.orchestra.run.smithi110.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-21T05:58:59.308 INFO:teuthology.orchestra.run.smithi110.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-09-21T05:58:59.332 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-21T05:58:59.333 INFO:teuthology.orchestra.run.smithi110.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 941 kB/s | 23 kB 00:00 2024-09-21T05:58:59.358 INFO:teuthology.orchestra.run.smithi110.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 390 kB/s | 19 kB 00:00 2024-09-21T05:58:59.383 INFO:teuthology.orchestra.run.smithi110.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 795 kB/s | 20 kB 00:00 2024-09-21T05:58:59.409 INFO:teuthology.orchestra.run.smithi110.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T05:58:59.426 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-21T05:58:59.434 INFO:teuthology.orchestra.run.smithi110.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:58:59.467 INFO:teuthology.orchestra.run.smithi110.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-21T05:58:59.500 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-21T05:58:59.500 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-21T05:58:59.501 INFO:teuthology.orchestra.run.smithi110.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T05:58:59.527 INFO:teuthology.orchestra.run.smithi110.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-21T05:58:59.562 INFO:teuthology.orchestra.run.smithi110.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-21T05:58:59.576 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-21T05:58:59.576 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-21T05:58:59.586 INFO:teuthology.orchestra.run.smithi110.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T05:58:59.611 INFO:teuthology.orchestra.run.smithi110.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-21T05:58:59.636 INFO:teuthology.orchestra.run.smithi110.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-21T05:58:59.687 INFO:teuthology.orchestra.run.smithi110.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-21T05:58:59.712 INFO:teuthology.orchestra.run.smithi110.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-09-21T05:58:59.737 INFO:teuthology.orchestra.run.smithi110.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T05:58:59.762 INFO:teuthology.orchestra.run.smithi110.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T05:58:59.787 INFO:teuthology.orchestra.run.smithi110.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 989 kB/s | 74 kB 00:00 2024-09-21T05:58:59.813 INFO:teuthology.orchestra.run.smithi110.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T05:58:59.838 INFO:teuthology.orchestra.run.smithi110.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-21T05:58:59.855 INFO:teuthology.orchestra.run.smithi110.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-09-21T05:58:59.880 INFO:teuthology.orchestra.run.smithi110.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T05:58:59.906 INFO:teuthology.orchestra.run.smithi110.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-21T05:58:59.956 INFO:teuthology.orchestra.run.smithi110.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T05:59:00.156 INFO:teuthology.orchestra.run.smithi110.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T05:59:00.515 INFO:teuthology.orchestra.run.smithi110.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 5.2 MB/s | 3.2 MB 00:00 2024-09-21T05:59:00.957 INFO:teuthology.orchestra.run.smithi110.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-09-21T05:59:01.091 INFO:teuthology.orchestra.run.smithi110.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 71 kB/s | 41 kB 00:00 2024-09-21T05:59:01.291 INFO:teuthology.orchestra.run.smithi110.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.6 MB/s | 1.8 MB 00:01 2024-09-21T05:59:01.385 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-21T05:59:01.385 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-21T05:59:01.708 INFO:teuthology.orchestra.run.smithi110.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 332 kB/s | 249 kB 00:00 2024-09-21T05:59:01.826 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-21T05:59:01.826 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-21T05:59:02.041 INFO:teuthology.orchestra.run.smithi110.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-21T05:59:02.300 INFO:teuthology.orchestra.run.smithi110.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-09-21T05:59:02.307 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:59:02.307 INFO:teuthology.orchestra.run.smithi110.stdout:Total 14 MB/s | 228 MB 00:16 2024-09-21T05:59:03.556 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-21T05:59:03.709 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-21T05:59:03.709 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-21T05:59:05.588 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-21T05:59:05.588 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-21T05:59:07.885 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-21T05:59:07.982 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:59:08.020 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:59:08.082 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:59:08.116 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:59:08.142 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:59:08.184 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:59:08.515 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:59:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-21T05:59:08.642 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:59:08.643 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:08.684 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:59:08.751 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:08.752 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:59:08.777 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:08.794 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:59:08.827 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:59:08.852 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:08.882 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:59:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:59:08.911 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:59:09.062 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:59:09.113 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:59:09.141 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:09.247 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:59:09.278 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:59:09.347 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:59:09.368 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:59:09.388 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:59:09.394 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:09.524 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:09.544 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:59:09.554 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:09.675 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:59:09.681 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:59:09.705 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:59:09.819 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:59:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:59:09.920 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:09.931 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:59:09.967 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:10.002 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:10.082 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:10.118 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:59:10.181 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:59:10.219 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:59:10.240 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:59:10.247 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:10.422 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:59:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:59:10.475 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:59:10.525 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:59:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:59:10.638 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:59:10.698 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:59:10.740 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:59:10.759 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:59:10.798 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:59:10.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:59:10.875 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:59:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:10.968 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:10.985 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:59:11.052 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:59:11.085 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:59:11.121 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:59:11.200 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:59:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:11.231 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:59:11.274 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:59:11.294 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:59:11.319 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:59:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:59:11.363 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:59:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:59:11.409 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:59:11.467 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:59:11.483 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:59:11.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:59:11.566 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:59:11.609 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:59:11.614 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:59:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:59:11.661 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:59:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:59:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:59:11.793 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:59:11.822 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:59:11.873 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:59:11.967 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:59:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:59:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:59:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:59:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:59:12.185 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:59:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:59:12.216 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-21T05:59:12.265 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:59:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:59:12.295 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:59:12.304 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:59:12.329 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:59:12.342 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:59:12.368 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:59:12.404 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:59:12.409 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:59:12.429 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:59:12.438 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:59:12.463 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:59:12.484 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:59:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:59:12.567 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:59:12.621 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:59:12.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:59:12.777 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:59:12.848 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:59:12.854 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:59:12.886 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:59:12.921 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:59:12.964 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:12.968 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:59:13.004 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:59:13.047 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:59:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:59:13.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:59:13.090 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:13.124 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:59:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:59:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:13.223 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:59:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:59:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:59:13.267 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:59:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:59:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:59:13.309 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:59:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:59:13.425 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:59:13.476 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:59:13.501 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:59:13.504 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:59:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:59:13.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:59:13.610 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:13.632 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:59:13.639 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:59:13.657 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:59:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:59:13.682 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:59:13.701 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:59:13.723 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:59:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:59:13.743 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:59:13.756 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:59:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:59:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:59:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:59:13.883 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:59:13.892 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:59:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:59:13.931 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:59:13.951 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:59:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:59:13.985 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:59:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:59:14.025 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:59:14.030 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:59:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:59:14.056 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:59:14.086 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:59:14.121 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:59:14.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:59:14.163 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:59:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:59:14.196 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:59:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:59:14.257 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:14.268 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:59:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:59:14.340 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:14.350 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:59:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:59:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:59:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:59:14.433 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:59:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:59:14.491 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:59:14.516 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:59:14.536 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:59:14.573 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:59:14.584 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:59:14.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:59:14.687 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:59:14.719 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:59:14.749 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:59:14.755 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:59:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:59:14.827 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:59:14.872 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:59:14.912 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:59:14.956 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:59:14.988 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:59:15.016 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:59:15.089 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:59:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:59:15.142 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:59:15.173 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:59:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:59:15.208 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:59:15.236 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:59:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:59:15.296 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:59:15.297 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:59:15.304 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:59:15.327 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:59:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:59:15.340 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:59:15.371 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:59:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:59:15.374 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:59:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:59:15.416 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:59:15.448 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:59:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:59:15.475 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:59:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:59:15.529 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:59:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:59:15.574 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:59:15.588 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:59:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:59:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:59:15.660 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:59:15.693 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:59:15.721 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:59:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:59:15.792 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:59:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:59:15.877 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:59:15.908 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:59:15.985 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:59:16.313 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:59:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:59:16.425 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:59:16.459 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:59:16.498 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:59:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:59:16.625 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:59:16.685 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:59:16.730 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:59:16.734 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:59:16.766 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:59:16.816 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:59:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:59:16.849 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:59:16.891 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:59:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:59:16.965 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:59:16.994 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:59:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:59:17.028 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:59:17.059 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:59:17.075 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:59:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:59:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:59:17.173 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:59:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:59:17.212 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:59:17.348 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:59:17.384 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:59:17.390 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:59:17.416 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:59:17.431 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:59:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:59:17.533 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:59:17.565 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:59:17.604 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:59:17.630 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:59:17.655 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:59:17.722 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:59:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:59:17.936 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:59:17.996 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:59:18.005 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:59:18.024 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:59:18.057 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:59:18.058 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:59:18.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:59:18.098 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:59:18.129 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:59:18.159 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:59:18.185 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:59:18.194 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:59:18.227 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:59:18.268 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:59:18.302 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:59:18.303 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:59:18.390 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:59:18.414 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:59:18.433 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:59:18.464 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:59:18.479 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:59:18.553 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:59:18.567 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:59:18.583 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:59:18.615 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:59:18.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:59:19.116 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:59:19.142 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:59:19.212 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:59:19.235 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:59:19.333 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:59:19.380 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:59:19.414 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:59:19.511 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:59:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:59:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:59:19.648 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:59:19.691 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:59:19.744 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:59:19.749 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:59:19.772 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:59:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:59:19.933 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:59:19.964 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:59:20.041 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:59:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:59:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:59:20.301 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:59:20.331 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:59:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:59:20.685 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:59:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:59:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:59:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:59:21.017 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:59:21.046 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:59:21.523 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:21.557 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:59:22.111 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:59:22.122 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:22.139 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:59:22.161 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:59:22.172 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:59:22.252 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:59:22.252 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:59:22.252 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:59:22.253 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:22.279 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:59:22.503 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:59:22.543 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:59:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:59:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:59:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:59:23.215 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:59:23.246 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:59:23.317 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:59:23.348 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:59:23.384 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:23.426 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:59:24.165 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:59:24.166 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:59:24.166 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:59:24.166 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:24.214 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:59:25.590 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:25.645 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:59:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:59:26.348 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:59:26.348 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:59:26.348 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:59:26.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:26.375 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-21T05:59:34.470 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:34.643 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:59:34.659 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:59:34.660 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:34.660 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:59:34.660 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:36.222 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:59:36.247 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:59:36.248 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:36.248 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:59:36.248 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:59:36.248 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:59:36.248 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:36.479 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:59:36.502 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:59:36.503 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:36.503 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:59:36.503 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:59:36.503 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:59:36.503 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-21T05:59:36.518 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-21T05:59:36.519 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:36.698 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:59:36.715 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:59:36.715 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:36.715 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:59:36.716 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:36.963 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:59:36.987 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:59:36.987 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:36.987 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:59:36.987 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:59:36.987 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:59:36.987 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:37.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:59:37.151 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:59:37.206 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:59:37.389 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:59:37.448 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:59:37.525 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:59:37.614 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:59:37.753 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:59:37.798 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:59:37.830 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:59:37.866 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:59:38.293 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:59:38.324 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:59:38.347 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-09-21T05:59:38.355 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:59:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:59:38.398 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:38.398 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:59:38.398 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:59:38.398 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:59:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:38.404 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:59:38.430 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:59:38.531 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:59:38.548 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:59:38.548 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:38.548 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:59:38.548 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:38.644 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:59:38.669 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:59:38.669 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:38.669 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:59:38.669 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:59:38.670 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:59:38.670 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:59:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:59:39.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:59:39.438 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:59:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:59:39.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:59:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:59:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:59:40.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:59:40.070 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:59:40.102 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:59:40.130 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:59:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:59:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:59:40.161 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:40.161 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:59:40.162 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:59:40.162 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:59:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:59:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:59:40.417 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:40.417 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:59:40.417 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:59:40.417 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:59:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:59:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:59:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:59:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:59:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:59:40.886 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:59:40.909 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:59:40.909 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:40.909 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:59:40.909 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:59:40.910 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:59:40.910 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:41.016 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:59:41.072 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:59:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:59:41.312 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:59:41.353 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:59:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:59:41.520 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:59:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:59:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:59:41.658 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:59:41.703 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:59:41.735 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:59:41.763 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:59:42.149 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:59:42.179 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:59:42.211 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:59:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:59:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:59:42.582 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:59:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:59:42.725 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:59:42.886 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:59:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:59:42.937 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:59:42.937 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:42.937 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:59:42.937 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:59:42.938 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:59:42.938 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:42.981 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:59:42.992 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:59:43.092 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:59:43.137 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:59:44.078 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:59:44.097 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:59:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:59:44.142 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:59:44.165 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:59:44.166 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:44.166 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:59:44.166 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:59:44.166 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:59:44.166 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:44.209 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:59:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:59:44.232 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:44.232 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:59:44.232 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:44.276 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:59:44.299 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:59:44.299 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:44.299 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:59:44.299 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:59:44.299 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:59:44.299 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:44.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:59:44.642 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:59:44.642 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:44.642 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:59:44.642 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:59:44.642 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:59:44.642 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T05:59:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:59:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:59:45.304 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:59:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:59:45.501 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:59:45.524 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:59:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:59:45.527 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:45.527 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:59:45.527 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:59:45.527 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:59:45.527 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:45.577 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:59:45.584 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:59:45.589 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:59:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:59:45.756 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:59:46.552 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:59:46.584 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:59:46.700 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:59:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:59:46.812 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:59:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:59:46.838 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:46.838 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:59:46.838 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:59:46.838 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:59:46.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:46.860 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:59:46.879 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:59:46.894 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:59:46.904 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:59:46.904 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:46.904 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:59:46.904 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:46.919 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:59:46.919 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:46.919 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:59:46.919 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:59:46.919 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:59:46.919 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:46.955 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:59:46.970 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:59:46.980 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:59:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:46.980 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:59:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:59:46.981 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:59:46.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:46.982 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:59:47.081 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:59:47.126 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:59:47.314 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:59:47.339 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:59:47.339 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:47.339 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:59:47.339 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:59:47.339 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:59:47.339 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T05:59:48.106 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:59:48.140 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:59:48.164 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:59:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:59:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:59:48.164 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:59:48.164 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:48.207 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:59:48.230 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:59:48.230 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:48.230 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:59:48.230 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:59:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:59:48.306 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:48.306 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:59:48.306 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:59:48.307 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:59:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:48.607 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:59:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:59:48.648 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:59:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:59:48.665 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:59:48.665 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:59:48.665 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:59:48.665 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:59:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T05:59:48.688 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:59:48.721 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:59:48.748 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:59:48.927 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:59:48.970 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:59:49.000 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:59:49.030 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:59:49.076 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:59:49.636 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:49.683 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:49.703 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:50.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:59:50.597 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:59:50.637 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:59:50.718 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:59:50.759 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:59:50.784 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:59:50.810 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:59:50.810 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:59:50.886 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:59:50.886 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:59:50.970 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:59:50.995 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:59:51.000 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:51.000 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:59:51.256 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:59:51.318 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:59:51.605 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:59:51.660 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:59:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:59:51.732 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:59:51.759 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:59:51.988 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:59:52.047 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:59:52.078 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:59:52.107 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:59:52.145 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:59:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:59:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:59:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:59:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:59:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:59:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:59:53.043 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:59:53.073 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:59:53.103 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:59:53.141 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:59:53.711 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:53.747 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:59:53.801 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:59:53.905 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:59:53.950 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:59:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:59:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:59:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:59:54.134 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:59:54.135 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:59:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:59:54.226 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:59:54.319 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:59:54.352 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:59:54.358 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:54.358 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:59:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:59:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:59:54.703 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:59:54.706 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:59:54.743 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:59:54.825 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:59:54.873 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:59:54.898 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:59:54.924 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:59:54.924 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:59:55.017 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:59:55.017 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:59:55.084 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:59:55.109 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:59:55.115 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:55.115 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:55.294 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:59:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:59:55.379 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:59:55.442 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:59:57.793 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:59:57.793 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T05:59:57.793 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T05:59:57.793 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T05:59:57.793 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T05:59:57.794 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T05:59:57.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T05:59:57.796 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T05:59:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T05:59:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T05:59:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T05:59:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T05:59:57.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T05:59:57.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T05:59:57.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T05:59:57.810 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T05:59:57.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T05:59:57.812 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T05:59:57.813 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T05:59:57.814 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T05:59:57.815 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T05:59:57.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.254 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.255 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.256 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:00:00.257 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:00:00.258 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:00:00.259 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:00:00.260 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:00:00.261 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:00:00.262 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:00:00.263 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:00:00.264 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:00:00.265 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:00:00.266 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:00:00.267 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:00:00.267 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T06:00:00.267 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-21T06:00:00.723 DEBUG:teuthology.parallel:result is None 2024-09-21T06:00:00.738 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:00:00.739 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:00:00.740 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:00:00.741 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:00:00.742 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:00:00.743 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:00:00.744 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:00:00.745 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:00:00.746 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:00:00.747 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:00:00.748 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:00:00.749 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:00:00.750 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:00:00.751 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:00:00.752 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:00:00.753 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:00:01.872 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:00:01.873 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:00:01.874 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:00:01.875 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:00:01.875 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:00:01.875 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:00:01.885 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:00:01.886 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:00:01.887 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:00:01.888 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:00:01.889 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:00:01.890 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:00:01.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:00:01.892 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:00:01.893 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:00:01.894 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:00:01.895 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:00:01.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:00:01.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:00:01.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:00:01.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:00:01.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:00:01.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:03.540 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.541 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.542 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:00:03.543 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:00:03.544 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:00:03.545 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:00:03.546 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:00:03.547 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:00:03.548 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:00:03.549 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:00:03.550 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:00:03.551 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:00:03.552 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:00:03.553 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:00:03.553 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:00:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-21T06:00:03.997 DEBUG:teuthology.parallel:result is None 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:00:04.470 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.471 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.472 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.473 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:00:04.474 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.475 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:00:04.476 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:00:04.477 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:00:04.478 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:00:04.479 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:00:04.480 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:00:04.481 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:00:04.482 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.483 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:00:04.484 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:00:04.485 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-21T06:00:04.901 DEBUG:teuthology.parallel:result is None 2024-09-21T06:00:04.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:00:05.053 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:00:05.080 INFO:teuthology.orchestra.run.smithi031.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:00:05.081 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:00:05.081 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:00:05.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:00:05.226 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:00:05.252 INFO:teuthology.orchestra.run.smithi107.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:00:05.252 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:00:05.252 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:00:05.254 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:00:05.391 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:00:05.415 INFO:teuthology.orchestra.run.smithi110.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:00:05.415 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:00:05.415 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:00:05.417 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T06:00:05.417 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:00:05.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:00:05.450 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:00:05.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:00:05.483 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:00:05.483 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:00:05.511 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T06:00:05.512 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:00:05.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:00:05.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:00:05.609 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:00:05.610 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:00:05.639 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:00:05.709 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:00:05.709 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:00:05.736 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:00:05.801 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T06:00:05.802 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:00:05.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:00:05.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:00:05.899 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:00:05.899 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:00:05.928 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:00:05.995 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:00:05.995 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:00:06.021 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:00:06.087 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T06:00:06.087 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:00:06.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:00:06.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:00:06.185 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:00:06.185 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:00:06.212 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:00:06.280 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:00:06.280 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:00:06.305 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:00:06.370 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T06:00:06.464 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T06:00:06.464 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:00:06.465 INFO:tasks.cephadm:Cluster fsid is c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:00:06.465 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T06:00:06.465 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.107', 'mon.c': '172.21.15.110'} 2024-09-21T06:00:06.465 INFO:tasks.cephadm:First mon is mon.a on smithi031 2024-09-21T06:00:06.465 INFO:tasks.cephadm:First mgr is x 2024-09-21T06:00:06.465 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T06:00:06.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-09-21T06:00:06.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-09-21T06:00:06.522 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-09-21T06:00:06.548 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T06:00:06.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:00:06.685 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T06:00:06.821 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:00:06.823 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:00:06.823 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:00:06.823 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:07.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:00 /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:07.365 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:07.911 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:00 /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:07.911 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:08.467 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:00 /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:08.467 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:08.487 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:08.507 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:00:08.541 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T06:00:08.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:00:08.545 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:00:08.550 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:00:10.049 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:00:10.169 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:00:10.212 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-21T06:00:42.549 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:00:49.886 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T06:00:49.886 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:00:49.886 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:00:49.886 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-09-21T06:00:49.887 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:00:49.887 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-21T06:00:49.887 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T06:00:49.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-09-21T06:00:49.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-09-21T06:00:49.978 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-09-21T06:00:50.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-09-21T06:00:50.035 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-09-21T06:00:50.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-09-21T06:00:50.089 INFO:tasks.cephadm:Writing seed config... 2024-09-21T06:00:50.090 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T06:00:50.090 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T06:00:50.090 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T06:00:50.090 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T06:00:50.090 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T06:00:50.091 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T06:00:50.092 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T06:00:50.093 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:00:50.093 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T06:00:50.111 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c04f7042-77de-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T06:00:50.111 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T06:00:50.153 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T06:00:50.196 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T06:00:50.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid c04f7042-77de-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:00:50.413 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:00:50.413 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-09-21T06:00:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T06:00:50.413 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-09-21T06:00:50.438 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:00:50.438 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-09-21T06:00:50.438 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-09-21T06:00:50.447 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:00:50.447 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:00:50.453 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:00:50.453 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T06:00:50.460 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-21T06:00:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-21T06:00:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:00:50.467 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-09-21T06:00:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:00:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T06:00:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-09-21T06:00:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-09-21T06:00:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:00:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:00:50.504 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:00:50.504 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T06:00:50.510 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-21T06:00:50.517 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-21T06:00:50.517 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:00:50.517 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-09-21T06:00:50.517 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:00:50.517 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140314454870384 on /run/cephadm/c04f7042-77de-11ef-baf5-efdc52797490.lock 2024-09-21T06:00:50.518 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140314454870384 acquired on /run/cephadm/c04f7042-77de-11ef-baf5-efdc52797490.lock 2024-09-21T06:00:50.518 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-09-21T06:00:50.518 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-09-21T06:00:50.518 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-21T06:00:50.521 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-09-21T06:00:50.522 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-09-21T06:00:50.524 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T06:00:50.524 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T06:00:50.524 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-09-21T06:00:50.527 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T06:00:50.527 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T06:00:50.527 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:00:50.527 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T06:00:50.527 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-21T06:00:50.527 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:00:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-21T06:00:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-21T06:00:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T06:00:50.528 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T06:00:50.529 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:00:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:00:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:00:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T06:00:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T06:00:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T06:00:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:00:51.008 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T06:00:51.946 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:00:51.946 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:00:51.946 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T06:01:00.031 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-09-21T06:01:00.031 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-09-21T06:01:06.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAeYe5mF3n5JhAAz6yRF0lKshX2Eel17B7wSQ== 2024-09-21T06:01:07.539 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAiYe5mZ9uiLRAATHxEwtKiwz9sCsIEi1NUYg== 2024-09-21T06:01:08.470 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQAjYe5m0xxFMxAAo78FrKCW9ihRJgB+L0a1Jw== 2024-09-21T06:01:08.471 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:01:09.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:01:09.327 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-09-21T06:01:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2024-09-21T06:01:10.461 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T06:01:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T06:01:10.862 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c04f7042-77de-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-c04f7042-77de-11ef-baf5-efdc52797490.target. 2024-09-21T06:01:10.862 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c04f7042-77de-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-c04f7042-77de-11ef-baf5-efdc52797490.target. 2024-09-21T06:01:11.128 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:01:11.128 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T06:01:11.312 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c04f7042-77de-11ef-baf5-efdc52797490.target.wants/ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-c04f7042-77de-11ef-baf5-efdc52797490@.service. 2024-09-21T06:01:11.373 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:11 smithi031 systemd[1]: Starting Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:01:11.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:11 smithi031 podman[30850]: 2024-09-21 06:01:11.470452692 +0000 UTC m=+0.020415078 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:01:12.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:11 smithi031 podman[30850]: 2024-09-21 06:01:11.797210005 +0000 UTC m=+0.347172392 container create eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T06:01:12.863 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:01:13.021 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T06:01:13.022 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:01:13.022 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T06:01:13.023 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T06:01:13.023 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:01:13.023 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-09-21T06:01:13.023 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-09-21T06:01:13.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:12 smithi031 podman[30850]: 2024-09-21 06:01:12.763743013 +0000 UTC m=+1.313705404 container init eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T06:01:13.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:12 smithi031 podman[30850]: 2024-09-21 06:01:12.767266837 +0000 UTC m=+1.317229225 container start eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T06:01:13.023 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:12 smithi031 ceph-mon[30870]: mkfs c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:13.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:12 smithi031 ceph-mon[30870]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:01:13.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:12 smithi031 bash[30850]: eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 2024-09-21T06:01:13.024 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:12 smithi031 systemd[1]: Started Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: monmap epoch 1 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: last_changed 2024-09-21T06:01:08.811930+0000 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: min_mon_release 19 (squid) 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: election_strategy: 1 2024-09-21T06:01:14.097 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:01:14.098 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: fsmap 2024-09-21T06:01:14.098 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:01:14.098 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: mgrmap e1: no daemons active 2024-09-21T06:01:14.098 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:13 smithi031 ceph-mon[30870]: from='client.? 172.21.15.31:0/778138981' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.684544s) 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T06:01:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-09-21T06:01:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T06:01:15.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:14 smithi031 ceph-mon[30870]: from='client.? 172.21.15.31:0/399140203' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:01:15.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:14 smithi031 ceph-mon[30870]: from='client.? 172.21.15.31:0/399140203' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T06:01:15.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:15.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:01:15.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:15.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:01:15.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:01:15.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:01:15.134 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-09-21T06:01:15.951 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:15 smithi031 ceph-mon[30870]: from='client.? 172.21.15.31:0/539925234' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:01:16.213 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-09-21T06:01:16.481 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:16 smithi031 systemd[1]: Stopping Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:01:16.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:16 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a[30866]: 2024-09-21T06:01:16.478+0000 7fd55014d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:01:16.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:16 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a[30866]: 2024-09-21T06:01:16.478+0000 7fd55014d640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T06:01:16.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:16 smithi031 podman[31114]: 2024-09-21 06:01:16.704133996 +0000 UTC m=+0.321171888 container died eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:01:17.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 podman[31114]: 2024-09-21 06:01:17.045320983 +0000 UTC m=+0.662358861 container cleanup eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T06:01:17.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 bash[31114]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:01:17.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 podman[31126]: 2024-09-21 06:01:17.196100433 +0000 UTC m=+0.486880647 container remove eee507efe5bb30ac5294a305e08aca0ec91e1181f934a802b997d87ed9388628 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T06:01:17.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 systemd[1]: ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T06:01:17.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 systemd[1]: Stopped Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:01:17.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 systemd[1]: Starting Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:01:17.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 podman[31218]: 2024-09-21 06:01:17.630245112 +0000 UTC m=+0.112705998 container create 8fe9989c0afa1aefd1e7d292dd85c81673a3c3bd31bca6addb8dda6d7ef9a043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T06:01:17.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 podman[31218]: 2024-09-21 06:01:17.533786957 +0000 UTC m=+0.016247798 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:01:17.946 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T06:01:18.052 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 podman[31218]: 2024-09-21 06:01:17.86610365 +0000 UTC m=+0.348564492 container init 8fe9989c0afa1aefd1e7d292dd85c81673a3c3bd31bca6addb8dda6d7ef9a043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 podman[31218]: 2024-09-21 06:01:17.869540531 +0000 UTC m=+0.352001375 container start 8fe9989c0afa1aefd1e7d292dd85c81673a3c3bd31bca6addb8dda6d7ef9a043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: pidfile_write: ignore empty --pid-file 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: load: jerasure load: lrc 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Git sha 0 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: DB SUMMARY 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: DB Session ID: RWPO3DKBH33MIM8EWE65 2024-09-21T06:01:18.053 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94586 ; 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:01:18.054 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.env: 0x562932e4ed60 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.info_log: 0x5629395f3360 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.statistics: (nil) 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.use_fsync: 0 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:01:18.055 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.db_log_dir: 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.wal_dir: 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:01:18.056 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.write_buffer_manager: 0x5629395f7d60 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:01:18.057 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.unordered_write: 0 2024-09-21T06:01:18.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:01:18.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:01:18.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:01:18.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:01:18.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.row_cache: None 2024-09-21T06:01:18.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.wal_filter: None 2024-09-21T06:01:18.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:01:18.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:01:18.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:01:18.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:01:18.059 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.wal_compression: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:01:18.060 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:01:18.061 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_open_files: -1 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Compression algorithms supported: 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kZSTD supported: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:01:18.062 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.merge_operator: 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_filter: None 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:01:18.063 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5629395f2fa0) 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:01:18.064 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x5629355b7610 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2024-09-21T06:01:18.065 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2024-09-21T06:01:18.066 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression: NoCompression 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.num_levels: 7 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:01:18.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:01:18.068 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:01:18.069 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:01:18.070 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:01:18.071 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.ttl: 2592000 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:01:18.072 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 29af5cd8-1a4e-46ba-800b-b72e5b9c42e9 2024-09-21T06:01:18.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898477905656, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898477907473, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90812, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 88853, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11567, "raw_average_key_size": 47, "raw_value_size": 82316, "raw_average_value_size": 340, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726898477, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "29af5cd8-1a4e-46ba-800b-b72e5b9c42e9", "db_session_id": "RWPO3DKBH33MIM8EWE65", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898477907611, "job": 1, "event": "recovery_finished"} 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562939616a80 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: DB pointer 0x56293a3bc000 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:01:18.074 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 90.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:01:18.075 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 90.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 83.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:01:18.076 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 16.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 16.81 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x5629355b7610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???) e1 preinit fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).mds e1 new map 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).mds e1 print_map 2024-09-21T06:01:18.077 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout: btime 2024-09-21T06:01:12:820344+0000 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T06:01:18.078 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 bash[31218]: 8fe9989c0afa1aefd1e7d292dd85c81673a3c3bd31bca6addb8dda6d7ef9a043 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 systemd[1]: Started Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: monmap epoch 1 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: last_changed 2024-09-21T06:01:08.811930+0000 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: min_mon_release 19 (squid) 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: election_strategy: 1 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: fsmap 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:01:18.079 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:17 smithi031 ceph-mon[31237]: mgrmap e1: no daemons active 2024-09-21T06:01:19.444 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T06:01:19.466 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:01:19.466 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-09-21T06:01:19.466 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T06:01:19.467 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T06:01:19.727 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:01:19.727 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T06:01:19.879 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c04f7042-77de-11ef-baf5-efdc52797490.target.wants/ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-c04f7042-77de-11ef-baf5-efdc52797490@.service. 2024-09-21T06:01:20.121 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:19 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/865004604' entity='client.admin' 2024-09-21T06:01:20.121 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:19 smithi031 systemd[1]: Starting Ceph mgr.x for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:01:20.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:20 smithi031 podman[31449]: 2024-09-21 06:01:20.022229537 +0000 UTC m=+0.015983798 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:01:20.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:20 smithi031 podman[31449]: 2024-09-21 06:01:20.134016332 +0000 UTC m=+0.127770579 container create 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:01:20.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:20 smithi031 podman[31449]: 2024-09-21 06:01:20.328035972 +0000 UTC m=+0.321790230 container init 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:01:20.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:20 smithi031 podman[31449]: 2024-09-21 06:01:20.33084691 +0000 UTC m=+0.324601159 container start 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:01:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:01:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T06:01:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:01:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T06:01:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T06:01:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:01:20.435 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:01:20.435 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T06:01:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:01:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T06:01:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T06:01:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T06:01:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-09-21T06:01:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-09-21T06:01:20.674 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:20 smithi031 bash[31449]: 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 2024-09-21T06:01:20.674 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:20 smithi031 systemd[1]: Started Ceph mgr.x for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:01:21.287 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:21 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2948526456' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "c04f7042-77de-11ef-baf5-efdc52797490", 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:01:21.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:01:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:01:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:01:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:01:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:01:21.623 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:01:21.623 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.624 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:01:12:820344+0000", 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:01:21.625 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:01:12.821205+0000", 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:01:21.626 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-09-21T06:01:24.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:24 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2762819627' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:01:24.716 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:24.716 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:01:24.716 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "c04f7042-77de-11ef-baf5-efdc52797490", 2024-09-21T06:01:24.716 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:01:24.716 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:01:24.716 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:01:24.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:01:24.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:01:24.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:01:24.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:01:12:820344+0000", 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:01:24.722 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:24.723 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:01:12.821205+0000", 2024-09-21T06:01:24.724 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:01:24.724 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:24.724 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:01:24.724 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:01:24.724 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-09-21T06:01:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: Activating manager daemon x 2024-09-21T06:01:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: mgrmap e2: x(active, starting, since 0.00402227s) 2024-09-21T06:01:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:01:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:01:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: Manager daemon x is now available 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' 2024-09-21T06:01:26.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:25 smithi031 ceph-mon[31237]: from='mgr.14100 172.21.15.31:0/1802703756' entity='mgr.x' 2024-09-21T06:01:28.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:27 smithi031 ceph-mon[31237]: mgrmap e3: x(active, since 1.00808s) 2024-09-21T06:01:28.016 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:27 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2500677203' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:01:28.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:28.020 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "c04f7042-77de-11ef-baf5-efdc52797490", 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:28.021 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:01:28.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:01:28.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:01:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:01:12:820344+0000", 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:01:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:01:12.821205+0000", 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:01:28.028 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-09-21T06:01:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:28 smithi031 ceph-mon[31237]: mgrmap e4: x(active, since 2s) 2024-09-21T06:01:28.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:28 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2052549009' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:01:29.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:29.175 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:01:29.176 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T06:01:29.177 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:01:29.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:01:29.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:01:29.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:01:29.178 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:01:29.178 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-09-21T06:01:30.194 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:29 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2876704688' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T06:01:30.194 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ignoring --setuser ceph since I am not root 2024-09-21T06:01:30.194 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ignoring --setgroup ceph since I am not root 2024-09-21T06:01:31.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:30 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2876704688' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T06:01:31.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:30 smithi031 ceph-mon[31237]: mgrmap e5: x(active, since 4s) 2024-09-21T06:01:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:01:31.736 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T06:01:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:01:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:01:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:01:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:01:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-21T06:01:31.737 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-09-21T06:01:31.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:31 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2360514704' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:01:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: Active manager daemon x restarted 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: Activating manager daemon x 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: mgrmap e6: x(active, starting, since 0.0038641s) 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:01:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:35 smithi031 ceph-mon[31237]: Manager daemon x is now available 2024-09-21T06:01:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:01:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T06:01:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:01:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:01:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-09-21T06:01:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying orchestrator module is enabled... 2024-09-21T06:01:37.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: Found migration_current of "None". Setting to last migration. 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:01:37.100 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:36 smithi031 ceph-mon[31237]: mgrmap e7: x(active, since 1.00629s) 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-mon[31237]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-mon[31237]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/407678944' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:01:37.715+0000 7f0f80034640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:01:37] ENGINE Error in HTTPServer.serve 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: Traceback (most recent call last): 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._connections.run(self.expiration_interval) 2024-09-21T06:01:38.136 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._run(expiration_interval) 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: s = self.context.wrap_socket( 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: return self.sslsocket_class._create( 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self.do_handshake() 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._sslobj.do_handshake() 2024-09-21T06:01:38.137 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:01:38.138 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:37 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:01:38.472 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T06:01:38.472 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-09-21T06:01:39.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: [21/Sep/2024:06:01:37] ENGINE Bus STARTING 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: [21/Sep/2024:06:01:37] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: [21/Sep/2024:06:01:37] ENGINE Error in HTTPServer.serve 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:01:39.386 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: [21/Sep/2024:06:01:37] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: [21/Sep/2024:06:01:37] ENGINE Bus STARTED 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/407678944' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T06:01:39.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:38 smithi031 ceph-mon[31237]: mgrmap e8: x(active, since 2s) 2024-09-21T06:01:40.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:40 smithi031 ceph-mon[31237]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:40.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:40 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:40.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:40 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:40.900 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T06:01:40.900 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-09-21T06:01:41.350 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-mon[31237]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:42.003 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: Generating public/private rsa key pair. 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: Your identification has been saved in /tmp/tmpmf3pa8vu/key 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: Your public key has been saved in /tmp/tmpmf3pa8vu/key.pub 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: The key fingerprint is: 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: SHA256:n+M5Q1EoliA/i6rlzstXHde9zi3DJaWpdvKgrYMzDtY ceph-c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: The key's randomart image is: 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: +---[RSA 3072]----+ 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | . .. . . | 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | o + . . | 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | o. . o . | 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | . o. o . . .| 2024-09-21T06:01:42.004 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | . ..So . = | 2024-09-21T06:01:42.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | . ...... = .| 2024-09-21T06:01:42.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | o .o Eo+ .= + | 2024-09-21T06:01:42.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: | * .. .+.+=+.B .| 2024-09-21T06:01:42.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: |..*o ..o=*.+.o | 2024-09-21T06:01:42.005 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: +----[SHA256]-----+ 2024-09-21T06:01:42.865 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:42 smithi031 ceph-mon[31237]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:42.866 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:42 smithi031 ceph-mon[31237]: Generating ssh key... 2024-09-21T06:01:42.866 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:42 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:42.866 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:42 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:43.429 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:01:43.429 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:01:43.429 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T06:01:43.437 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-09-21T06:01:43.977 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:43 smithi031 ceph-mon[31237]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:45.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:45 smithi031 ceph-mon[31237]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:46.266 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:46 smithi031 ceph-mon[31237]: Deploying cephadm binary to smithi031 2024-09-21T06:01:48.519 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-09-21T06:01:48.519 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2024-09-21T06:01:49.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:48 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:49.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:48 smithi031 ceph-mon[31237]: Added host smithi031 2024-09-21T06:01:49.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:48 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:49.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T06:01:49.744 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2024-09-21T06:01:50.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:50 smithi031 ceph-mon[31237]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:50.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:50 smithi031 ceph-mon[31237]: Saving service mon spec with placement count:5 2024-09-21T06:01:50.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:50 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:52.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:51 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:52.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:51 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:52.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:51 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:52.153 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T06:01:52.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:52 smithi031 ceph-mon[31237]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:01:52.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:52 smithi031 ceph-mon[31237]: Saving service mgr spec with placement count:2 2024-09-21T06:01:54.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:53 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1045960471' entity='client.admin' 2024-09-21T06:01:54.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:53 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-09-21T06:01:55.089 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:54 smithi031 ceph-mon[31237]: from='mgr.14118 172.21.15.31:0/298681149' entity='mgr.x' 2024-09-21T06:01:55.089 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:54 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2884381053' entity='client.admin' 2024-09-21T06:01:56.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:56 smithi031 ceph-mon[31237]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:01:56.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:56 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2474305389' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T06:01:56.506 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ignoring --setuser ceph since I am not root 2024-09-21T06:01:56.506 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:01:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ignoring --setgroup ceph since I am not root 2024-09-21T06:01:57.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:57 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2474305389' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T06:01:57.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:57 smithi031 ceph-mon[31237]: mgrmap e9: x(active, since 21s) 2024-09-21T06:01:58.039 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-21T06:01:58.040 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-09-21T06:01:58.309 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:01:58 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2197828631' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:02:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: Active manager daemon x restarted 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: Activating manager daemon x 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: mgrmap e10: x(active, starting, since 0.00624979s) 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:02:01.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:01 smithi031 ceph-mon[31237]: Manager daemon x is now available 2024-09-21T06:02:02.784 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:02 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:02.784 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:02 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:02:02.784 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:02 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:02:02.784 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:02 smithi031 ceph-mon[31237]: mgrmap e11: x(active, since 1.00783s) 2024-09-21T06:02:03.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:02:03.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T06:02:03.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:02:03.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:02:03.054 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-09-21T06:02:03.054 INFO:teuthology.orchestra.run.smithi031.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T06:02:03.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:03 smithi031 ceph-mon[31237]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:02:03.706 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:03 smithi031 ceph-mon[31237]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:02:04.126+0000 7fa57246c640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:02:04] ENGINE Error in HTTPServer.serve 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: Traceback (most recent call last): 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._connections.run(self.expiration_interval) 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._run(expiration_interval) 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:02:04.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: s = self.context.wrap_socket( 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: return self.sslsocket_class._create( 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self.do_handshake() 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._sslobj.do_handshake() 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:02:04.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:02:04.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: mgrmap e12: x(active, since 2s) 2024-09-21T06:02:04.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: [21/Sep/2024:06:02:04] ENGINE Bus STARTING 2024-09-21T06:02:04.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: [21/Sep/2024:06:02:04] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T06:02:04.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: [21/Sep/2024:06:02:04] ENGINE Error in HTTPServer.serve 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-21T06:02:04.837 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: [21/Sep/2024:06:02:04] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T06:02:04.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:04 smithi031 ceph-mon[31237]: [21/Sep/2024:06:02:04] ENGINE Bus STARTED 2024-09-21T06:02:05.421 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUNTiQDyIRQ/C5yVkCtm0lP3InMScwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTA2MDIwNFoXDTM0\nMDkyMjA2MDIwNFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjMxMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAwamVX3wUkd3Xp7pzVs/yfHJ2R27bnTHYN3I3\n031xzxbm2VryQEJAeFkVJx3Huk+RuxUDQjDlk9nzu89hlKaxI8FHUIbe2+3yT6hk\ntUpNMNhJG+Pf4UaAecHfwl6PePeCir9WQPAMsODBz9K/ChHKhylNCKJcWuCCQKJV\nRH+vfzedoEOFvnApsaAGQD7D3ppqdNq+Y+T7JFfa8eZgQCBVVSO1R9vQDchJF5Fo\nViS6mqc7jmpAyox5Sh/vQQ0q0sMkGbY5hQql/t3NFJoP/I6bYtGgRmxFfiICcb3v\nXhKmL6/hM1FMyRO9/CfmRSkEDS70oHyY5EgsuZBfd530iWyP3pvsAlzEwoVdFVCO\ngY85T+si8wl1T9lEZlXdEybpwSNYvjpgy0am9N43IqM7U8kSkrmssRo8YEh8Bn1p\nju6DoansgunkWTkhWucjGcQpYF2hiCKDR1nGwfT+GJPW4R2QhrX4SPAlso993kaW\nVvBXjGo1TwKmiO0Dh41SHE5zvkx64b47R8bqS0skkJR7dAwnFrWc6nRGdBLIYTzu\nBCU4c93fkHjzC7afOzQxDNPcMb4JAHQLAHfpXRFEx5lSRvjUFRa3l0SNbmv9ENlM\nC70v0jZm8VrN0vuQ7WCeVuhAHi3Y6DDUty8Tm9IuYAHl6V4QttCttT/E283T4l0Y\nGTy2wl8CAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDMxghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPHzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBl\nFmZ3ftaBrLSs1rqVPZBzHXzNXn+kDZQs9H/5Dqd24H78PWsv56sREG+0m392Fbk6\nKKPYVRiNw1gBTZXq3Gz+XiXl+p0gZVxBK06rTt5LLalrkQBb0ymtSWperw8G1PGj\nDXqdw7RE4Q5Hlcw62Na5q0FwuwCHKRpjOWFi8Ty51VrfRwBg3MjmP3L3L4ZdY1dL\nLuRY8j1lWki5vQ5Fg4w3EzkXrP3H6ewjTEHYlccE4SUZUvLZCpDct257R1aeqXMZ\n4h/zkp8i54hxYpuG2vc9q9SG2WWYuSTt/sI2qJomWdGewK8IZaDhr/bgAsQK/7Kq\n7UoLe8Ni7mje6Nact75VGtaDmF/MQ8vfpnW1n8Bv2bl//syP1ZmwCQn79haghu8T\nRBhticu4nE/vPFW4Hu9Wa/ctUfChVIUU4uenBcQVyb9YT/b3WjBsctt5rAgeHeEW\noER0hfpu9dvm1BCO9t7xdqRgS2eHQmQDrZ3CxDToasm4dGoO7aTdVfyBFDWpox6z\nHqfU9IcKXeQ/PxfEgAFdzqA6ypWdpAaj7A6KtnwhXC8bmiGB0ymYUgzxBy6TJU4g\n20CWj/JtiGalazQKxFqMc/8eyV+1nAmB0uPns+QE2ZVmELSVl6zsAI523p+izmej\nqAfpfUv8HFBM06tDOghG3bZcscnzun+GCTH7oFrG7w==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAwamVX3wUkd3Xp7pzVs/yfHJ2R27bnTHYN3I3031xzxbm2Vry\nQEJAeFkVJx3Huk+RuxUDQjDlk9nzu89hlKaxI8FHUIbe2+3yT6hktUpNMNhJG+Pf\n4UaAecHfwl6PePeCir9WQPAMsODBz9K/ChHKhylNCKJcWuCCQKJVRH+vfzedoEOF\nvnApsaAGQD7D3ppqdNq+Y+T7JFfa8eZgQCBVVSO1R9vQDchJF5FoViS6mqc7jmpA\nyox5Sh/vQQ0q0sMkGbY5hQql/t3NFJoP/I6bYtGgRmxFfiICcb3vXhKmL6/hM1FM\nyRO9/CfmRSkEDS70oHyY5EgsuZBfd530iWyP3pvsAlzEwoVdFVCOgY85T+si8wl1\nT9lEZlXdEybpwSNYvjpgy0am9N43IqM7U8kSkrmssRo8YEh8Bn1pju6Doansgunk\nWTkhWucjGcQpYF2hiCKDR1nGwfT+GJPW4R2QhrX4SPAlso993kaWVvBXjGo1TwKm\niO0Dh41SHE5zvkx64b47R8bqS0skkJR7dAwnFrWc6nRGdBLIYTzuBCU4c93fkHjz\nC7afOzQxDNPcMb4JAHQLAHfpXRFEx5lSRvjUFRa3l0SNbmv9ENlMC70v0jZm8VrN\n0vuQ7WCeVuhAHi3Y6DDUty8Tm9IuYAHl6V4QttCttT/E283T4l0YGTy2wl8CAwEA\nAQKCAgAS4cvNw1rRaA2tQwluMZiPwqyT8BPwjS9qzzObPBQQdXKs8ZLvafuViwCm\nz4lYOdwtyfGHOXOWM/4cKy71SpVjNOaU9uQcEbn6Q023vAFaXamEfct9JZ3EK/96\n7jOolIqr/YDVlFzLEsxCzOZR1ISTI2yT3BoLcEVdiOuttzyvYjGiXRhZW4qX0NBF\n4PjBp6ZGiMdfFQXZjXQBpDfGjispFxyx32rh1SEvROmlMIU91p1WGDet/VPxbWYA\nnoiKLnrtbi4d3HkX0vIPwVUFXQrQb+Y5HOMKVnA3knsYCH3J5ijxEyBcoJSkVRLn\nXlrl8Cnrdd/bA3XSN+iJpfh1kln/1iDLchj910o7m767fpf3CmNLD6e/h6/wnkcG\ngEtC3GbEzdxJgJN2tfXguO82AvQMjphPL/cyqd8o7QQRmzrGJo1wxlrk9zFZolyZ\nOHZ8E5OX7JNXXzmdSrvaGsSTZpllBPlq2Dur/oM7dniqy7gP3+4jPczvTu81t3Y8\nSvIhoFO5t/nF3azJ6pPyPJvpoPj7JVHx88k8A5sg05FqLTlNi/x+BzGgtzgwILyj\n9VopvPdJOOE3Q6TsmK/4RKoD4gwTLr5q8LBEcTcMj7JRbZSV+bKUsfptfn/mY20B\nVXGQEIdgvOgiLFt7Vr+YiysfiMptbdMsCDrteOBCtcFNKjmRLQKCAQEA/Ewc4i9g\ncRWKRFdSCV/1SQz+geGPgt4W1Doi000pqNTGx6HvgfiTxQQ4vOVsDlwHceyDuorJ\nSehBJolbQiGwagqpEqzv6DS+dbIl831KbpgYXoShh1VDynWHfZgowShHDzQnh0OX\nEPlzQxGQW/F5GcHV6g2BzNQsmnllQUUoZHZCSDjpSFSCgLFMlJFzuj+SGLD3EEAb\n0rrDf6jK357lNyGCQjmA+FbLVmQJBW6VHFW2O3I/sPtXnEvxgbzzHaGTUMi3BT+K\nqnTfazaZnvmNqcPajsuoC503NSQNhYgHfCvKMo+T/Ompy4yE2lgNmMwSx6OIwQCi\nZqfwP7euvb62SwKCAQEAxIEtkHer9WoFoUda5hH3G4ucXWyrIhB7ZOKTTKS5oFZJ\nKowVUz++0t0VVhUZBWVcgcLzSmC1ojezCV2/7k/mBMV3Uau+TQukAbUvJZy9Xt/Y\nBI2mfLN0+kVi8S+CXkIi/bvFaS7cODrZS0tM5cl7ojUFMccChg2Aip3ATjwqDMu1\n7bn3lGngqtvhFI+WjngySke0opcMnBmytG08zHEHWEa0khH/2v4O6lfmqQz0ZjRy\nlBx4R+TNwCmCO8ocCdVDUIXIGsxHTuPmqwg2jdr+bu14tBJ7Pbn4lRHWRzTJUiBz\nH65+twx6OD88ASPIJ/yJlUxrCCILkJFv1yvbD3ZnvQKCAQA/rYI4wFZqFFKyjwuA\nnDyPdH0kEOck06MVa5rwk/Ao0ZNePYvy1OeO+VLMsmcEdMvYQCa7RV4qjjkrXc2v\nsksRaKFlukpO5EkR8aXMf9nycebsrDMMskOK7QEGfQoEPoM43iP/6tpNCXjC+8Lg\nweCb1KWGDmStavva/TrGL4tTV9nA2sReYh4SFy3J7bA02Oip3Lhu3I9pL4Q6S0Lx\nD+ywbn+u8U6Q328w3cy9Fz0BSszU0jArHjTtmOlaonMGDorc/9vZskDwZtEhAUXi\nrv/HUkxucA545/RQMEpNYpjpCTBedIiYc+1MF6pMZZ8A6rjutGLGu79Bg6+KlXTA\nJUtjAoIBAE0vKESMnQCiil+U9YFVISiyp9KSco3Ix48S6CS8tyThf4+0wlOWwE93\nA4ByfQd/jtxxRLq8TdvP4QUWy+LBg2nwXjO4L+F3Zw48qq4taLyxU9pGAVUN8C61\nvm1Z6e04MSJU9O+htmJnyx4mD6hagbNu/AFXWAhrmUfZOsYAH8wy39zmpPaot9Sk\n1T4MQGjpkWdXzv3mMs/F0EnKdo0fAHGOJDMT2tmb3SPzLujL69uRUkgW/q/OkXqf\nU0ZE1sqN+AjS44yTHJdCMUKQ0Uza7gGISVcBw2K1f4yeGvqRI1qc4FtbYO5pekBJ\npjoUYfRC34/DGm9xX/WDCtkLotnp+kkCggEBAKOo7uuDmJr6oWL9oLjnYzNgntn2\no65Jf2pYbJPNpVLBJdWpPJUTU9vsZo6ZA/Xm8PfJj0m4/6c+s5TYswM9B3HOhQuU\n2stqvjXR1nJiEfw52nk5JupS7NtbEYva+zSZIt1ZQlcFBj0jAgz5ForJG/lAgbnw\nDAZcPRX5K34K1YFGb/DvTv+vklB+l9aNfN0LtIg2dk6xxgrAmDmbBbT5g4f1cp1e\nDR7OHNqjYoH/xRKt2CPkn83OwUKt1eeUzZoWi/CAm9rudQ4YKFQFe2FMEWRNkC0w\nx8jBdR8U8QbY5DCMfTR4/4HxCTrxxBpEoCkouQ3uoG2X6/lY4MIq8nnArmk=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T06:02:05.998 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:05 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:07.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T06:02:07.574 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:07 smithi031 ceph-mon[31237]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:07.574 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:07 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:08.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T06:02:08.573 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-09-21T06:02:09.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:08 smithi031 ceph-mon[31237]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:09.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:08 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:10.093 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:09 smithi031 ceph-mon[31237]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:10.093 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:09 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:10.096 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$gZPXBXnU8tFQ1c4wm62mQuOKIQGsFgHFryuA/4/mj1OpafqWFG6T.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726898529, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T06:02:10.096 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-09-21T06:02:11.282 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:10 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2220652366' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T06:02:11.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T06:02:11.295 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:02:11.295 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-21T06:02:11.303 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:02:11.303 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-21T06:02:11.303 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-21T06:02:11.303 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T06:02:11.305 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-09-21T06:02:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2024-09-21T06:02:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-09-21T06:02:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: Password: o24hphhty3 2024-09-21T06:02:11.305 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:11.306 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config directory 2024-09-21T06:02:12.632 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T06:02:12.632 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T06:02:12.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.633 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T06:02:12.634 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:12.634 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-09-21T06:02:12.671 INFO:tasks.cephadm:Fetching config... 2024-09-21T06:02:12.671 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:02:12.672 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T06:02:12.690 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T06:02:12.690 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:02:12.690 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T06:02:12.748 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T06:02:12.748 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:02:12.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T06:02:12.813 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T06:02:12.813 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:02:12.813 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T06:02:12.870 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T06:02:12.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:02:12.948 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:12.959 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:02:13.002 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:13.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:02:13.059 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRsVerR3Kq/pAFtT6fmOHzvmaidiuPqqEFXMPgJB5tvR9GVcA22KcGNc4TW1ROdI+bmDXGWBwcjS2JJCqt4xbdxpDkWytsvHqjts8G0coGuZDb43UhoMM77GwUS+rhcBMqSeg80Wa8zh1EEtjmyn+LsZ5N6LM4xsd6+bBisNhi+ZcczxjNQa4SRYgRL1SXjmDNYeLwTYBPflbtdbUkzJlpQwM9a1y4PQp1zQ1htHLAxG5xBk53ynS4xn0rhdGAd6VPVIapGZ04hHlFvGchN7JBAbeYQWWFFQaaz6qqZ1tAbCo1+P4xsd/1IxjCTuZadMcee14siQxT5cBelyTeLlEgMHsw0WkO9ilQkjlf18tD7HzuQPxc02fMEkpOwmmdm2lFqthqnsu2PJfCoHC6ja5zGRcwXV3JsPoNBVTw5+T+549oHi0Q5HC31iCf+6HDMJx4vEMrjgzx55efXzfwiycNY7BGlWUwlWWOjasyH8MC3uQu5awv2E0TknlmIPNOt9c= ceph-c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:13.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T06:02:13.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:13 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3621466868' entity='client.admin' 2024-09-21T06:02:13.330 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:15.224 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T06:02:15.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/665271959' entity='client.admin' 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: mgrmap e13: x(active, since 12s) 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:15.331 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:15 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:15.558 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:17.448 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi107 2024-09-21T06:02:17.448 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:02:17.448 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:02:17.467 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:02:17.467 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:17.525 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2024-09-21T06:02:17.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch host add smithi107 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:02:17.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:17.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:17.551 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:17 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:17.780 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:19 smithi031 ceph-mon[31237]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:20.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:20 smithi031 ceph-mon[31237]: Deploying cephadm binary to smithi107 2024-09-21T06:02:22.170 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi107' with addr '172.21.15.107' 2024-09-21T06:02:22.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:02:23.078 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:23.476 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:23 smithi031 ceph-mon[31237]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:23.476 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:23 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:23.476 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:23 smithi031 ceph-mon[31237]: Added host smithi107 2024-09-21T06:02:23.476 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:23 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:23.476 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:23 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:23.476 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:23 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:23.773 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:23.774 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}] 2024-09-21T06:02:24.403 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2024-09-21T06:02:24.404 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:02:24.404 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:02:24.422 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:02:24.423 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:24.480 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2024-09-21T06:02:24.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch host add smithi110 2024-09-21T06:02:24.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:24 smithi031 ceph-mon[31237]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:24.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:24 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:24.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:24 smithi031 ceph-mon[31237]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:02:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:27.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"} : dispatch 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:27.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:26 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:28.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:28 smithi031 ceph-mon[31237]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:02:28.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:28 smithi031 ceph-mon[31237]: Deploying cephadm binary to smithi110 2024-09-21T06:02:29.167 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi110' with addr '172.21.15.110' 2024-09-21T06:02:29.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:29 smithi031 ceph-mon[31237]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:29.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:02:30.024 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:30.414 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:30 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:30.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:30 smithi031 ceph-mon[31237]: Added host smithi110 2024-09-21T06:02:30.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:30 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:30.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:30 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:30.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:30 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:02:30.714 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2024-09-21T06:02:31.291 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:31 smithi031 ceph-mon[31237]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:31.291 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:31 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:31.320 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T06:02:31.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T06:02:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:32.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:32 smithi031 ceph-mon[31237]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:02:33.177 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-21T06:02:33.555 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:33 smithi031 ceph-mon[31237]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:33.556 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:33 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/588753940' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:02:33.887 INFO:tasks.cephadm:Adding mon.a on smithi031 2024-09-21T06:02:33.887 INFO:tasks.cephadm:Adding mon.b on smithi107 2024-09-21T06:02:33.887 INFO:tasks.cephadm:Adding mon.c on smithi110 2024-09-21T06:02:33.887 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi107:172.21.15.107=b;smithi110:172.21.15.110=c' 2024-09-21T06:02:34.120 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/588753940' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T06:02:34.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:34 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:34.742 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mon update... 2024-09-21T06:02:35.366 DEBUG:teuthology.orchestra.run.smithi107:mon.b> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T06:02:35.368 DEBUG:teuthology.orchestra.run.smithi110:mon.c> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T06:02:35.371 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:02:35.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:02:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:35 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:36.262 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:02:36.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:36 smithi031 ceph-mon[31237]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi107:172.21.15.107=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:36.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:36 smithi031 ceph-mon[31237]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi107:172.21.15.107=b;smithi110:172.21.15.110=c;count:3 2024-09-21T06:02:36.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:36 smithi031 ceph-mon[31237]: Deploying daemon mon.c on smithi110 2024-09-21T06:02:37.155 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:02:37.155 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":1,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","modified":"2024-09-21T06:01:08.811930Z","created":"2024-09-21T06:01:08.811930Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T06:02:37.155 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 1 2024-09-21T06:02:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:37 smithi031 ceph-mon[31237]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:37.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:37 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/1693689463' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:38.780 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:02:38.781 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:02:39.172 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:02:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:39 smithi110 systemd[1]: Starting Ceph mon.c for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:02:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:39 smithi031 ceph-mon[31237]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:39.997 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:02:39.997 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":1,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","modified":"2024-09-21T06:01:08.811930Z","created":"2024-09-21T06:01:08.811930Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T06:02:39.997 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 1 2024-09-21T06:02:40.167 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:39 smithi110 podman[34518]: 2024-09-21 06:02:39.892958297 +0000 UTC m=+0.107985432 container create f53d66b96a12c65417f063af8c5a8948b01c87b8796ac9b43ffd0189bb18f15c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:02:40.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:39 smithi110 podman[34518]: 2024-09-21 06:02:39.800699153 +0000 UTC m=+0.015726307 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:02:40.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 podman[34518]: 2024-09-21 06:02:40.128699837 +0000 UTC m=+0.343726974 container init f53d66b96a12c65417f063af8c5a8948b01c87b8796ac9b43ffd0189bb18f15c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:02:40.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 podman[34518]: 2024-09-21 06:02:40.132917457 +0000 UTC m=+0.347944593 container start f53d66b96a12c65417f063af8c5a8948b01c87b8796ac9b43ffd0189bb18f15c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T06:02:40.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:02:40.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:02:40.168 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pidfile_write: ignore empty --pid-file 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: load: jerasure load: lrc 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Git sha 0 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: DB SUMMARY 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: DB Session ID: 18UMLVMH487CMWDMQV7N 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T06:02:40.448 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.env: 0x5603f54cdd60 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.info_log: 0x5603fb21f360 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.statistics: (nil) 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.use_fsync: 0 2024-09-21T06:02:40.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.db_log_dir: 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.wal_dir: 2024-09-21T06:02:40.450 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:02:40.451 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:02:40.451 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:02:40.451 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:02:40.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:02:40.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:02:40.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:02:40.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:02:40.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.write_buffer_manager: 0x5603fb223d60 2024-09-21T06:02:40.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.unordered_write: 0 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.row_cache: None 2024-09-21T06:02:40.453 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.wal_filter: None 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.wal_compression: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:02:40.454 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:02:40.455 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_open_files: -1 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Compression algorithms supported: 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kZSTD supported: 0 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:02:40.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.merge_operator: 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_filter: None 2024-09-21T06:02:40.457 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5603fb21efa0) 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_type: 0 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_index_type: 0 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_shortening: 1 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: checksum: 4 2024-09-21T06:02:40.458 INFO:journalctl@ceph.mon.c.smithi110.stdout: no_block_cache: 0 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache: 0x5603f71e3610 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_options: 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: capacity : 536870912 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_shard_bits : 4 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: strict_capacity_limit : 0 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_compressed: (nil) 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: persistent_cache: (nil) 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size: 4096 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size_deviation: 10 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_restart_interval: 16 2024-09-21T06:02:40.459 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_block_restart_interval: 1 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: metadata_block_size: 4096 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: partition_filters: 0 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: use_delta_encoding: 1 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: filter_policy: bloomfilter 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: whole_key_filtering: 1 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: verify_compression: 0 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: format_version: 5 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_index_compression: 1 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_align: 0 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: max_auto_readahead_size: 262144 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: prepopulate_block_cache: 0 2024-09-21T06:02:40.460 INFO:journalctl@ceph.mon.c.smithi110.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression: NoCompression 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.num_levels: 7 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:02:40.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:02:40.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:02:40.463 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:02:40.464 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:02:40.465 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.ttl: 2592000 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:02:40.466 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 05df4b98-2955-46a4-a382-31a09ee3582c 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898560171722, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898560172177, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726898560, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "05df4b98-2955-46a4-a382-31a09ee3582c", "db_session_id": "18UMLVMH487CMWDMQV7N", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:02:40.467 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726898560172277, "job": 1, "event": "recovery_finished"} 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5603fb242a80 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: DB pointer 0x5603fbfe8000 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** DB Stats ** 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:02:40.468 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:02:40.469 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Block cache BinnedLRUCache@0x5603f71e3610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:02:40.470 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: using public_addr v2:172.21.15.110:0/0 -> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: starting mon.c rank -1 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(???) e0 preinit fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: e1 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: btime 2024-09-21T06:01:12:820344+0000 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: legacy client fscid: -1 2024-09-21T06:02:40.471 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout: No filesystems configured 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Deploying cephadm binary to smithi110 2024-09-21T06:02:40.472 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Added host smithi110 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/588753940' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/588753940' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:02:40.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi107:172.21.15.107=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi107:172.21.15.107=b;smithi110:172.21.15.110=c;count:3 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: Deploying daemon mon.c on smithi110 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/1693689463' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 ceph-mon[34550]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T06:02:40.475 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 bash[34518]: f53d66b96a12c65417f063af8c5a8948b01c87b8796ac9b43ffd0189bb18f15c 2024-09-21T06:02:40.476 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:40 smithi110 systemd[1]: Started Ceph mon.c for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:02:41.895 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:02:41.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:02:42.163 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:02:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: mon.a calling monitor election 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: mon.c calling monitor election 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:02:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: monmap epoch 2 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: last_changed 2024-09-21T06:02:40.200059+0000 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: min_mon_release 19 (squid) 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: election_strategy: 1 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: fsmap 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: mgrmap e13: x(active, since 43s) 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: overall HEALTH_OK 2024-09-21T06:02:45.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:45.531 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:45.531 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:45.531 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:45.531 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:45 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:45.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:02:45.724 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: mon.a calling monitor election 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: mon.c calling monitor election 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:02:45.725 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: monmap epoch 2 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: last_changed 2024-09-21T06:02:40.200059+0000 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: min_mon_release 19 (squid) 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: election_strategy: 1 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: fsmap 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: mgrmap e13: x(active, since 43s) 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: overall HEALTH_OK 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:45.726 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:45.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:45.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:45.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:45 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:45.828 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:02:45.828 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","modified":"2024-09-21T06:02:40.200059Z","created":"2024-09-21T06:01:08.811930Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:02:45.828 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 2 2024-09-21T06:02:46.398 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:46 smithi110 ceph-mon[34550]: Deploying daemon mon.b on smithi107 2024-09-21T06:02:46.398 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:46 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/479706175' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:46.398 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:46 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:46.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:46 smithi031 ceph-mon[31237]: Deploying daemon mon.b on smithi107 2024-09-21T06:02:46.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:46 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/479706175' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:46.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:46 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:47.426 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:02:47.426 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:02:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:47 smithi031 ceph-mon[31237]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:47.679 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:02:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:47 smithi110 ceph-mon[34550]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:48.381 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:02:48.381 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","modified":"2024-09-21T06:02:40.200059Z","created":"2024-09-21T06:01:08.811930Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:02:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 2 2024-09-21T06:02:48.381 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:48 smithi110 ceph-mon[34550]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:48.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:48 smithi031 ceph-mon[31237]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:48 smithi107 systemd[1]: Started Ceph mon.b for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:02:49.908 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:02:49.909 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:02:50.159 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: mon.c calling monitor election 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: mon.a calling monitor election 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:54.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: monmap epoch 3 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: last_changed 2024-09-21T06:02:48.922359+0000 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: min_mon_release 19 (squid) 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: election_strategy: 1 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: fsmap 2024-09-21T06:02:54.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:54.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: mgrmap e13: x(active, since 52s) 2024-09-21T06:02:54.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: overall HEALTH_OK 2024-09-21T06:02:54.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:53 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: mon.c calling monitor election 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: mon.a calling monitor election 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: monmap epoch 3 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: last_changed 2024-09-21T06:02:48.922359+0000 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: min_mon_release 19 (squid) 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: election_strategy: 1 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-21T06:02:54.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: fsmap 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: mgrmap e13: x(active, since 52s) 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: overall HEALTH_OK 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:53 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:54.491 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:02:54.491 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":3,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","modified":"2024-09-21T06:02:48.922359Z","created":"2024-09-21T06:01:08.811930Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:02:54.491 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 3 2024-09-21T06:02:55.173 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:54 smithi110 ceph-mon[34550]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:55.173 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:54 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:55.173 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:54 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/2439361047' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:55.173 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:54 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:55.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:54 smithi031 ceph-mon[31237]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:55.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:54 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:55.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:54 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/2439361047' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:55.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:54 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:55.627 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T06:02:55.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T06:02:56.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: mon.c calling monitor election 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: mon.a calling monitor election 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:02:56.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: monmap epoch 3 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: last_changed 2024-09-21T06:02:48.922359+0000 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: min_mon_release 19 (squid) 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: election_strategy: 1 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: fsmap 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: mgrmap e13: x(active, since 52s) 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: overall HEALTH_OK 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:56.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:56.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:56.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:56.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:56.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:56.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/2439361047' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:02:56.292 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:55 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: mon.b calling monitor election 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: Reconfiguring daemon mon.a on smithi031 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: mon.b calling monitor election 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: mon.c calling monitor election 2024-09-21T06:02:56.680 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: mon.a calling monitor election 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: monmap epoch 3 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: last_changed 2024-09-21T06:02:48.922359+0000 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: min_mon_release 19 (squid) 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: election_strategy: 1 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: fsmap 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:56.681 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: mgrmap e13: x(active, since 54s) 2024-09-21T06:02:56.682 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:56 smithi031 ceph-mon[31237]: overall HEALTH_OK 2024-09-21T06:02:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: mon.b calling monitor election 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: Reconfiguring daemon mon.a on smithi031 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: mon.b calling monitor election 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: mon.c calling monitor election 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: mon.a calling monitor election 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: monmap epoch 3 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: last_changed 2024-09-21T06:02:48.922359+0000 2024-09-21T06:02:56.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: min_mon_release 19 (squid) 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: election_strategy: 1 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: fsmap 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: mgrmap e13: x(active, since 54s) 2024-09-21T06:02:56.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:56 smithi110 ceph-mon[34550]: overall HEALTH_OK 2024-09-21T06:02:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: mon.b calling monitor election 2024-09-21T06:02:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:02:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: Reconfiguring daemon mon.a on smithi031 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: mon.b calling monitor election 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: mon.c calling monitor election 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: mon.a calling monitor election 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: monmap epoch 3 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: last_changed 2024-09-21T06:02:48.922359+0000 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: created 2024-09-21T06:01:08.811930+0000 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: min_mon_release 19 (squid) 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: election_strategy: 1 2024-09-21T06:02:56.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-21T06:02:56.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-09-21T06:02:56.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: 2: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-09-21T06:02:56.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: fsmap 2024-09-21T06:02:56.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:02:56.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: mgrmap e13: x(active, since 54s) 2024-09-21T06:02:56.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:56 smithi107 ceph-mon[34380]: overall HEALTH_OK 2024-09-21T06:02:57.419 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:57.419 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-21T06:02:57.419 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:02:57.419 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-09-21T06:02:58.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:02:58.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:02:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: Reconfiguring daemon mon.b on smithi107 2024-09-21T06:02:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:58.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:57 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2709362581' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:02:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:02:58.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:58.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: Reconfiguring daemon mon.b on smithi107 2024-09-21T06:02:58.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:58.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:57 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2709362581' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: Reconfiguring daemon mon.b on smithi107 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:02:58.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:57 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2709362581' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:58.040 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T06:02:58.041 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:02:58.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:02:58.070 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:02:58.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:58.136 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:02:58.137 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:02:58.164 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:02:58.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:58.229 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:02:58.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:02:58.256 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:02:58.256 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:02:58.320 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T06:02:58.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T06:02:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:02:59.048 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:59.049 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:02:58 smithi031 ceph-mon[31237]: Reconfiguring daemon mon.c on smithi110 2024-09-21T06:02:59.189 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:59.190 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:02:58 smithi110 ceph-mon[34550]: Reconfiguring daemon mon.c on smithi110 2024-09-21T06:02:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:02:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:02:59.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:02:59.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:02:59.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:02:59.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:59.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:02:58 smithi107 ceph-mon[34380]: Reconfiguring daemon mon.c on smithi110 2024-09-21T06:03:00.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:00.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:00.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:00.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:00 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2994486369' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:03:00.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:00.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:00.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:00.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:00 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2994486369' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:03:00.600 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ignoring --setuser ceph since I am not root 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ignoring --setgroup ceph since I am not root 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='mgr.14152 172.21.15.31:0/3763935221' entity='mgr.x' 2024-09-21T06:03:00.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:00 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2994486369' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:03:00.939 INFO:tasks.cephadm:Adding mgr.x on smithi031 2024-09-21T06:03:00.939 INFO:tasks.cephadm:Adding mgr.y on smithi107 2024-09-21T06:03:00.939 INFO:tasks.cephadm:Adding mgr.z on smithi110 2024-09-21T06:03:00.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi031=x;smithi107=y;smithi110=z' 2024-09-21T06:03:01.190 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:03:01.525 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:01 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2994486369' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:03:01.526 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:01 smithi110 ceph-mon[34550]: mgrmap e14: x(active, since 58s) 2024-09-21T06:03:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:01 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2994486369' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:03:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:01 smithi107 ceph-mon[34380]: mgrmap e14: x(active, since 58s) 2024-09-21T06:03:01.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:01 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2994486369' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:03:01.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:01 smithi031 ceph-mon[31237]: mgrmap e14: x(active, since 58s) 2024-09-21T06:03:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: Active manager daemon x restarted 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: Activating manager daemon x 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: mgrmap e15: x(active, starting, since 0.00487382s) 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:03:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:05 smithi031 ceph-mon[31237]: Manager daemon x is now available 2024-09-21T06:03:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: Active manager daemon x restarted 2024-09-21T06:03:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: Activating manager daemon x 2024-09-21T06:03:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:03:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: mgrmap e15: x(active, starting, since 0.00487382s) 2024-09-21T06:03:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:03:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:05 smithi110 ceph-mon[34550]: Manager daemon x is now available 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: Active manager daemon x restarted 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: Activating manager daemon x 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: mgrmap e15: x(active, starting, since 0.00487382s) 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:03:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:03:06.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:03:06.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:03:06.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:03:06.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:05 smithi107 ceph-mon[34380]: Manager daemon x is now available 2024-09-21T06:03:06.561 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mgr update... 2024-09-21T06:03:06.861 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:06.862 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:03:06.862 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:03:06.862 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:06 smithi110 ceph-mon[34550]: mgrmap e16: x(active, since 1.01096s) 2024-09-21T06:03:06.862 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:06.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:06.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:03:06.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:03:06.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:06 smithi031 ceph-mon[31237]: mgrmap e16: x(active, since 1.01096s) 2024-09-21T06:03:06.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:06.892 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:06.892 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:03:06.892 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:03:06.893 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:06 smithi107 ceph-mon[34380]: mgrmap e16: x(active, since 1.01096s) 2024-09-21T06:03:06.893 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:07.139 DEBUG:teuthology.orchestra.run.smithi107:mgr.y> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T06:03:07.141 DEBUG:teuthology.orchestra.run.smithi110:mgr.z> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T06:03:07.143 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T06:03:07.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T06:03:07.146 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T06:03:07.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T06:03:07.188 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T06:03:07.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T06:03:07.231 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T06:03:07.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T06:03:07.274 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T06:03:07.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T06:03:07.318 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T06:03:07.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T06:03:07.361 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T06:03:07.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T06:03:07.405 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T06:03:07.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T06:03:07.448 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T06:03:07.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T06:03:07.495 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T06:03:07.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T06:03:07.498 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:07.542 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T06:03:07.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T06:03:07.597 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T06:03:07.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T06:03:07.641 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T06:03:07.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T06:03:07.695 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T06:03:07.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T06:03:07.749 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T06:03:07.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T06:03:07.799 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T06:03:07.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T06:03:07.853 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T06:03:07.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T06:03:07.866 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:07.874 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T06:03:07.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T06:03:07.935 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T06:03:07.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T06:03:07.991 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T06:03:07.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T06:03:08.035 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T06:03:08.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T06:03:08.090 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T06:03:08.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T06:03:08.178 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T06:03:08.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T06:03:08.226 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T06:03:08.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T06:03:08.280 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T06:03:08.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T06:03:08.331 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T06:03:08.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T06:03:08.331 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:08.399 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T06:03:08.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: Saving service mgr spec with placement smithi031=x;smithi107=y;smithi110=z;count:3 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.400 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:08.450 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T06:03:08.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T06:03:08.504 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T06:03:08.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T06:03:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: Saving service mgr spec with placement smithi031=x;smithi107=y;smithi110=z;count:3 2024-09-21T06:03:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: Saving service mgr spec with placement smithi031=x;smithi107=y;smithi110=z;count:3 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:08.561 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T06:03:08.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T06:03:08.613 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T06:03:08.613 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi031') 2024-09-21T06:03:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:09 smithi031 ceph-mon[31237]: mgrmap e17: x(active, since 2s) 2024-09-21T06:03:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:09.626 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:09 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/798100034' entity='client.admin' 2024-09-21T06:03:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:09 smithi110 ceph-mon[34550]: mgrmap e17: x(active, since 2s) 2024-09-21T06:03:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:09 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/798100034' entity='client.admin' 2024-09-21T06:03:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:09 smithi107 ceph-mon[34380]: mgrmap e17: x(active, since 2s) 2024-09-21T06:03:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:09 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/798100034' entity='client.admin' 2024-09-21T06:03:10.109 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi031') 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 -- 172.21.15.31:0/2170349692 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39cc03fb70 msgr2=0x7f39cc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 --2- 172.21.15.31:0/2170349692 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39cc03fb70 0x7f39cc042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009880 tx=0x7f39d402f680 comp rx=0 tx=0).stop 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 -- 172.21.15.31:0/2170349692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e4105180 msgr2=0x7f39e419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 --2- 172.21.15.31:0/2170349692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e4105180 0x7f39e419af60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00d750 tx=0x7f39dc00dc10 comp rx=0 tx=0).stop 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 -- 172.21.15.31:0/2170349692 shutdown_connections 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 --2- 172.21.15.31:0/2170349692 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39cc03fb70 0x7f39cc042080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 --2- 172.21.15.31:0/2170349692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39e4105180 0x7f39e419af60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 --2- 172.21.15.31:0/2170349692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39e41047e0 0x7f39e419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 --2- 172.21.15.31:0/2170349692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39e4103750 0x7f39e4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.521+0000 7f39ebe31640 1 -- 172.21.15.31:0/2170349692 >> 172.21.15.31:0/2170349692 conn(0x7f39e40ff0e0 msgr2=0x7f39e4107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:10.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.522+0000 7f39ebe31640 1 -- 172.21.15.31:0/2170349692 shutdown_connections 2024-09-21T06:03:10.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:10.522+0000 7f39ebe31640 1 -- 172.21.15.31:0/2170349692 wait complete. 2024-09-21T06:03:10.532 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.534 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.537 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.538 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.539 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.550 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:10 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1889037671' entity='client.admin' 2024-09-21T06:03:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:10 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1889037671' entity='client.admin' 2024-09-21T06:03:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:10.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:10 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1889037671' entity='client.admin' 2024-09-21T06:03:10.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:10.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:10.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:10.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:10.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:11.297 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi031') 2024-09-21T06:03:11.519 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:03:11.176+0000 7f24c6c7a640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:03:11] ENGINE Error in HTTPServer.serve 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: Traceback (most recent call last): 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._connections.run(self.expiration_interval) 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._run(expiration_interval) 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:03:11.523 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: s = self.context.wrap_socket( 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: return self.sslsocket_class._create( 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self.do_handshake() 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: self._sslobj.do_handshake() 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:03:11.524 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2170349692' entity='client.admin' 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.720 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: [21/Sep/2024:06:03:10] ENGINE Bus STARTING 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: [21/Sep/2024:06:03:11] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.721 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: Deploying daemon mgr.z on smithi110 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: [21/Sep/2024:06:03:11] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: [21/Sep/2024:06:03:11] ENGINE Bus STARTED 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:11 smithi110 ceph-mon[34550]: [21/Sep/2024:06:03:11] ENGINE Error in HTTPServer.serve 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout: Traceback (most recent call last): 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:03:11.722 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: self._run(expiration_interval) 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: s = self.context.wrap_socket( 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: return self.sslsocket_class._create( 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: self.do_handshake() 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: self._sslobj.do_handshake() 2024-09-21T06:03:11.723 INFO:journalctl@ceph.mon.c.smithi110.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2170349692' entity='client.admin' 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: [21/Sep/2024:06:03:10] ENGINE Bus STARTING 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: [21/Sep/2024:06:03:11] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: Deploying daemon mgr.z on smithi110 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: [21/Sep/2024:06:03:11] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: [21/Sep/2024:06:03:11] ENGINE Bus STARTED 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:11 smithi107 ceph-mon[34380]: [21/Sep/2024:06:03:11] ENGINE Error in HTTPServer.serve 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout: Traceback (most recent call last): 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout: self._run(expiration_interval) 2024-09-21T06:03:11.792 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: s = self.context.wrap_socket( 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: return self.sslsocket_class._create( 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: self.do_handshake() 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: self._sslobj.do_handshake() 2024-09-21T06:03:11.793 INFO:journalctl@ceph.mon.b.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:03:11.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-21T06:03:11.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-21T06:03:11.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi110:/etc/ceph/ceph.conf 2024-09-21T06:03:11.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2170349692' entity='client.admin' 2024-09-21T06:03:11.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Updating smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: [21/Sep/2024:06:03:10] ENGINE Bus STARTING 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: [21/Sep/2024:06:03:11] ENGINE Serving on http://172.21.15.31:8765 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: Deploying daemon mgr.z on smithi110 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: [21/Sep/2024:06:03:11] ENGINE Serving on https://172.21.15.31:7150 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: [21/Sep/2024:06:03:11] ENGINE Bus STARTED 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:11 smithi031 ceph-mon[31237]: [21/Sep/2024:06:03:11] ENGINE Error in HTTPServer.serve 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout: Traceback (most recent call last): 2024-09-21T06:03:11.848 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._run(expiration_interval) 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: s = self.context.wrap_socket( 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: return self.sslsocket_class._create( 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: self.do_handshake() 2024-09-21T06:03:11.849 INFO:journalctl@ceph.mon.a.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:03:11.850 INFO:journalctl@ceph.mon.a.smithi031.stdout: self._sslobj.do_handshake() 2024-09-21T06:03:11.850 INFO:journalctl@ceph.mon.a.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:03:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 -- 172.21.15.31:0/4015069498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 msgr2=0x7f8a94102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:11.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/4015069498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a94102e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009900 tx=0x7f8a8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 -- 172.21.15.31:0/4015069498 shutdown_connections 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/4015069498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a94108cf0 0x7f8a9410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/4015069498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a941083a0 0x7f8a941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/4015069498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a94102e70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.849+0000 7f8a9aec0640 1 -- 172.21.15.31:0/4015069498 >> 172.21.15.31:0/4015069498 conn(0x7f8a94079730 msgr2=0x7f8a94079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.850+0000 7f8a9aec0640 1 -- 172.21.15.31:0/4015069498 shutdown_connections 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.850+0000 7f8a9aec0640 1 -- 172.21.15.31:0/4015069498 wait complete. 2024-09-21T06:03:11.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 Processor -- start 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 -- start start 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a9419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a941083a0 0x7f8a9419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a94108cf0 0x7f8a9419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94112720 con 0x7f8a94102a60 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941125a0 con 0x7f8a941083a0 2024-09-21T06:03:11.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a9aec0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a941128a0 con 0x7f8a94108cf0 2024-09-21T06:03:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a98c5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a9419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a98c5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a9419b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43456/0 (socket says 172.21.15.31:43456) 2024-09-21T06:03:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.851+0000 7f8a98c5c640 1 -- 172.21.15.31:0/3149855198 learned_addr learned my addr 172.21.15.31:0/3149855198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a9945d640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a94108cf0 0x7f8a9419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a93fff640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a941083a0 0x7f8a9419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a98c5c640 1 -- 172.21.15.31:0/3149855198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a94108cf0 msgr2=0x7f8a9419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a98c5c640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a94108cf0 0x7f8a9419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a98c5c640 1 -- 172.21.15.31:0/3149855198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a941083a0 msgr2=0x7f8a9419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a98c5c640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a941083a0 0x7f8a9419b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a98c5c640 1 -- 172.21.15.31:0/3149855198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a941a6330 con 0x7f8a94102a60 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a98c5c640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a9419b130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c005940 tx=0x7f8a8c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c03f070 con 0x7f8a94102a60 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f8a8c031b20 con 0x7f8a94102a60 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.852+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941a65c0 con 0x7f8a94102a60 2024-09-21T06:03:11.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.853+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c035030 con 0x7f8a94102a60 2024-09-21T06:03:11.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.853+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941a6aa0 con 0x7f8a94102a60 2024-09-21T06:03:11.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.854+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f8a8c042070 con 0x7f8a94102a60 2024-09-21T06:03:11.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.854+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94102e70 con 0x7f8a94102a60 2024-09-21T06:03:11.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.854+0000 7f8a91ffb640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a8003fba0 0x7f8a800420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.855+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8a8c074ec0 con 0x7f8a94102a60 2024-09-21T06:03:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.855+0000 7f8a93fff640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a8003fba0 0x7f8a800420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:11.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.855+0000 7f8a93fff640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a8003fba0 0x7f8a800420b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c005e10 tx=0x7f8a7c005da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:11.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:11.861+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a8c042360 con 0x7f8a94102a60 2024-09-21T06:03:12.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.051+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f8a9410aac0 con 0x7f8a94102a60 2024-09-21T06:03:12.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.056+0000 7f8a91ffb640 1 -- 172.21.15.31:0/3149855198 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v13)=0 v13) ==== 117+0+0 (secure 0 0 0) 0x7f8a8c03ae10 con 0x7f8a94102a60 2024-09-21T06:03:12.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a8003fba0 msgr2=0x7f8a800420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a8003fba0 0x7f8a800420b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c005e10 tx=0x7f8a7c005da0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 msgr2=0x7f8a9419b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a9419b130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c005940 tx=0x7f8a8c003c10 comp rx=0 tx=0).stop 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 shutdown_connections 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a94108cf0 0x7f8a9419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a941083a0 0x7f8a9419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a8003fba0 0x7f8a800420b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 --2- 172.21.15.31:0/3149855198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a94102a60 0x7f8a9419b130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 >> 172.21.15.31:0/3149855198 conn(0x7f8a94079730 msgr2=0x7f8a94101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 shutdown_connections 2024-09-21T06:03:12.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.058+0000 7f8a9aec0640 1 -- 172.21.15.31:0/3149855198 wait complete. 2024-09-21T06:03:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 -- 172.21.15.31:0/304838676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 msgr2=0x7f12641098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 --2- 172.21.15.31:0/304838676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f12641098b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1258009900 tx=0x7f125802a410 comp rx=0 tx=0).stop 2024-09-21T06:03:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 -- 172.21.15.31:0/304838676 shutdown_connections 2024-09-21T06:03:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 --2- 172.21.15.31:0/304838676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264109df0 0x7f1264110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 --2- 172.21.15.31:0/304838676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f12641098b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 --2- 172.21.15.31:0/304838676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1264103b60 0x7f1264103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 -- 172.21.15.31:0/304838676 >> 172.21.15.31:0/304838676 conn(0x7f126407b850 msgr2=0x7f126407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 -- 172.21.15.31:0/304838676 shutdown_connections 2024-09-21T06:03:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.543+0000 7f12691d9640 1 -- 172.21.15.31:0/304838676 wait complete. 2024-09-21T06:03:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.544+0000 7f12691d9640 1 Processor -- start 2024-09-21T06:03:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.544+0000 7f12691d9640 1 -- start start 2024-09-21T06:03:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.544+0000 7f12691d9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264103b60 0x7f126419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.544+0000 7f12691d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f126419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.544+0000 7f12691d9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1264109df0 0x7f126419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f12691d9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12641138c0 con 0x7f12641094a0 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f12691d9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1264113740 con 0x7f1264103b60 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f12691d9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1264113a40 con 0x7f1264109df0 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f126419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f126419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43474/0 (socket says 172.21.15.31:43474) 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 -- 172.21.15.31:0/2520325809 learned_addr learned my addr 172.21.15.31:0/2520325809 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126359e640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1264109df0 0x7f126419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f1262d9d640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264103b60 0x7f126419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 -- 172.21.15.31:0/2520325809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1264109df0 msgr2=0x7f126419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1264109df0 0x7f126419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 -- 172.21.15.31:0/2520325809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264103b60 msgr2=0x7f126419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264103b60 0x7f126419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 -- 172.21.15.31:0/2520325809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641a0220 con 0x7f12641094a0 2024-09-21T06:03:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f126259c640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f126419a850 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1258005ab0 tx=0x7f1258003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125803f070 con 0x7f12641094a0 2024-09-21T06:03:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f1258031d20 con 0x7f12641094a0 2024-09-21T06:03:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.545+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12641a04b0 con 0x7f12641094a0 2024-09-21T06:03:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.546+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258035030 con 0x7f12641094a0 2024-09-21T06:03:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.546+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12641a6c70 con 0x7f12641094a0 2024-09-21T06:03:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.546+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f1258042070 con 0x7f12641094a0 2024-09-21T06:03:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.547+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1264103f70 con 0x7f12641094a0 2024-09-21T06:03:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.547+0000 7f124bfff640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f124c03fb70 0x7f124c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.547+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1258074d20 con 0x7f12641094a0 2024-09-21T06:03:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.547+0000 7f1262d9d640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f124c03fb70 0x7f124c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.547+0000 7f1262d9d640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f124c03fb70 0x7f124c042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1254005fd0 tx=0x7f1254009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:12.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.552+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1258036df0 con 0x7f12641094a0 2024-09-21T06:03:12.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.744+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f126410bb70 con 0x7f12641094a0 2024-09-21T06:03:12.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.748+0000 7f124bfff640 1 -- 172.21.15.31:0/2520325809 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v14)=0 v14) ==== 116+0+0 (secure 0 0 0) 0x7f125803b4f0 con 0x7f12641094a0 2024-09-21T06:03:12.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f124c03fb70 msgr2=0x7f124c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f124c03fb70 0x7f124c042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1254005fd0 tx=0x7f1254009760 comp rx=0 tx=0).stop 2024-09-21T06:03:12.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 msgr2=0x7f126419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:12.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f126419a850 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1258005ab0 tx=0x7f1258003cd0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 shutdown_connections 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1264109df0 0x7f126419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f124c03fb70 0x7f124c042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12641094a0 0x7f126419a850 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 --2- 172.21.15.31:0/2520325809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1264103b60 0x7f126419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 >> 172.21.15.31:0/2520325809 conn(0x7f126407b850 msgr2=0x7f1264107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 shutdown_connections 2024-09-21T06:03:12.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:12.751+0000 7f12691d9640 1 -- 172.21.15.31:0/2520325809 wait complete. 2024-09-21T06:03:13.028 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:12 smithi110 systemd[1]: Starting Ceph mgr.z for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:03:13.115 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:13 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3149855198' entity='client.admin' 2024-09-21T06:03:13.115 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:13 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2520325809' entity='client.admin' 2024-09-21T06:03:13.290 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:13 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3149855198' entity='client.admin' 2024-09-21T06:03:13.291 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:13 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2520325809' entity='client.admin' 2024-09-21T06:03:13.291 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 podman[36510]: 2024-09-21 06:03:12.964278171 +0000 UTC m=+0.015580426 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:03:13.291 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 podman[36510]: 2024-09-21 06:03:13.06579125 +0000 UTC m=+0.117093505 container create 7b6497934dcbf6abef57a30141aaff5158dd600836a11c870ed406e2d754a84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-z, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T06:03:13.291 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 podman[36510]: 2024-09-21 06:03:13.25143733 +0000 UTC m=+0.302739585 container init 7b6497934dcbf6abef57a30141aaff5158dd600836a11c870ed406e2d754a84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T06:03:13.291 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 podman[36510]: 2024-09-21 06:03:13.254849731 +0000 UTC m=+0.306151989 container start 7b6497934dcbf6abef57a30141aaff5158dd600836a11c870ed406e2d754a84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-z, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:03:13.291 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 ceph-mgr[36528]: -- 172.21.15.110:0/3121213028 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa7149f0e0 con 0x55fa76242000 2024-09-21T06:03:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:13 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3149855198' entity='client.admin' 2024-09-21T06:03:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:13 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2520325809' entity='client.admin' 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.617+0000 7fdaa87b3640 1 -- 172.21.15.31:0/1188878344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 msgr2=0x7fdaa0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.617+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/1188878344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa0110640 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda98009ca0 tx=0x7fda9802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 -- 172.21.15.31:0/1188878344 shutdown_connections 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/1188878344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa0110640 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/1188878344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa01094c0 0x7fdaa01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/1188878344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaa0101fe0 0x7fdaa01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 -- 172.21.15.31:0/1188878344 >> 172.21.15.31:0/1188878344 conn(0x7fdaa0077bc0 msgr2=0x7fdaa0078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 -- 172.21.15.31:0/1188878344 shutdown_connections 2024-09-21T06:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 -- 172.21.15.31:0/1188878344 wait complete. 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 Processor -- start 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.618+0000 7fdaa87b3640 1 -- start start 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa87b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaa0101fe0 0x7fdaa019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa87b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa01094c0 0x7fdaa019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa87b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa87b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01137f0 con 0x7fdaa0109e10 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa87b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0113670 con 0x7fdaa01094c0 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa87b3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0113970 con 0x7fdaa0101fe0 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:13.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa019faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43496/0 (socket says 172.21.15.31:43496) 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 -- 172.21.15.31:0/839217153 learned_addr learned my addr 172.21.15.31:0/839217153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa654f640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaa0101fe0 0x7fdaa019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa5d4e640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa01094c0 0x7fdaa019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 -- 172.21.15.31:0/839217153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaa0101fe0 msgr2=0x7fdaa019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaa0101fe0 0x7fdaa019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 -- 172.21.15.31:0/839217153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa01094c0 msgr2=0x7fdaa019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa01094c0 0x7fdaa019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.619+0000 7fdaa6d50640 1 -- 172.21.15.31:0/839217153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa01a01d0 con 0x7fdaa0109e10 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.620+0000 7fdaa6d50640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa019faa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda98006010 tx=0x7fda980093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:13.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.620+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98003cf0 con 0x7fdaa0109e10 2024-09-21T06:03:13.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.620+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fda980044b0 con 0x7fdaa0109e10 2024-09-21T06:03:13.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.620+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa01a0460 con 0x7fdaa0109e10 2024-09-21T06:03:13.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.620+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98030a50 con 0x7fdaa0109e10 2024-09-21T06:03:13.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.620+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a6c20 con 0x7fdaa0109e10 2024-09-21T06:03:13.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.621+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fda98030c30 con 0x7fdaa0109e10 2024-09-21T06:03:13.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.621+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa01023f0 con 0x7fdaa0109e10 2024-09-21T06:03:13.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.621+0000 7fda977fe640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fda8c03fb70 0x7fda8c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:13.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.621+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fda980758b0 con 0x7fdaa0109e10 2024-09-21T06:03:13.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.621+0000 7fdaa654f640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fda8c03fb70 0x7fda8c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:13.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.622+0000 7fdaa654f640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fda8c03fb70 0x7fda8c042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005d00 tx=0x7fda9c00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:13.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.626+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda9803b410 con 0x7fdaa0109e10 2024-09-21T06:03:13.778 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 bash[36510]: 7b6497934dcbf6abef57a30141aaff5158dd600836a11c870ed406e2d754a84d 2024-09-21T06:03:13.778 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:03:13 smithi110 systemd[1]: Started Ceph mgr.z for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:03:13.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.817+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fdaa010bbe0 con 0x7fdaa0109e10 2024-09-21T06:03:13.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.821+0000 7fda977fe640 1 -- 172.21.15.31:0/839217153 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v15)=0 v15) ==== 108+0+0 (secure 0 0 0) 0x7fda98044cc0 con 0x7fdaa0109e10 2024-09-21T06:03:13.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fda8c03fb70 msgr2=0x7fda8c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fda8c03fb70 0x7fda8c042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005d00 tx=0x7fda9c00b410 comp rx=0 tx=0).stop 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 msgr2=0x7fdaa019faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa019faa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda98006010 tx=0x7fda980093e0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 shutdown_connections 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fda8c03fb70 0x7fda8c042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaa0109e10 0x7fdaa019faa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa01094c0 0x7fdaa019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 --2- 172.21.15.31:0/839217153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaa0101fe0 0x7fdaa019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 >> 172.21.15.31:0/839217153 conn(0x7fdaa0077bc0 msgr2=0x7fdaa0107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 shutdown_connections 2024-09-21T06:03:13.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:13.823+0000 7fdaa87b3640 1 -- 172.21.15.31:0/839217153 wait complete. 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: Deploying daemon mgr.y on smithi107 2024-09-21T06:03:14.727 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:14 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/839217153' entity='client.admin' 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:14.741 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:14.742 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: Deploying daemon mgr.y on smithi107 2024-09-21T06:03:14.742 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:14 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/839217153' entity='client.admin' 2024-09-21T06:03:14.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:14.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:14.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:03:14.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:14.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:14.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: Deploying daemon mgr.y on smithi107 2024-09-21T06:03:14.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:14 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/839217153' entity='client.admin' 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 -- 172.21.15.31:0/74099380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff048109df0 msgr2=0x7ff048110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 --2- 172.21.15.31:0/74099380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff048109df0 0x7ff048110620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff040009ca0 tx=0x7ff04002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 -- 172.21.15.31:0/74099380 shutdown_connections 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 --2- 172.21.15.31:0/74099380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff048109df0 0x7ff048110620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 --2- 172.21.15.31:0/74099380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0481094a0 0x7ff0481098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 --2- 172.21.15.31:0/74099380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048103b60 0x7ff048103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:14.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 -- 172.21.15.31:0/74099380 >> 172.21.15.31:0/74099380 conn(0x7ff04807b850 msgr2=0x7ff04807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 -- 172.21.15.31:0/74099380 shutdown_connections 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 -- 172.21.15.31:0/74099380 wait complete. 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.864+0000 7ff05001c640 1 Processor -- start 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 -- start start 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff048103b60 0x7ff04819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 0x7ff04819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048109df0 0x7ff04819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff048113820 con 0x7ff0481094a0 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481136a0 con 0x7ff048103b60 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff05001c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff0481139a0 con 0x7ff048109df0 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 0x7ff04819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:14.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 0x7ff04819a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43514/0 (socket says 172.21.15.31:43514) 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 -- 172.21.15.31:0/390089137 learned_addr learned my addr 172.21.15.31:0/390089137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04ddb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff048103b60 0x7ff04819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04e5b9640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048109df0 0x7ff04819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 -- 172.21.15.31:0/390089137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048109df0 msgr2=0x7ff04819faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048109df0 0x7ff04819faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 -- 172.21.15.31:0/390089137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff048103b60 msgr2=0x7ff04819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff048103b60 0x7ff04819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04d5b7640 1 -- 172.21.15.31:0/390089137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0481a0220 con 0x7ff0481094a0 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04e5b9640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048109df0 0x7ff04819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:03:14.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.865+0000 7ff04ddb8640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff048103b60 0x7ff04819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.866+0000 7ff04d5b7640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 0x7ff04819a850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d740 tx=0x7ff03800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.866+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038017070 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.866+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0481a0510 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.866+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7ff03800de50 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.866+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03800f620 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.866+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0481a6c70 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.867+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7ff03801a070 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.867+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff048103f70 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.867+0000 7ff03effd640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff03403fb70 0x7ff034042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.867+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff038053f70 con 0x7ff0481094a0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.867+0000 7ff04ddb8640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff03403fb70 0x7ff034042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:14.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.868+0000 7ff04ddb8640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff03403fb70 0x7ff034042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff044005d00 tx=0x7ff04400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:14.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:14.872+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff03800fdd0 con 0x7ff0481094a0 2024-09-21T06:03:15.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/651539851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 msgr2=0x7fcd241087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/651539851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 0x7fcd241087b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009900 tx=0x7fcd1402a410 comp rx=0 tx=0).stop 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/651539851 shutdown_connections 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/651539851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 0x7fcd2410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/651539851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 0x7fcd241087b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/651539851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd24102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/651539851 >> 172.21.15.31:0/651539851 conn(0x7fcd24079730 msgr2=0x7fcd24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/651539851 shutdown_connections 2024-09-21T06:03:15.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.044+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/651539851 wait complete. 2024-09-21T06:03:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.045+0000 7fcd2bfa8640 1 Processor -- start 2024-09-21T06:03:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.045+0000 7fcd2bfa8640 1 -- start start 2024-09-21T06:03:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2bfa8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd240ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2bfa8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 0x7fcd240ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2bfa8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 0x7fcd241000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:15.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2bfa8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24112770 con 0x7fcd24102a60 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2bfa8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241125f0 con 0x7fcd24108cf0 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2bfa8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241128f0 con 0x7fcd241083a0 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd240ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd240ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43534/0 (socket says 172.21.15.31:43534) 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 -- 172.21.15.31:0/509621592 learned_addr learned my addr 172.21.15.31:0/509621592 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2a545640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 0x7fcd241000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29543640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 0x7fcd240ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 -- 172.21.15.31:0/509621592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 msgr2=0x7fcd240ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 0x7fcd240ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 -- 172.21.15.31:0/509621592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 msgr2=0x7fcd241000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 0x7fcd241000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd29d44640 1 -- 172.21.15.31:0/509621592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241a8590 con 0x7fcd24102a60 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.046+0000 7fcd2a545640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 0x7fcd241000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.047+0000 7fcd29d44640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd240ff670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e9f0 tx=0x7fcd2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.047+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000ccc0 con 0x7fcd24102a60 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.047+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241a8880 con 0x7fcd24102a60 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.047+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fcd200108a0 con 0x7fcd24102a60 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.047+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20018680 con 0x7fcd24102a60 2024-09-21T06:03:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.047+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241a8de0 con 0x7fcd24102a60 2024-09-21T06:03:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.048+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fcd200188e0 con 0x7fcd24102a60 2024-09-21T06:03:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.048+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd24102e70 con 0x7fcd24102a60 2024-09-21T06:03:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.049+0000 7fcd1affd640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd1003fb70 0x7fcd10042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.049+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcd20054b20 con 0x7fcd24102a60 2024-09-21T06:03:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.049+0000 7fcd29543640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd1003fb70 0x7fcd10042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:15.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.049+0000 7fcd29543640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd1003fb70 0x7fcd10042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd14031440 tx=0x7fcd14005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:15.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.054+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd200071c0 con 0x7fcd24102a60 2024-09-21T06:03:15.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.071+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ff04810bb70 con 0x7ff0481094a0 2024-09-21T06:03:15.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.074+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7ff038010e10 con 0x7ff0481094a0 2024-09-21T06:03:15.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.075+0000 7ff03effd640 1 -- 172.21.15.31:0/390089137 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff038009c30 con 0x7ff0481094a0 2024-09-21T06:03:15.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.075+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd2005a080 con 0x7fcd24102a60 2024-09-21T06:03:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff03403fb70 msgr2=0x7ff034042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff03403fb70 0x7ff034042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff044005d00 tx=0x7ff04400b410 comp rx=0 tx=0).stop 2024-09-21T06:03:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 msgr2=0x7ff04819a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 0x7ff04819a850 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff03800d740 tx=0x7ff03800dc00 comp rx=0 tx=0).stop 2024-09-21T06:03:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 shutdown_connections 2024-09-21T06:03:15.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff048109df0 0x7ff04819faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff03403fb70 0x7ff034042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0481094a0 0x7ff04819a850 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 --2- 172.21.15.31:0/390089137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff048103b60 0x7ff04819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 >> 172.21.15.31:0/390089137 conn(0x7ff04807b850 msgr2=0x7ff048107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:15.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 shutdown_connections 2024-09-21T06:03:15.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.077+0000 7ff05001c640 1 -- 172.21.15.31:0/390089137 wait complete. 2024-09-21T06:03:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.251+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fcd2410baf0 con 0x7fcd24102a60 2024-09-21T06:03:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.255+0000 7fcd1affd640 1 -- 172.21.15.31:0/509621592 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v17)=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7fcd20010d50 con 0x7fcd24102a60 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd1003fb70 msgr2=0x7fcd10042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd1003fb70 0x7fcd10042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd14031440 tx=0x7fcd14005a20 comp rx=0 tx=0).stop 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 msgr2=0x7fcd240ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd240ff670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e9f0 tx=0x7fcd2000edf0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 shutdown_connections 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd24108cf0 0x7fcd241000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd241083a0 0x7fcd240ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd1003fb70 0x7fcd10042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 --2- 172.21.15.31:0/509621592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd24102a60 0x7fcd240ff670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 >> 172.21.15.31:0/509621592 conn(0x7fcd24079730 msgr2=0x7fcd2406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 shutdown_connections 2024-09-21T06:03:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:15.258+0000 7fcd2bfa8640 1 -- 172.21.15.31:0/509621592 wait complete. 2024-09-21T06:03:15.789 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 06:03:15 smithi107 systemd[1]: Started Ceph mgr.y for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:03:16.111 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:16.111 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/390089137' entity='client.admin' 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/509621592' entity='client.admin' 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.393 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/390089137' entity='client.admin' 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/509621592' entity='client.admin' 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:16.394 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.321+0000 7f1458695640 1 -- 172.21.15.31:0/2870259802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 msgr2=0x7f1450112580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.321+0000 7f1458695640 1 --2- 172.21.15.31:0/2870259802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450112580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1448009ca0 tx=0x7f144802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 -- 172.21.15.31:0/2870259802 shutdown_connections 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 --2- 172.21.15.31:0/2870259802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450112580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 --2- 172.21.15.31:0/2870259802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450076080 0x7f145010fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 --2- 172.21.15.31:0/2870259802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450075730 0x7f1450075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 -- 172.21.15.31:0/2870259802 >> 172.21.15.31:0/2870259802 conn(0x7f14500ff610 msgr2=0x7f1450101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 -- 172.21.15.31:0/2870259802 shutdown_connections 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 -- 172.21.15.31:0/2870259802 wait complete. 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.322+0000 7f1458695640 1 Processor -- start 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 -- start start 2024-09-21T06:03:16.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450075730 0x7f14501075f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450076080 0x7f14501038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1450115b80 con 0x7f1450110160 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1450115a00 con 0x7f1450075730 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1458695640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1450115d00 con 0x7f1450076080 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1456c32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1456c32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450103ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43552/0 (socket says 172.21.15.31:43552) 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1456c32640 1 -- 172.21.15.31:0/2447234668 learned_addr learned my addr 172.21.15.31:0/2447234668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1456431640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450075730 0x7f14501075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1455c30640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450076080 0x7f14501038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1455c30640 1 -- 172.21.15.31:0/2447234668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450075730 msgr2=0x7f14501075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1455c30640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450075730 0x7f14501075f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1455c30640 1 -- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 msgr2=0x7f1450103ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1455c30640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450103ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.323+0000 7f1455c30640 1 -- 172.21.15.31:0/2447234668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14501acac0 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f1456431640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450075730 0x7f14501075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f1455c30640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450076080 0x7f14501038d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f145006aa90 tx=0x7f144000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144000cd00 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f14400035f0 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14501accf0 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440010690 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.324+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14501ad180 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.325+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1450075b40 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.325+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f1440010830 con 0x7f1450076080 2024-09-21T06:03:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.326+0000 7f14477fe640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f143c03fb70 0x7f143c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.326+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f14400548d0 con 0x7f1450076080 2024-09-21T06:03:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.326+0000 7f1456431640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f143c03fb70 0x7f143c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.326+0000 7f1456431640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f143c03fb70 0x7f143c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1450107340 tx=0x7f144c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.331+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1440016330 con 0x7f1450076080 2024-09-21T06:03:16.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/390089137' entity='client.admin' 2024-09-21T06:03:16.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/509621592' entity='client.admin' 2024-09-21T06:03:16.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:16.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:16.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.524+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f14501ad410 con 0x7f1450076080 2024-09-21T06:03:16.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.529+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1440013190 con 0x7f1450076080 2024-09-21T06:03:16.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.529+0000 7f14477fe640 1 -- 172.21.15.31:0/2447234668 <== mon.1 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) ==== 123+0+0 (secure 0 0 0) 0x7f1440003800 con 0x7f1450076080 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f143c03fb70 msgr2=0x7f143c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f143c03fb70 0x7f143c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1450107340 tx=0x7f144c005f90 comp rx=0 tx=0).stop 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450076080 msgr2=0x7f14501038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450076080 0x7f14501038d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f145006aa90 tx=0x7f144000ee50 comp rx=0 tx=0).stop 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 shutdown_connections 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f143c03fb70 0x7f143c042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1450110160 0x7f1450103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1450076080 0x7f14501038d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 --2- 172.21.15.31:0/2447234668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1450075730 0x7f14501075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:16.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 >> 172.21.15.31:0/2447234668 conn(0x7f14500ff610 msgr2=0x7f1450109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:16.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 shutdown_connections 2024-09-21T06:03:16.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.532+0000 7f1458695640 1 -- 172.21.15.31:0/2447234668 wait complete. 2024-09-21T06:03:16.831 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.033+0000 7fbcc47c9640 1 -- 172.21.15.31:0/3848286454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc102a80 msgr2=0x7fbcbc102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.033+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/3848286454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc102a80 0x7fbcbc102e90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009ca0 tx=0x7fbcac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.033+0000 7fbcc47c9640 1 -- 172.21.15.31:0/3848286454 shutdown_connections 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.033+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/3848286454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc108d10 0x7fbcbc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.034+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/3848286454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc1083c0 0x7fbcbc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.034+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/3848286454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc102a80 0x7fbcbc102e90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.034+0000 7fbcc47c9640 1 -- 172.21.15.31:0/3848286454 >> 172.21.15.31:0/3848286454 conn(0x7fbcbc079740 msgr2=0x7fbcbc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:17.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.034+0000 7fbcc47c9640 1 -- 172.21.15.31:0/3848286454 shutdown_connections 2024-09-21T06:03:17.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.034+0000 7fbcc47c9640 1 -- 172.21.15.31:0/3848286454 wait complete. 2024-09-21T06:03:17.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.035+0000 7fbcc47c9640 1 Processor -- start 2024-09-21T06:03:17.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.035+0000 7fbcc47c9640 1 -- start start 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.035+0000 7fbcc47c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc102a80 0x7fbcbc1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.035+0000 7fbcc47c9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc1083c0 0x7fbcbc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc47c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc108d10 0x7fbcbc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc47c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc112980 con 0x7fbcbc108d10 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc47c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc112800 con 0x7fbcbc102a80 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2565640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc102a80 0x7fbcbc1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2565640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc102a80 0x7fbcbc1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36434/0 (socket says 172.21.15.31:36434) 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2565640 1 -- 172.21.15.31:0/1373041096 learned_addr learned my addr 172.21.15.31:0/1373041096 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc112b00 con 0x7fbcbc1083c0 2024-09-21T06:03:17.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2d66640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc108d10 0x7fbcbc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc1d64640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc1083c0 0x7fbcbc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2d66640 1 -- 172.21.15.31:0/1373041096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc1083c0 msgr2=0x7fbcbc199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2d66640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc1083c0 0x7fbcbc199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2d66640 1 -- 172.21.15.31:0/1373041096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc102a80 msgr2=0x7fbcbc1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2d66640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc102a80 0x7fbcbc1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc1d64640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc1083c0 0x7fbcbc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.036+0000 7fbcc2d66640 1 -- 172.21.15.31:0/1373041096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc19f360 con 0x7fbcbc108d10 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.037+0000 7fbcc2d66640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc108d10 0x7fbcbc19ec30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400e970 tx=0x7fbcb400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.037+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400ccc0 con 0x7fbcbc108d10 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.037+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb4005020 con 0x7fbcbc108d10 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.037+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc19f650 con 0x7fbcbc108d10 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.038+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4004a00 con 0x7fbcbc108d10 2024-09-21T06:03:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.038+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc1a5db0 con 0x7fbcbc108d10 2024-09-21T06:03:17.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.039+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fbcb4004ba0 con 0x7fbcbc108d10 2024-09-21T06:03:17.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.039+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc102e90 con 0x7fbcbc108d10 2024-09-21T06:03:17.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.040+0000 7fbcbb7fe640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbca403fbc0 0x7fbca40420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.040+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbcb4054420 con 0x7fbcbc108d10 2024-09-21T06:03:17.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.040+0000 7fbcc2565640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbca403fbc0 0x7fbca40420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.041+0000 7fbcc2565640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbca403fbc0 0x7fbca40420d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbcac004af0 tx=0x7fbcac009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:17.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.046+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbcb4013190 con 0x7fbcbc108d10 2024-09-21T06:03:17.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 -- 172.21.15.31:0/686227104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5441083a0 msgr2=0x7fd5441087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 --2- 172.21.15.31:0/686227104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5441083a0 0x7fd5441087b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009900 tx=0x7fd52c02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:17.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 -- 172.21.15.31:0/686227104 shutdown_connections 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 --2- 172.21.15.31:0/686227104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd544108cf0 0x7fd54410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 --2- 172.21.15.31:0/686227104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5441083a0 0x7fd5441087b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 --2- 172.21.15.31:0/686227104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd544102a60 0x7fd544102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.405+0000 7fd549319640 1 -- 172.21.15.31:0/686227104 >> 172.21.15.31:0/686227104 conn(0x7fd544079730 msgr2=0x7fd544079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.406+0000 7fd549319640 1 -- 172.21.15.31:0/686227104 shutdown_connections 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.406+0000 7fd549319640 1 -- 172.21.15.31:0/686227104 wait complete. 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 Processor -- start 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 -- start start 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd544102a60 0x7fd54419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 0x7fd54419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd544108cf0 0x7fd54419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd544112890 con 0x7fd544108cf0 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd544112710 con 0x7fd544102a60 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd549319640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd544112a10 con 0x7fd5441083a0 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 0x7fd54419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd5437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd544108cf0 0x7fd54419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd53bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 0x7fd54419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55542/0 (socket says 172.21.15.31:55542) 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd53bfff640 1 -- 172.21.15.31:0/2612277877 learned_addr learned my addr 172.21.15.31:0/2612277877 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.407+0000 7fd542ffd640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd544102a60 0x7fd54419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd53bfff640 1 -- 172.21.15.31:0/2612277877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd544102a60 msgr2=0x7fd54419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd53bfff640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd544102a60 0x7fd54419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd53bfff640 1 -- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd544108cf0 msgr2=0x7fd54419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd53bfff640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd544108cf0 0x7fd54419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd53bfff640 1 -- 172.21.15.31:0/2612277877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5441a03a0 con 0x7fd5441083a0 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd5437fe640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd544108cf0 0x7fd54419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:17.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd53bfff640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 0x7fd54419a9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd52c02a930 tx=0x7fd52c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c03f070 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5441a0630 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd52c03baf0 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c035030 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.408+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5441a6df0 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.409+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd544102e70 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.410+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fd52c042070 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.410+0000 7fd540ff9640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd53003fb70 0x7fd530042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.410+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd52c074c80 con 0x7fd5441083a0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.410+0000 7fd542ffd640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd53003fb70 0x7fd530042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.411+0000 7fd542ffd640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd53003fb70 0x7fd530042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd534009bd0 tx=0x7fd534005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:17.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.415+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd52c03ae40 con 0x7fd5441083a0 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.529+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd52c031400 con 0x7fd5441083a0 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.614+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fd5441a7020 con 0x7fd5441083a0 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.619+0000 7fd540ff9640 1 -- 172.21.15.31:0/2612277877 <== mon.1 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) ==== 106+0+0 (secure 0 0 0) 0x7fd52c03bca0 con 0x7fd5441083a0 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd53003fb70 msgr2=0x7fd530042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd53003fb70 0x7fd530042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd534009bd0 tx=0x7fd534005f90 comp rx=0 tx=0).stop 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 msgr2=0x7fd54419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 0x7fd54419a9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd52c02a930 tx=0x7fd52c003b90 comp rx=0 tx=0).stop 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 shutdown_connections 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd53003fb70 0x7fd530042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd544108cf0 0x7fd54419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5441083a0 0x7fd54419a9d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 --2- 172.21.15.31:0/2612277877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd544102a60 0x7fd54419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 >> 172.21.15.31:0/2612277877 conn(0x7fd544079730 msgr2=0x7fd544101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:17.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 shutdown_connections 2024-09-21T06:03:17.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:16.621+0000 7fd549319640 1 -- 172.21.15.31:0/2612277877 wait complete. 2024-09-21T06:03:17.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.235+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fbcbc02e640 con 0x7fbcbc108d10 2024-09-21T06:03:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.239+0000 7fbcbb7fe640 1 -- 172.21.15.31:0/1373041096 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v20)=0 v20) ==== 127+0+0 (secure 0 0 0) 0x7fbcb40051d0 con 0x7fbcbc108d10 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbca403fbc0 msgr2=0x7fbca40420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbca403fbc0 0x7fbca40420d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbcac004af0 tx=0x7fbcac009360 comp rx=0 tx=0).stop 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc108d10 msgr2=0x7fbcbc19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc108d10 0x7fbcbc19ec30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400e970 tx=0x7fbcb400ee30 comp rx=0 tx=0).stop 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 shutdown_connections 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbca403fbc0 0x7fbca40420d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcbc108d10 0x7fbcbc19ec30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcbc1083c0 0x7fbcbc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 --2- 172.21.15.31:0/1373041096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbcbc102a80 0x7fbcbc1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 >> 172.21.15.31:0/1373041096 conn(0x7fbcbc079740 msgr2=0x7fbcbc101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 shutdown_connections 2024-09-21T06:03:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:17.241+0000 7fbcc47c9640 1 -- 172.21.15.31:0/1373041096 wait complete. 2024-09-21T06:03:17.574 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:17.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='client.? ' entity='client.admin' 2024-09-21T06:03:17.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='client.? ' entity='client.admin' 2024-09-21T06:03:17.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: Reconfiguring daemon mgr.x on smithi031 2024-09-21T06:03:17.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:17 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1373041096' entity='client.admin' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='client.? ' entity='client.admin' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='client.? ' entity='client.admin' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:03:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:17.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:17.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:17.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: Reconfiguring daemon mgr.x on smithi031 2024-09-21T06:03:17.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:17 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1373041096' entity='client.admin' 2024-09-21T06:03:18.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='client.? ' entity='client.admin' 2024-09-21T06:03:18.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='client.? ' entity='client.admin' 2024-09-21T06:03:18.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:18.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: Reconfiguring daemon mgr.x on smithi031 2024-09-21T06:03:18.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:17 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1373041096' entity='client.admin' 2024-09-21T06:03:18.561 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:18 smithi031 ceph-mon[31237]: Standby manager daemon z started 2024-09-21T06:03:18.561 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:18 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:03:18.561 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:18 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:03:18.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:18 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:03:18.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:18 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:03:18.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:18 smithi107 ceph-mon[34380]: Standby manager daemon z started 2024-09-21T06:03:18.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:18 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:03:18.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:18 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:03:18.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:18 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:03:18.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:18 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:03:19.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:18 smithi110 ceph-mon[34550]: Standby manager daemon z started 2024-09-21T06:03:19.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:18 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:03:19.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:18 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:03:19.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:18 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:03:19.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:18 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.110:0/322261330' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:03:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:19 smithi031 ceph-mon[31237]: mgrmap e18: x(active, since 13s), standbys: z 2024-09-21T06:03:19.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:03:19.998 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T06:03:19.998 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi031') 2024-09-21T06:03:19.998 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi031') 2024-09-21T06:03:19.999 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi031') 2024-09-21T06:03:19.999 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi031') 2024-09-21T06:03:19.999 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi031') 2024-09-21T06:03:19.999 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi031') 2024-09-21T06:03:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:19 smithi110 ceph-mon[34550]: mgrmap e18: x(active, since 13s), standbys: z 2024-09-21T06:03:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:03:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:19 smithi107 ceph-mon[34380]: mgrmap e18: x(active, since 13s), standbys: z 2024-09-21T06:03:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:03:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 -- 172.21.15.31:0/2373515835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba4102a60 msgr2=0x7fcba4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/2373515835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba4102a60 0x7fcba4102e70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009900 tx=0x7fcb8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 -- 172.21.15.31:0/2373515835 shutdown_connections 2024-09-21T06:03:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/2373515835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcba4108cf0 0x7fcba410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/2373515835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba41083a0 0x7fcba41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/2373515835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba4102a60 0x7fcba4102e70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 -- 172.21.15.31:0/2373515835 >> 172.21.15.31:0/2373515835 conn(0x7fcba4079730 msgr2=0x7fcba4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 -- 172.21.15.31:0/2373515835 shutdown_connections 2024-09-21T06:03:20.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.573+0000 7fcba9a0f640 1 -- 172.21.15.31:0/2373515835 wait complete. 2024-09-21T06:03:20.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:20 smithi031 ceph-mon[31237]: Standby manager daemon y started 2024-09-21T06:03:20.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:20 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:03:20.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:20 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:03:20.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:20 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:03:20.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:20 smithi031 ceph-mon[31237]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:03:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 Processor -- start 2024-09-21T06:03:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 -- start start 2024-09-21T06:03:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 0x7fcba419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba41083a0 0x7fcba419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:20.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcba4108cf0 0x7fcba41a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4113a80 con 0x7fcba41083a0 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4113900 con 0x7fcba4102a60 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba9a0f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4113c00 con 0x7fcba4108cf0 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 0x7fcba419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba41083a0 0x7fcba419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba41083a0 0x7fcba419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56128/0 (socket says 172.21.15.31:56128) 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.574+0000 7fcba2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 0x7fcba419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55734/0 (socket says 172.21.15.31:55734) 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba27fc640 1 -- 172.21.15.31:0/1945853179 learned_addr learned my addr 172.21.15.31:0/1945853179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba37fe640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcba4108cf0 0x7fcba41a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba2ffd640 1 -- 172.21.15.31:0/1945853179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcba4108cf0 msgr2=0x7fcba41a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba2ffd640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcba4108cf0 0x7fcba41a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba2ffd640 1 -- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba41083a0 msgr2=0x7fcba419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:20.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba2ffd640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba41083a0 0x7fcba419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba2ffd640 1 -- 172.21.15.31:0/1945853179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba41a2540 con 0x7fcba4102a60 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba2ffd640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 0x7fcba419c690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c006010 tx=0x7fcb8c004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c03f070 con 0x7fcba4102a60 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb8c003ca0 con 0x7fcba4102a60 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c035070 con 0x7fcba4102a60 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba41a27d0 con 0x7fcba4102a60 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.575+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba41a2d00 con 0x7fcba4102a60 2024-09-21T06:03:20.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.576+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba4102e70 con 0x7fcba4102a60 2024-09-21T06:03:20.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.578+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fcb8c03b460 con 0x7fcba4102a60 2024-09-21T06:03:20.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.579+0000 7fcba8a0d640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb900b6580 0x7fcb900b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:20.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.579+0000 7fcba27fc640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb900b6580 0x7fcb900b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:20.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.579+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcb8c105b70 con 0x7fcba4102a60 2024-09-21T06:03:20.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.579+0000 7fcba27fc640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb900b6580 0x7fcb900b8a90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcba4100c50 tx=0x7fcb94005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:20.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.583+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcba41a2d00 con 0x7fcba4102a60 2024-09-21T06:03:20.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.773+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fcba419d7c0 con 0x7fcba4102a60 2024-09-21T06:03:20.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.775+0000 7fcba8a0d640 1 -- 172.21.15.31:0/1945853179 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v20)=0 v20) ==== 108+0+0 (secure 0 0 0) 0x7fcb8c0cdfe0 con 0x7fcba4102a60 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb900b6580 msgr2=0x7fcb900b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb900b6580 0x7fcb900b8a90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcba4100c50 tx=0x7fcb94005f90 comp rx=0 tx=0).stop 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 msgr2=0x7fcba419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 0x7fcba419c690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c006010 tx=0x7fcb8c004060 comp rx=0 tx=0).stop 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 shutdown_connections 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcba4108cf0 0x7fcba41a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb900b6580 0x7fcb900b8a90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcba41083a0 0x7fcba419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 --2- 172.21.15.31:0/1945853179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcba4102a60 0x7fcba419c690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 >> 172.21.15.31:0/1945853179 conn(0x7fcba4079730 msgr2=0x7fcba41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 shutdown_connections 2024-09-21T06:03:20.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.779+0000 7fcba9a0f640 1 -- 172.21.15.31:0/1945853179 wait complete. 2024-09-21T06:03:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:20 smithi110 ceph-mon[34550]: Standby manager daemon y started 2024-09-21T06:03:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:20 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:03:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:20 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:03:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:20 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:03:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:20 smithi110 ceph-mon[34550]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:03:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:20 smithi107 ceph-mon[34380]: Standby manager daemon y started 2024-09-21T06:03:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:20 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:03:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:20 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:03:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:20 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:03:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:20 smithi107 ceph-mon[34380]: from='mgr.? 172.21.15.107:0/2785197776' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:03:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 -- 172.21.15.31:0/3103792085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57941083a0 msgr2=0x7f57941087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 --2- 172.21.15.31:0/3103792085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57941083a0 0x7f57941087b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5788009900 tx=0x7f578802a410 comp rx=0 tx=0).stop 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 -- 172.21.15.31:0/3103792085 shutdown_connections 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 --2- 172.21.15.31:0/3103792085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5794108cf0 0x7f579410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 --2- 172.21.15.31:0/3103792085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57941083a0 0x7f57941087b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 --2- 172.21.15.31:0/3103792085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f5794102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 -- 172.21.15.31:0/3103792085 >> 172.21.15.31:0/3103792085 conn(0x7f5794079730 msgr2=0x7f5794079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 -- 172.21.15.31:0/3103792085 shutdown_connections 2024-09-21T06:03:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 -- 172.21.15.31:0/3103792085 wait complete. 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.131+0000 7f579a0a0640 1 Processor -- start 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 -- start start 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f579419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57941083a0 0x7f579419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 0x7f579419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5794113a80 con 0x7f5794108cf0 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5794113900 con 0x7f5794102a60 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f579a0a0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5794113c00 con 0x7f57941083a0 2024-09-21T06:03:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 0x7f579419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f57937fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f579419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 0x7f579419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56176/0 (socket says 172.21.15.31:56176) 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f57937fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f579419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55782/0 (socket says 172.21.15.31:55782) 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 -- 172.21.15.31:0/2412873341 learned_addr learned my addr 172.21.15.31:0/2412873341 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5792ffd640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57941083a0 0x7f579419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 -- 172.21.15.31:0/2412873341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57941083a0 msgr2=0x7f579419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57941083a0 0x7f579419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 -- 172.21.15.31:0/2412873341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 msgr2=0x7f579419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f579419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f5793fff640 1 -- 172.21.15.31:0/2412873341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57941a03e0 con 0x7f5794108cf0 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.132+0000 7f57937fe640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f579419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.133+0000 7f5793fff640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 0x7f579419fcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f578c00eaa0 tx=0x7f578c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:21.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.133+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c00cdb0 con 0x7f5794108cf0 2024-09-21T06:03:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.133+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f578c005020 con 0x7f5794108cf0 2024-09-21T06:03:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.133+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c0116d0 con 0x7f5794108cf0 2024-09-21T06:03:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.133+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57941a06d0 con 0x7f5794108cf0 2024-09-21T06:03:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.133+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941a6e30 con 0x7f5794108cf0 2024-09-21T06:03:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.134+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5794103ce0 con 0x7f5794108cf0 2024-09-21T06:03:21.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.136+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f578c0051d0 con 0x7f5794108cf0 2024-09-21T06:03:21.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.137+0000 7f5790ff9640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f577c0b6550 0x7f577c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.137+0000 7f57937fe640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f577c0b6550 0x7f577c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.137+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f578c0e5b70 con 0x7f5794108cf0 2024-09-21T06:03:21.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.137+0000 7f57937fe640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f577c0b6550 0x7f577c0b8a60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5784009fc0 tx=0x7f5784005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:21.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.140+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f578c016030 con 0x7f5794108cf0 2024-09-21T06:03:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.331+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f5794102e70 con 0x7f5794108cf0 2024-09-21T06:03:21.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.337+0000 7f5790ff9640 1 -- 172.21.15.31:0/2412873341 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v22)=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f5794102e70 con 0x7f5794108cf0 2024-09-21T06:03:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f577c0b6550 msgr2=0x7f577c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f577c0b6550 0x7f577c0b8a60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5784009fc0 tx=0x7f5784005f90 comp rx=0 tx=0).stop 2024-09-21T06:03:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 msgr2=0x7f579419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 0x7f579419fcb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f578c00eaa0 tx=0x7f578c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:03:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 shutdown_connections 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f577c0b6550 0x7f577c0b8a60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5794108cf0 0x7f579419fcb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57941083a0 0x7f579419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 --2- 172.21.15.31:0/2412873341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5794102a60 0x7f579419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.339+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 >> 172.21.15.31:0/2412873341 conn(0x7f5794079730 msgr2=0x7f57941052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.340+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 shutdown_connections 2024-09-21T06:03:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.340+0000 7f579a0a0640 1 -- 172.21.15.31:0/2412873341 wait complete. 2024-09-21T06:03:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:21 smithi031 ceph-mon[31237]: mgrmap e19: x(active, since 15s), standbys: z, y 2024-09-21T06:03:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:03:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:21 smithi031 ceph-mon[31237]: from='client.? ' entity='client.admin' 2024-09-21T06:03:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:21 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2412873341' entity='client.admin' 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/2541497038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 msgr2=0x7f5d1410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/2541497038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 0x7f5d1410d890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009900 tx=0x7f5d0402a410 comp rx=0 tx=0).stop 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/2541497038 shutdown_connections 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/2541497038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1410de50 0x7f5d14110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/2541497038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 0x7f5d1410d890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/2541497038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d14104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/2541497038 >> 172.21.15.31:0/2541497038 conn(0x7f5d140797e0 msgr2=0x7f5d14079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:21.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/2541497038 shutdown_connections 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.810+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/2541497038 wait complete. 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 Processor -- start 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 -- start start 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d1419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 0x7f5d1419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1410de50 0x7f5d1419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14113ab0 con 0x7f5d14105340 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14113930 con 0x7f5d1410de50 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.811+0000 7f5d1c1ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14113c30 con 0x7f5d141049f0 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d1419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d1419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42178/0 (socket says 172.21.15.31:42178) 2024-09-21T06:03:21.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 -- 172.21.15.31:0/3482852150 learned_addr learned my addr 172.21.15.31:0/3482852150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19755640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 0x7f5d1419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d1a757640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1410de50 0x7f5d1419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 -- 172.21.15.31:0/3482852150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1410de50 msgr2=0x7f5d1419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1410de50 0x7f5d1419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 -- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 msgr2=0x7f5d1419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 0x7f5d1419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.812+0000 7f5d19f56640 1 -- 172.21.15.31:0/3482852150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d141a0410 con 0x7f5d141049f0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.813+0000 7f5d19f56640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d1419a4a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000e9c0 tx=0x7f5d1000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.813+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000cd80 con 0x7f5d141049f0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.813+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d100035f0 con 0x7f5d141049f0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.813+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d141a0700 con 0x7f5d141049f0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.813+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10010690 con 0x7f5d141049f0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.813+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141a6e60 con 0x7f5d141049f0 2024-09-21T06:03:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.814+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d14104e00 con 0x7f5d141049f0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.816+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f5d10004000 con 0x7f5d141049f0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.816+0000 7f5d0affd640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d000b64d0 0x7f5d000b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.817+0000 7f5d19755640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d000b64d0 0x7f5d000b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.817+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5d100e5df0 con 0x7f5d141049f0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.817+0000 7f5d19755640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d000b64d0 0x7f5d000b89e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009230 tx=0x7f5d04031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:20.820+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d100ae1e0 con 0x7f5d141049f0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.009+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f5d141089c0 con 0x7f5d141049f0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.018+0000 7f5d0affd640 1 -- 172.21.15.31:0/3482852150 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v21)=0 v21) ==== 111+0+0 (secure 0 0 0) 0x7f5d100b25c0 con 0x7f5d141049f0 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d000b64d0 msgr2=0x7f5d000b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d000b64d0 0x7f5d000b89e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009230 tx=0x7f5d04031400 comp rx=0 tx=0).stop 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 msgr2=0x7f5d1419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d1419a4a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000e9c0 tx=0x7f5d1000ee80 comp rx=0 tx=0).stop 2024-09-21T06:03:21.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 shutdown_connections 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d1410de50 0x7f5d1419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d000b64d0 0x7f5d000b89e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d14105340 0x7f5d1419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 --2- 172.21.15.31:0/3482852150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d141049f0 0x7f5d1419a4a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 >> 172.21.15.31:0/3482852150 conn(0x7f5d140797e0 msgr2=0x7f5d141008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 shutdown_connections 2024-09-21T06:03:21.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:21.020+0000 7f5d1c1ba640 1 -- 172.21.15.31:0/3482852150 wait complete. 2024-09-21T06:03:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:21 smithi110 ceph-mon[34550]: mgrmap e19: x(active, since 15s), standbys: z, y 2024-09-21T06:03:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:03:22.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:21 smithi110 ceph-mon[34550]: from='client.? ' entity='client.admin' 2024-09-21T06:03:22.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:21 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2412873341' entity='client.admin' 2024-09-21T06:03:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:21 smithi107 ceph-mon[34380]: mgrmap e19: x(active, since 15s), standbys: z, y 2024-09-21T06:03:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:03:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:21 smithi107 ceph-mon[34380]: from='client.? ' entity='client.admin' 2024-09-21T06:03:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:21 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2412873341' entity='client.admin' 2024-09-21T06:03:23.115 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.124 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.138 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:23.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi031') 2024-09-21T06:03:23.663 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:24.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:24.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:24.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:24.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:24.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:24.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:24.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:24.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:24.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:24.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 -- 172.21.15.31:0/3982517184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94102a60 msgr2=0x7fcf94102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/3982517184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94102a60 0x7fcf94102e70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009900 tx=0x7fcf8402a410 comp rx=0 tx=0).stop 2024-09-21T06:03:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 -- 172.21.15.31:0/3982517184 shutdown_connections 2024-09-21T06:03:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/3982517184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf94108cf0 0x7fcf9410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/3982517184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf941083a0 0x7fcf941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:24.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/3982517184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94102a60 0x7fcf94102e70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 -- 172.21.15.31:0/3982517184 >> 172.21.15.31:0/3982517184 conn(0x7fcf94079730 msgr2=0x7fcf94079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.906+0000 7fcf9a96b640 1 -- 172.21.15.31:0/3982517184 shutdown_connections 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.907+0000 7fcf9a96b640 1 -- 172.21.15.31:0/3982517184 wait complete. 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.907+0000 7fcf9a96b640 1 Processor -- start 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 -- start start 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf94102a60 0x7fcf9419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf941083a0 0x7fcf9419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 0x7fcf9419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:24.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94112960 con 0x7fcf94108cf0 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941127e0 con 0x7fcf941083a0 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf9a96b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94112ae0 con 0x7fcf94102a60 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf98f08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 0x7fcf9419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf98f08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 0x7fcf9419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56198/0 (socket says 172.21.15.31:56198) 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf98f08640 1 -- 172.21.15.31:0/1496281696 learned_addr learned my addr 172.21.15.31:0/1496281696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf93fff640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf94102a60 0x7fcf9419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.908+0000 7fcf937fe640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf941083a0 0x7fcf9419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf98f08640 1 -- 172.21.15.31:0/1496281696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf94102a60 msgr2=0x7fcf9419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf98f08640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf94102a60 0x7fcf9419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:24.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf98f08640 1 -- 172.21.15.31:0/1496281696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf941083a0 msgr2=0x7fcf9419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf98f08640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf941083a0 0x7fcf9419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf98f08640 1 -- 172.21.15.31:0/1496281696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf941a6490 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf98f08640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 0x7fcf9419bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00eaa0 tx=0x7fcf8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00cdb0 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf941a6780 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf8c005020 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0116d0 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.909+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a6cc0 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.910+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf94102e70 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.912+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fcf8c0051d0 con 0x7fcf94108cf0 2024-09-21T06:03:24.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.913+0000 7fcf917fa640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcf7c0b64d0 0x7fcf7c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:24.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.913+0000 7fcf93fff640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcf7c0b64d0 0x7fcf7c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:24.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.913+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcf8c0e59a0 con 0x7fcf94108cf0 2024-09-21T06:03:24.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.914+0000 7fcf93fff640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcf7c0b64d0 0x7fcf7c0b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000c00 tx=0x7fcf84031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:24.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.917+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf8c016030 con 0x7fcf94108cf0 2024-09-21T06:03:25.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.111+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fcf9402e640 con 0x7fcf94108cf0 2024-09-21T06:03:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.986+0000 7fd5379af640 1 -- 172.21.15.31:0/251470651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd530108cf0 msgr2=0x7fd53010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.986+0000 7fd5379af640 1 --2- 172.21.15.31:0/251470651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd530108cf0 0x7fd53010f520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd520009900 tx=0x7fd52002a410 comp rx=0 tx=0).stop 2024-09-21T06:03:25.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 -- 172.21.15.31:0/251470651 shutdown_connections 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 --2- 172.21.15.31:0/251470651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd530108cf0 0x7fd53010f520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 --2- 172.21.15.31:0/251470651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5301083a0 0x7fd5301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 --2- 172.21.15.31:0/251470651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd530102a60 0x7fd530102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 -- 172.21.15.31:0/251470651 >> 172.21.15.31:0/251470651 conn(0x7fd530079730 msgr2=0x7fd530079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 -- 172.21.15.31:0/251470651 shutdown_connections 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.987+0000 7fd5379af640 1 -- 172.21.15.31:0/251470651 wait complete. 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 Processor -- start 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 -- start start 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd530102a60 0x7fd53019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 0x7fd53019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd530108cf0 0x7fd53019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd530112960 con 0x7fd5301083a0 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5301127e0 con 0x7fd530108cf0 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd5379af640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd530112ae0 con 0x7fd530102a60 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 0x7fd53019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 0x7fd53019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56222/0 (socket says 172.21.15.31:56222) 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 -- 172.21.15.31:0/2919876750 learned_addr learned my addr 172.21.15.31:0/2919876750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd535f4c640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd530108cf0 0x7fd53019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd53574b640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd530102a60 0x7fd53019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 -- 172.21.15.31:0/2919876750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd530102a60 msgr2=0x7fd53019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd530102a60 0x7fd53019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 -- 172.21.15.31:0/2919876750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd530108cf0 msgr2=0x7fd53019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd530108cf0 0x7fd53019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.988+0000 7fd534f4a640 1 -- 172.21.15.31:0/2919876750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5301a0350 con 0x7fd5301083a0 2024-09-21T06:03:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.989+0000 7fd534f4a640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 0x7fd53019a980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd52400eb90 tx=0x7fd52400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.989+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524010630 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.989+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5301a0640 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.989+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd52400ee90 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.989+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524018ac0 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.989+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5301a6da0 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.990+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd530102e70 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.992+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fd524018c60 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.993+0000 7fd52e7fc640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6550 0x7fd5180b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.994+0000 7fd53574b640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6550 0x7fd5180b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.994+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd5240e6260 con 0x7fd5301083a0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.994+0000 7fd53574b640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6550 0x7fd5180b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd528005d00 tx=0x7fd528005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:24.998+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5240ae650 con 0x7fd5301083a0 2024-09-21T06:03:25.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.116+0000 7fcf917fa640 1 -- 172.21.15.31:0/1496281696 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fcf8c0add90 con 0x7fcf94108cf0 2024-09-21T06:03:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcf7c0b64d0 msgr2=0x7fcf7c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcf7c0b64d0 0x7fcf7c0b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000c00 tx=0x7fcf84031400 comp rx=0 tx=0).stop 2024-09-21T06:03:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 msgr2=0x7fcf9419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 0x7fcf9419bc90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00eaa0 tx=0x7fcf8c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:03:25.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 shutdown_connections 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcf7c0b64d0 0x7fcf7c0b89e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf94108cf0 0x7fcf9419bc90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf941083a0 0x7fcf9419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 --2- 172.21.15.31:0/1496281696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf94102a60 0x7fcf9419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.120+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 >> 172.21.15.31:0/1496281696 conn(0x7fcf94079730 msgr2=0x7fcf941013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.121+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 shutdown_connections 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.121+0000 7fcf9a96b640 1 -- 172.21.15.31:0/1496281696 wait complete. 2024-09-21T06:03:25.123 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:25.124 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.207+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd53019b510 con 0x7fd5301083a0 2024-09-21T06:03:25.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.211+0000 7fd52e7fc640 1 -- 172.21.15.31:0/2919876750 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fd53019b510 con 0x7fd5301083a0 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6550 msgr2=0x7fd5180b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6550 0x7fd5180b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd528005d00 tx=0x7fd528005c50 comp rx=0 tx=0).stop 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 msgr2=0x7fd53019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 0x7fd53019a980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd52400eb90 tx=0x7fd52400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 shutdown_connections 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd530108cf0 0x7fd53019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6550 0x7fd5180b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5301083a0 0x7fd53019a980 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 --2- 172.21.15.31:0/2919876750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd530102a60 0x7fd53019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 >> 172.21.15.31:0/2919876750 conn(0x7fd530079730 msgr2=0x7fd53006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 shutdown_connections 2024-09-21T06:03:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.214+0000 7fd5379af640 1 -- 172.21.15.31:0/2919876750 wait complete. 2024-09-21T06:03:25.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 -- 172.21.15.31:0/3506016153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 msgr2=0x7fba641026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 --2- 172.21.15.31:0/3506016153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba641026a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba54009900 tx=0x7fba5402a410 comp rx=0 tx=0).stop 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 -- 172.21.15.31:0/3506016153 shutdown_connections 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 --2- 172.21.15.31:0/3506016153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 0x7fba6410ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 --2- 172.21.15.31:0/3506016153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba64107bd0 0x7fba64107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 --2- 172.21.15.31:0/3506016153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba641026a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 -- 172.21.15.31:0/3506016153 >> 172.21.15.31:0/3506016153 conn(0x7fba64079ea0 msgr2=0x7fba6407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 -- 172.21.15.31:0/3506016153 shutdown_connections 2024-09-21T06:03:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.707+0000 7fba68a55640 1 -- 172.21.15.31:0/3506016153 wait complete. 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.708+0000 7fba68a55640 1 Processor -- start 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 -- start start 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba6419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba64107bd0 0x7fba6419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 0x7fba6419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba64112aa0 con 0x7fba64102290 2024-09-21T06:03:25.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba64112920 con 0x7fba64108520 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba68a55640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba64112c20 con 0x7fba64107bd0 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba6259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba6419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba6259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba6419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56234/0 (socket says 172.21.15.31:56234) 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba6259c640 1 -- 172.21.15.31:0/54078498 learned_addr learned my addr 172.21.15.31:0/54078498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba62d9d640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 0x7fba6419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.709+0000 7fba59d9b640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba64107bd0 0x7fba6419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba6259c640 1 -- 172.21.15.31:0/54078498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba64107bd0 msgr2=0x7fba6419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba6259c640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba64107bd0 0x7fba6419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba6259c640 1 -- 172.21.15.31:0/54078498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 msgr2=0x7fba6419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba6259c640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 0x7fba6419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba6259c640 1 -- 172.21.15.31:0/54078498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba641a04f0 con 0x7fba64102290 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba62d9d640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 0x7fba6419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba6259c640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba6419a580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba54005940 tx=0x7fba540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5403f030 con 0x7fba64102290 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba54003cb0 con 0x7fba64102290 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba641a0780 con 0x7fba64102290 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54035070 con 0x7fba64102290 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.710+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba641a6f40 con 0x7fba64102290 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.712+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba64103510 con 0x7fba64102290 2024-09-21T06:03:25.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.713+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fba54042030 con 0x7fba64102290 2024-09-21T06:03:25.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.714+0000 7fba5bfff640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba500b64d0 0x7fba500b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:25.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.714+0000 7fba59d9b640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba500b64d0 0x7fba500b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.714+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fba54105f10 con 0x7fba64102290 2024-09-21T06:03:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.714+0000 7fba59d9b640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba500b64d0 0x7fba500b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fba641049d0 tx=0x7fba4c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.719+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba540ce3a0 con 0x7fba64102290 2024-09-21T06:03:25.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.910+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fba6410b300 con 0x7fba64102290 2024-09-21T06:03:25.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.915+0000 7fba5bfff640 1 -- 172.21.15.31:0/54078498 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v25)=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fba540d2780 con 0x7fba64102290 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba500b64d0 msgr2=0x7fba500b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba500b64d0 0x7fba500b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fba641049d0 tx=0x7fba4c009110 comp rx=0 tx=0).stop 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 msgr2=0x7fba6419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba6419a580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba54005940 tx=0x7fba540042f0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 shutdown_connections 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba64108520 0x7fba6419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba64107bd0 0x7fba6419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba500b64d0 0x7fba500b89e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 --2- 172.21.15.31:0/54078498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba64102290 0x7fba6419a580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 >> 172.21.15.31:0/54078498 conn(0x7fba64079ea0 msgr2=0x7fba6410faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 shutdown_connections 2024-09-21T06:03:25.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:25.917+0000 7fba68a55640 1 -- 172.21.15.31:0/54078498 wait complete. 2024-09-21T06:03:26.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:26 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1496281696' entity='client.admin' 2024-09-21T06:03:26.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:26 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2919876750' entity='client.admin' 2024-09-21T06:03:26.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:26 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/54078498' entity='client.admin' 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.394+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2793681618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 msgr2=0x7f7c34110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.394+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2793681618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c34110320 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009ca0 tx=0x7f7c2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2793681618 shutdown_connections 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2793681618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c34110320 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2793681618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34105320 0x7f7c3410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2793681618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c341049d0 0x7f7c34104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2793681618 >> 172.21.15.31:0/2793681618 conn(0x7f7c34079740 msgr2=0x7f7c34079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2793681618 shutdown_connections 2024-09-21T06:03:26.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2793681618 wait complete. 2024-09-21T06:03:26.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 Processor -- start 2024-09-21T06:03:26.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.395+0000 7f7c39fe0640 1 -- start start 2024-09-21T06:03:26.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c39fe0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c341049d0 0x7f7c3419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:26.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c39fe0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c34105320 0x7f7c3419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:26.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c39fe0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c3419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c39fe0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34113a90 con 0x7f7c3410de30 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c39fe0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34113910 con 0x7f7c341049d0 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c39fe0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34113c10 con 0x7f7c34105320 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c3419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c3419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56250/0 (socket says 172.21.15.31:56250) 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 -- 172.21.15.31:0/2125035115 learned_addr learned my addr 172.21.15.31:0/2125035115 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c2affd640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c34105320 0x7f7c3419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c337fe640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c341049d0 0x7f7c3419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 -- 172.21.15.31:0/2125035115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c34105320 msgr2=0x7f7c3419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c34105320 0x7f7c3419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 -- 172.21.15.31:0/2125035115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c341049d0 msgr2=0x7f7c3419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.396+0000 7f7c33fff640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c341049d0 0x7f7c3419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c33fff640 1 -- 172.21.15.31:0/2125035115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c341a0450 con 0x7f7c3410de30 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c2affd640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c34105320 0x7f7c3419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c33fff640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c3419fd20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0098d0 tx=0x7f7c2c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c003b70 con 0x7f7c3410de30 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c2c003d10 con 0x7f7c3410de30 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c341a06e0 con 0x7f7c3410de30 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c02fa20 con 0x7f7c3410de30 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.397+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a6ea0 con 0x7f7c3410de30 2024-09-21T06:03:26.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.398+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3406c240 con 0x7f7c3410de30 2024-09-21T06:03:26.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.400+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f7c2c02fbe0 con 0x7f7c3410de30 2024-09-21T06:03:26.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.401+0000 7f7c317fa640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7c200b64d0 0x7f7c200b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:26.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.401+0000 7f7c337fe640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7c200b64d0 0x7f7c200b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:26.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.401+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7c2c106bf0 con 0x7f7c3410de30 2024-09-21T06:03:26.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.401+0000 7f7c337fe640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7c200b64d0 0x7f7c200b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c3406c940 tx=0x7f7c2400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:26.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.404+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c2c0cefe0 con 0x7f7c3410de30 2024-09-21T06:03:26.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:26 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1496281696' entity='client.admin' 2024-09-21T06:03:26.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:26 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2919876750' entity='client.admin' 2024-09-21T06:03:26.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:26 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/54078498' entity='client.admin' 2024-09-21T06:03:26.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:26 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1496281696' entity='client.admin' 2024-09-21T06:03:26.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:26 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2919876750' entity='client.admin' 2024-09-21T06:03:26.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:26 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/54078498' entity='client.admin' 2024-09-21T06:03:26.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.602+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f7c34104de0 con 0x7f7c3410de30 2024-09-21T06:03:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.608+0000 7f7c317fa640 1 -- 172.21.15.31:0/2125035115 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) ==== 120+0+0 (secure 0 0 0) 0x7f7c2c0d33c0 con 0x7f7c3410de30 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.610+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7c200b64d0 msgr2=0x7f7c200b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.610+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7c200b64d0 0x7f7c200b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c3406c940 tx=0x7f7c2400b390 comp rx=0 tx=0).stop 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.610+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 msgr2=0x7f7c3419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.610+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c3419fd20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0098d0 tx=0x7f7c2c0317b0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 shutdown_connections 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7c200b64d0 0x7f7c200b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c3410de30 0x7f7c3419fd20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c34105320 0x7f7c3419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 --2- 172.21.15.31:0/2125035115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c341049d0 0x7f7c3419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 >> 172.21.15.31:0/2125035115 conn(0x7f7c34079740 msgr2=0x7f7c340ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 shutdown_connections 2024-09-21T06:03:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.611+0000 7f7c39fe0640 1 -- 172.21.15.31:0/2125035115 wait complete. 2024-09-21T06:03:27.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 -- 172.21.15.31:0/248441584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f4108cf0 msgr2=0x7ff5f410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/248441584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f4108cf0 0x7ff5f410f520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009ca0 tx=0x7ff5ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 -- 172.21.15.31:0/248441584 shutdown_connections 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/248441584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f4108cf0 0x7ff5f410f520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/248441584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f41083a0 0x7ff5f41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/248441584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f4102a60 0x7ff5f4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 -- 172.21.15.31:0/248441584 >> 172.21.15.31:0/248441584 conn(0x7ff5f4079730 msgr2=0x7ff5f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 -- 172.21.15.31:0/248441584 shutdown_connections 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.574+0000 7ff5f92fc640 1 -- 172.21.15.31:0/248441584 wait complete. 2024-09-21T06:03:27.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 Processor -- start 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 -- start start 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f4102a60 0x7ff5f419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 0x7ff5f419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f4108cf0 0x7ff5f41a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f41129b0 con 0x7ff5f41083a0 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4112830 con 0x7ff5f4102a60 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.575+0000 7ff5f92fc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f4112b30 con 0x7ff5f4108cf0 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 0x7ff5f419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 0x7ff5f419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56270/0 (socket says 172.21.15.31:56270) 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 -- 172.21.15.31:0/1948287495 learned_addr learned my addr 172.21.15.31:0/1948287495 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f37fe640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f4108cf0 0x7ff5f41a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f2ffd640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f4102a60 0x7ff5f419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 -- 172.21.15.31:0/1948287495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f4108cf0 msgr2=0x7ff5f41a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f4108cf0 0x7ff5f41a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 -- 172.21.15.31:0/1948287495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f4102a60 msgr2=0x7ff5f419c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f4102a60 0x7ff5f419c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 -- 172.21.15.31:0/1948287495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f41a2560 con 0x7ff5f41083a0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f27fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 0x7ff5f419cb90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800e9c0 tx=0x7ff5e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800cd80 con 0x7ff5f41083a0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5e80054d0 con 0x7ff5f41083a0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8010430 con 0x7ff5f41083a0 2024-09-21T06:03:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.576+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f41a2850 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.577+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f41a8fb0 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.578+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f4102e70 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.580+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7ff5e8004fb0 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.581+0000 7ff5dbfff640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff5dc0b64d0 0x7ff5dc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.581+0000 7ff5f2ffd640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff5dc0b64d0 0x7ff5dc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.581+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff5e80e5d50 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.581+0000 7ff5f2ffd640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff5dc0b64d0 0x7ff5dc0b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400bbe0 tx=0x7ff5e400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.585+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5e8056710 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.779+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7ff5f419d720 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.779+0000 7ff5dbfff640 1 -- 172.21.15.31:0/1948287495 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v26)=0 v26) ==== 106+0+0 (secure 0 0 0) 0x7ff5f419d720 con 0x7ff5f41083a0 2024-09-21T06:03:27.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff5dc0b64d0 msgr2=0x7ff5dc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff5dc0b64d0 0x7ff5dc0b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400bbe0 tx=0x7ff5e400b390 comp rx=0 tx=0).stop 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 msgr2=0x7ff5f419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 0x7ff5f419cb90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800e9c0 tx=0x7ff5e800ee80 comp rx=0 tx=0).stop 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 shutdown_connections 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff5f4108cf0 0x7ff5f41a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff5dc0b64d0 0x7ff5dc0b89e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5f41083a0 0x7ff5f419cb90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 --2- 172.21.15.31:0/1948287495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f4102a60 0x7ff5f419c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.781+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 >> 172.21.15.31:0/1948287495 conn(0x7ff5f4079730 msgr2=0x7ff5f41042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.782+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 shutdown_connections 2024-09-21T06:03:27.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:26.782+0000 7ff5f92fc640 1 -- 172.21.15.31:0/1948287495 wait complete. 2024-09-21T06:03:27.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:27 smithi031 ceph-mon[31237]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:27.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:27 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2125035115' entity='client.admin' 2024-09-21T06:03:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:27 smithi110 ceph-mon[34550]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:27 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2125035115' entity='client.admin' 2024-09-21T06:03:27.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:27 smithi107 ceph-mon[34380]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:27.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:27 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2125035115' entity='client.admin' 2024-09-21T06:03:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.847+0000 7fd38d343640 1 -- 172.21.15.31:0/3986108805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388103bb0 msgr2=0x7fd388103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.847+0000 7fd38d343640 1 --2- 172.21.15.31:0/3986108805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388103bb0 0x7fd388103fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd378009ca0 tx=0x7fd37802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.847+0000 7fd38d343640 1 -- 172.21.15.31:0/3986108805 shutdown_connections 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 --2- 172.21.15.31:0/3986108805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388109e40 0x7fd388110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 --2- 172.21.15.31:0/3986108805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3881094f0 0x7fd388109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 --2- 172.21.15.31:0/3986108805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd388103bb0 0x7fd388103fc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 -- 172.21.15.31:0/3986108805 >> 172.21.15.31:0/3986108805 conn(0x7fd3880ff600 msgr2=0x7fd388101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 -- 172.21.15.31:0/3986108805 shutdown_connections 2024-09-21T06:03:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 -- 172.21.15.31:0/3986108805 wait complete. 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 Processor -- start 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.848+0000 7fd38d343640 1 -- start start 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd38d343640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd388103bb0 0x7fd388075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd38d343640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 0x7fd388075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd38d343640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388109e40 0x7fd388076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd38d343640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113b00 con 0x7fd3881094f0 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd38d343640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113980 con 0x7fd388109e40 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd38d343640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113c80 con 0x7fd388103bb0 2024-09-21T06:03:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 0x7fd388075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 0x7fd388075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56294/0 (socket says 172.21.15.31:56294) 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 -- 172.21.15.31:0/1898509951 learned_addr learned my addr 172.21.15.31:0/1898509951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd386ffd640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd388103bb0 0x7fd388075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3877fe640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388109e40 0x7fd388076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 -- 172.21.15.31:0/1898509951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd388103bb0 msgr2=0x7fd388075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd388103bb0 0x7fd388075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 -- 172.21.15.31:0/1898509951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388109e40 msgr2=0x7fd388076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388109e40 0x7fd388076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 -- 172.21.15.31:0/1898509951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3881a8700 con 0x7fd3881094f0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.849+0000 7fd3867fc640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 0x7fd388075c50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d950 tx=0x7fd37c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.850+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c017070 con 0x7fd3881094f0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.850+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37c009c30 con 0x7fd3881094f0 2024-09-21T06:03:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.850+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881a89f0 con 0x7fd3881094f0 2024-09-21T06:03:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.850+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0047a0 con 0x7fd3881094f0 2024-09-21T06:03:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.850+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a8f80 con 0x7fd3881094f0 2024-09-21T06:03:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.851+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd388103fc0 con 0x7fd3881094f0 2024-09-21T06:03:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.852+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fd37c004020 con 0x7fd3881094f0 2024-09-21T06:03:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.853+0000 7fd36ffff640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3700b6550 0x7fd3700b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.853+0000 7fd386ffd640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3700b6550 0x7fd3700b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.854+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd37c0e5810 con 0x7fd3881094f0 2024-09-21T06:03:27.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.854+0000 7fd386ffd640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3700b6550 0x7fd3700b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd378005e30 tx=0x7fd378009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:27.856+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd37c020020 con 0x7fd3881094f0 2024-09-21T06:03:28.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.049+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fd388076a10 con 0x7fd3881094f0 2024-09-21T06:03:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.053+0000 7fd36ffff640 1 -- 172.21.15.31:0/1898509951 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v27)=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fd388076a10 con 0x7fd3881094f0 2024-09-21T06:03:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3700b6550 msgr2=0x7fd3700b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3700b6550 0x7fd3700b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd378005e30 tx=0x7fd378009360 comp rx=0 tx=0).stop 2024-09-21T06:03:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 msgr2=0x7fd388075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 0x7fd388075c50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d950 tx=0x7fd37c00de10 comp rx=0 tx=0).stop 2024-09-21T06:03:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 shutdown_connections 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388109e40 0x7fd388076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3700b6550 0x7fd3700b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3881094f0 0x7fd388075c50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 --2- 172.21.15.31:0/1898509951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd388103bb0 0x7fd388075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 >> 172.21.15.31:0/1898509951 conn(0x7fd3880ff600 msgr2=0x7fd388101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 shutdown_connections 2024-09-21T06:03:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:28.056+0000 7fd38d343640 1 -- 172.21.15.31:0/1898509951 wait complete. 2024-09-21T06:03:29.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:29 smithi031 ceph-mon[31237]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:29.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:29 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1898509951' entity='client.admin' 2024-09-21T06:03:29.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:29.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:29.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:29.304 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:29 smithi110 ceph-mon[34550]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:29 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1898509951' entity='client.admin' 2024-09-21T06:03:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:29 smithi107 ceph-mon[34380]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:29 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1898509951' entity='client.admin' 2024-09-21T06:03:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:29.930 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi031') 2024-09-21T06:03:29.930 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi031') 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.658+0000 7f560fa0c640 1 -- 172.21.15.31:0/4232182105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5608102a60 msgr2=0x7f5608102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.658+0000 7f560fa0c640 1 --2- 172.21.15.31:0/4232182105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5608102a60 0x7f5608102e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5604009900 tx=0x7f560402a410 comp rx=0 tx=0).stop 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 -- 172.21.15.31:0/4232182105 shutdown_connections 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 --2- 172.21.15.31:0/4232182105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608108cf0 0x7f560810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 --2- 172.21.15.31:0/4232182105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56081083a0 0x7f56081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 --2- 172.21.15.31:0/4232182105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5608102a60 0x7f5608102e70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 -- 172.21.15.31:0/4232182105 >> 172.21.15.31:0/4232182105 conn(0x7f5608079730 msgr2=0x7f5608079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 -- 172.21.15.31:0/4232182105 shutdown_connections 2024-09-21T06:03:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.659+0000 7f560fa0c640 1 -- 172.21.15.31:0/4232182105 wait complete. 2024-09-21T06:03:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.660+0000 7f560fa0c640 1 Processor -- start 2024-09-21T06:03:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.660+0000 7f560fa0c640 1 -- start start 2024-09-21T06:03:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.660+0000 7f560fa0c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 0x7f560819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.660+0000 7f560fa0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56081083a0 0x7f560819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.660+0000 7f560fa0c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608108cf0 0x7f560819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560fa0c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5608112960 con 0x7f56081083a0 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560fa0c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56081127e0 con 0x7f5608108cf0 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560fa0c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5608112ae0 con 0x7f5608102a60 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 0x7f560819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 0x7f560819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55776/0 (socket says 172.21.15.31:55776) 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 -- 172.21.15.31:0/2241825159 learned_addr learned my addr 172.21.15.31:0/2241825159 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560cfa7640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56081083a0 0x7f560819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560dfa9640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608108cf0 0x7f560819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 -- 172.21.15.31:0/2241825159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608108cf0 msgr2=0x7f560819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608108cf0 0x7f560819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 -- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56081083a0 msgr2=0x7f560819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56081083a0 0x7f560819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 -- 172.21.15.31:0/2241825159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56081a6570 con 0x7f5608102a60 2024-09-21T06:03:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560cfa7640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56081083a0 0x7f560819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.661+0000 7f560d7a8640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 0x7f560819b2b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5604005940 tx=0x7f56040042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.662+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560403f030 con 0x7f5608102a60 2024-09-21T06:03:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.662+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56081a6800 con 0x7f5608102a60 2024-09-21T06:03:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.662+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5604003cb0 con 0x7f5608102a60 2024-09-21T06:03:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.662+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604035070 con 0x7f5608102a60 2024-09-21T06:03:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.662+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081a6ce0 con 0x7f5608102a60 2024-09-21T06:03:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.663+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5608102e70 con 0x7f5608102a60 2024-09-21T06:03:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.666+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f5604042030 con 0x7f5608102a60 2024-09-21T06:03:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.666+0000 7f55fe7fc640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55f40b64d0 0x7f55f40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.667+0000 7f560cfa7640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55f40b64d0 0x7f55f40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.667+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f56041061e0 con 0x7f5608102a60 2024-09-21T06:03:30.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.667+0000 7f560cfa7640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55f40b64d0 0x7f55f40b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5608100c50 tx=0x7f55f800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:30.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.669+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56040d2a50 con 0x7f5608102a60 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 -- 172.21.15.31:0/3012615332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe01049b0 msgr2=0x7fabe0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 --2- 172.21.15.31:0/3012615332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe01049b0 0x7fabe0104dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009900 tx=0x7fabdc02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 -- 172.21.15.31:0/3012615332 shutdown_connections 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 --2- 172.21.15.31:0/3012615332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe010de10 0x7fabe0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 --2- 172.21.15.31:0/3012615332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabe0105300 0x7fabe010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 --2- 172.21.15.31:0/3012615332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe01049b0 0x7fabe0104dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 -- 172.21.15.31:0/3012615332 >> 172.21.15.31:0/3012615332 conn(0x7fabe0079730 msgr2=0x7fabe0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 -- 172.21.15.31:0/3012615332 shutdown_connections 2024-09-21T06:03:30.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.932+0000 7fabe76d9640 1 -- 172.21.15.31:0/3012615332 wait complete. 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 Processor -- start 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 -- start start 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe01049b0 0x7fabe019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 0x7fabe019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabe010de10 0x7fabe019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0113ae0 con 0x7fabe0105300 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0113960 con 0x7fabe01049b0 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe76d9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0113c60 con 0x7fabe010de10 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe4c74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 0x7fabe019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe4c74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 0x7fabe019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47588/0 (socket says 172.21.15.31:47588) 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe4c74640 1 -- 172.21.15.31:0/2258991875 learned_addr learned my addr 172.21.15.31:0/2258991875 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:30.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe5c76640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabe010de10 0x7fabe019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.933+0000 7fabe5475640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe01049b0 0x7fabe019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe4c74640 1 -- 172.21.15.31:0/2258991875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabe010de10 msgr2=0x7fabe019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe4c74640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabe010de10 0x7fabe019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe4c74640 1 -- 172.21.15.31:0/2258991875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe01049b0 msgr2=0x7fabe019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe4c74640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe01049b0 0x7fabe019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe4c74640 1 -- 172.21.15.31:0/2258991875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe01a0420 con 0x7fabe0105300 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe4c74640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 0x7fabe019aa50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d750 tx=0x7fabd400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4017070 con 0x7fabe0105300 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe01a0710 con 0x7fabe0105300 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd40054d0 con 0x7fabe0105300 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004c30 con 0x7fabe0105300 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.934+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a6e70 con 0x7fabe0105300 2024-09-21T06:03:30.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.935+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe0104dc0 con 0x7fabe0105300 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.937+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fabd4004dd0 con 0x7fabe0105300 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.937+0000 7fabd27fc640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabcc0b64d0 0x7fabcc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.937+0000 7fabe5475640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabcc0b64d0 0x7fabcc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.937+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fabd40e54f0 con 0x7fabe0105300 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.938+0000 7fabe5475640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabcc0b64d0 0x7fabcc0b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006010 tx=0x7fabdc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:29.941+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fabd40ad980 con 0x7fabe0105300 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.134+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fabe019b5e0 con 0x7fabe0105300 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.140+0000 7fabd27fc640 1 -- 172.21.15.31:0/2258991875 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v28)=0 v28) ==== 119+0+0 (secure 0 0 0) 0x7fabe019b5e0 con 0x7fabe0105300 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.142+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabcc0b64d0 msgr2=0x7fabcc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.142+0000 7fabe76d9640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabcc0b64d0 0x7fabcc0b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006010 tx=0x7fabdc031400 comp rx=0 tx=0).stop 2024-09-21T06:03:30.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.142+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 msgr2=0x7fabe019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.142+0000 7fabe76d9640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 0x7fabe019aa50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fabd400d750 tx=0x7fabd400dc10 comp rx=0 tx=0).stop 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 shutdown_connections 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabe010de10 0x7fabe019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabcc0b64d0 0x7fabcc0b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabe0105300 0x7fabe019aa50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 --2- 172.21.15.31:0/2258991875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabe01049b0 0x7fabe019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 >> 172.21.15.31:0/2258991875 conn(0x7fabe0079730 msgr2=0x7fabe01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 shutdown_connections 2024-09-21T06:03:30.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.143+0000 7fabe76d9640 1 -- 172.21.15.31:0/2258991875 wait complete. 2024-09-21T06:03:30.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.861+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f560806b290 con 0x7f5608102a60 2024-09-21T06:03:30.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.868+0000 7f55fe7fc640 1 -- 172.21.15.31:0/2241825159 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f560806b290 con 0x7f5608102a60 2024-09-21T06:03:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55f40b64d0 msgr2=0x7f55f40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55f40b64d0 0x7f55f40b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5608100c50 tx=0x7f55f800b390 comp rx=0 tx=0).stop 2024-09-21T06:03:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 msgr2=0x7f560819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 0x7f560819b2b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5604005940 tx=0x7f56040042f0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 shutdown_connections 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5608108cf0 0x7f560819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55f40b64d0 0x7f55f40b89e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56081083a0 0x7f560819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 --2- 172.21.15.31:0/2241825159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5608102a60 0x7f560819b2b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.870+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 >> 172.21.15.31:0/2241825159 conn(0x7f5608079730 msgr2=0x7f5608101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.871+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 shutdown_connections 2024-09-21T06:03:30.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:30.871+0000 7f560fa0c640 1 -- 172.21.15.31:0/2241825159 wait complete. 2024-09-21T06:03:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2258991875' entity='client.admin' 2024-09-21T06:03:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:31 smithi031 ceph-mon[31237]: from='client.? ' entity='client.admin' 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2258991875' entity='client.admin' 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:31 smithi110 ceph-mon[34550]: from='client.? ' entity='client.admin' 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2258991875' entity='client.admin' 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:31 smithi107 ceph-mon[34380]: from='client.? ' entity='client.admin' 2024-09-21T06:03:31.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 -- 172.21.15.31:0/1836138754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0103bb0 msgr2=0x7effb0103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 --2- 172.21.15.31:0/1836138754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0103bb0 0x7effb0103fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7effa0009900 tx=0x7effa002a410 comp rx=0 tx=0).stop 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 -- 172.21.15.31:0/1836138754 shutdown_connections 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 --2- 172.21.15.31:0/1836138754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effb0109e40 0x7effb0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 --2- 172.21.15.31:0/1836138754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb01094f0 0x7effb0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 --2- 172.21.15.31:0/1836138754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0103bb0 0x7effb0103fc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 -- 172.21.15.31:0/1836138754 >> 172.21.15.31:0/1836138754 conn(0x7effb00ff600 msgr2=0x7effb0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 -- 172.21.15.31:0/1836138754 shutdown_connections 2024-09-21T06:03:31.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.991+0000 7effb48c6640 1 -- 172.21.15.31:0/1836138754 wait complete. 2024-09-21T06:03:31.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.992+0000 7effb48c6640 1 Processor -- start 2024-09-21T06:03:31.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 -- start start 2024-09-21T06:03:31.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb0103bb0 0x7effb019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:31.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effb01094f0 0x7effb019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 0x7effb019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7effb0113b00 con 0x7effb0109e40 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7effb0113980 con 0x7effb0103bb0 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effb48c6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7effb0113c80 con 0x7effb01094f0 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 0x7effb019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 0x7effb019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47628/0 (socket says 172.21.15.31:47628) 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 -- 172.21.15.31:0/314241732 learned_addr learned my addr 172.21.15.31:0/314241732 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effae59c640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb0103bb0 0x7effb019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effadd9b640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effb01094f0 0x7effb019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:31.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 -- 172.21.15.31:0/314241732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effb01094f0 msgr2=0x7effb019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effb01094f0 0x7effb019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 -- 172.21.15.31:0/314241732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb0103bb0 msgr2=0x7effb019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb0103bb0 0x7effb019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.993+0000 7effaed9d640 1 -- 172.21.15.31:0/314241732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb01a0440 con 0x7effb0109e40 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.994+0000 7effaed9d640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 0x7effb019fd10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7effa800cad0 tx=0x7effa800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.994+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa800ae20 con 0x7effb0109e40 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.994+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effa8005020 con 0x7effb0109e40 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.994+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb01a0730 con 0x7effb0109e40 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.994+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa800f6b0 con 0x7effb0109e40 2024-09-21T06:03:31.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.994+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb01a6e90 con 0x7effb0109e40 2024-09-21T06:03:32.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.995+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb0103fc0 con 0x7effb0109e40 2024-09-21T06:03:32.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.996+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7effa80051d0 con 0x7effb0109e40 2024-09-21T06:03:32.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.997+0000 7eff9f7fe640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7eff980b6550 0x7eff980b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.997+0000 7effae59c640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7eff980b6550 0x7eff980b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.997+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7effa80e58b0 con 0x7effb0109e40 2024-09-21T06:03:32.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:31.998+0000 7effae59c640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7eff980b6550 0x7eff980b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7effa0031440 tx=0x7effa0005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:32.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.001+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7effa8014030 con 0x7effb0109e40 2024-09-21T06:03:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.193+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7effb002e640 con 0x7effb0109e40 2024-09-21T06:03:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.199+0000 7eff9f7fe640 1 -- 172.21.15.31:0/314241732 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7effa80adca0 con 0x7effb0109e40 2024-09-21T06:03:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7eff980b6550 msgr2=0x7eff980b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7eff980b6550 0x7eff980b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7effa0031440 tx=0x7effa0005d50 comp rx=0 tx=0).stop 2024-09-21T06:03:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 msgr2=0x7effb019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 0x7effb019fd10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7effa800cad0 tx=0x7effa800cf90 comp rx=0 tx=0).stop 2024-09-21T06:03:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 shutdown_connections 2024-09-21T06:03:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7eff980b6550 0x7eff980b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb0109e40 0x7effb019fd10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 -- 172.21.15.31:0/2987846196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b81049b0 msgr2=0x7f77b8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 --2- 172.21.15.31:0/2987846196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b81049b0 0x7f77b8104dc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009900 tx=0x7f77a802a410 comp rx=0 tx=0).stop 2024-09-21T06:03:32.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 -- 172.21.15.31:0/2987846196 shutdown_connections 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 --2- 172.21.15.31:0/2987846196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 0x7f77b8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 --2- 172.21.15.31:0/2987846196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b8105300 0x7f77b810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 --2- 172.21.15.31:0/2987846196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b81049b0 0x7f77b8104dc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 -- 172.21.15.31:0/2987846196 >> 172.21.15.31:0/2987846196 conn(0x7f77b8079730 msgr2=0x7f77b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effb01094f0 0x7effb019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 --2- 172.21.15.31:0/314241732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effb0103bb0 0x7effb019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 >> 172.21.15.31:0/314241732 conn(0x7effb00ff600 msgr2=0x7effb0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 -- 172.21.15.31:0/2987846196 shutdown_connections 2024-09-21T06:03:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 shutdown_connections 2024-09-21T06:03:32.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7effb48c6640 1 -- 172.21.15.31:0/314241732 wait complete. 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.202+0000 7f77bf090640 1 -- 172.21.15.31:0/2987846196 wait complete. 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.203+0000 7f77bf090640 1 Processor -- start 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.203+0000 7f77bf090640 1 -- start start 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bf090640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b81049b0 0x7f77b819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bf090640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 0x7f77b819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bf090640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 0x7f77b819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bf090640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8113a90 con 0x7f77b8105300 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bf090640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8113910 con 0x7f77b810de10 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bf090640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8113c10 con 0x7f77b81049b0 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 0x7f77b819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bce2c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b81049b0 0x7f77b819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 0x7f77b819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47654/0 (socket says 172.21.15.31:47654) 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bd62d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 0x7f77b819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 -- 172.21.15.31:0/1425650259 learned_addr learned my addr 172.21.15.31:0/1425650259 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 -- 172.21.15.31:0/1425650259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b81049b0 msgr2=0x7f77b819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b81049b0 0x7f77b819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 -- 172.21.15.31:0/1425650259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 msgr2=0x7f77b819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 0x7f77b819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 -- 172.21.15.31:0/1425650259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b81a04d0 con 0x7f77b8105300 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bd62d640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 0x7f77b819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77bce2c640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b81049b0 0x7f77b819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.204+0000 7f77b7fff640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 0x7f77b819ab00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f77a000d750 tx=0x7f77a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.205+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0017070 con 0x7f77b8105300 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.205+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b81a07c0 con 0x7f77b8105300 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.205+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a00054d0 con 0x7f77b8105300 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.205+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0004c30 con 0x7f77b8105300 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.205+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b81a6f20 con 0x7f77b8105300 2024-09-21T06:03:32.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.206+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b8104dc0 con 0x7f77b8105300 2024-09-21T06:03:32.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.208+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f77a0004dd0 con 0x7f77b8105300 2024-09-21T06:03:32.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.209+0000 7f77b5ffb640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77a40b6550 0x7f77a40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.209+0000 7f77bce2c640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77a40b6550 0x7f77a40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.209+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f77a00e53d0 con 0x7f77b8105300 2024-09-21T06:03:32.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.209+0000 7f77bce2c640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77a40b6550 0x7f77a40b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77a8000c00 tx=0x7f77a8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:32.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.212+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77a00b1cc0 con 0x7f77b8105300 2024-09-21T06:03:32.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.403+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f77b819b690 con 0x7f77b8105300 2024-09-21T06:03:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.410+0000 7f77b5ffb640 1 -- 172.21.15.31:0/1425650259 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f77b819b690 con 0x7f77b8105300 2024-09-21T06:03:32.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77a40b6550 msgr2=0x7f77a40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77a40b6550 0x7f77a40b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77a8000c00 tx=0x7f77a8031400 comp rx=0 tx=0).stop 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 msgr2=0x7f77b819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 0x7f77b819ab00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f77a000d750 tx=0x7f77a000dc10 comp rx=0 tx=0).stop 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 shutdown_connections 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77b810de10 0x7f77b819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77a40b6550 0x7f77a40b8a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77b8105300 0x7f77b819ab00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 --2- 172.21.15.31:0/1425650259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b81049b0 0x7f77b819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 >> 172.21.15.31:0/1425650259 conn(0x7f77b8079730 msgr2=0x7f77b80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 shutdown_connections 2024-09-21T06:03:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.412+0000 7f77bf090640 1 -- 172.21.15.31:0/1425650259 wait complete. 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 -- 172.21.15.31:0/1999424256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79301083d0 msgr2=0x7f79301087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 --2- 172.21.15.31:0/1999424256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79301083d0 0x7f79301087e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7920009900 tx=0x7f792002a410 comp rx=0 tx=0).stop 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 -- 172.21.15.31:0/1999424256 shutdown_connections 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 --2- 172.21.15.31:0/1999424256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7930108d20 0x7f793010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 --2- 172.21.15.31:0/1999424256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79301083d0 0x7f79301087e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 --2- 172.21.15.31:0/1999424256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7930102ae0 0x7f7930102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 -- 172.21.15.31:0/1999424256 >> 172.21.15.31:0/1999424256 conn(0x7f7930079760 msgr2=0x7f7930079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:32.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 -- 172.21.15.31:0/1999424256 shutdown_connections 2024-09-21T06:03:32.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 -- 172.21.15.31:0/1999424256 wait complete. 2024-09-21T06:03:32.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 Processor -- start 2024-09-21T06:03:32.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.836+0000 7f7937ff3640 1 -- start start 2024-09-21T06:03:32.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7937ff3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 0x7f793019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7937ff3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79301083d0 0x7f793019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7937ff3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7930108d20 0x7f793019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7937ff3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f79301129e0 con 0x7f7930102ae0 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7937ff3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7930112860 con 0x7f7930108d20 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7937ff3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7930112b60 con 0x7f79301083d0 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 0x7f793019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 0x7f793019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47666/0 (socket says 172.21.15.31:47666) 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 -- 172.21.15.31:0/3395107418 learned_addr learned my addr 172.21.15.31:0/3395107418 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f793558e640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79301083d0 0x7f793019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7936590640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7930108d20 0x7f793019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 -- 172.21.15.31:0/3395107418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79301083d0 msgr2=0x7f793019b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79301083d0 0x7f793019b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 -- 172.21.15.31:0/3395107418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7930108d20 msgr2=0x7f793019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:32.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7930108d20 0x7f793019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.837+0000 7f7935d8f640 1 -- 172.21.15.31:0/3395107418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79301a6580 con 0x7f7930102ae0 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.838+0000 7f7935d8f640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 0x7f793019b2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f792c00c970 tx=0x7f792c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.838+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c007bb0 con 0x7f7930102ae0 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.838+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792c007d50 con 0x7f7930102ae0 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.838+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79301a6870 con 0x7f7930102ae0 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.838+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c00f450 con 0x7f7930102ae0 2024-09-21T06:03:32.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.838+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79301a6d80 con 0x7f7930102ae0 2024-09-21T06:03:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.839+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7930103d10 con 0x7f7930102ae0 2024-09-21T06:03:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.840+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f792c004f00 con 0x7f7930102ae0 2024-09-21T06:03:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.841+0000 7f7926ffd640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f791c0b6600 0x7f791c0b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.841+0000 7f793558e640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f791c0b6600 0x7f791c0b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.841+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f792c0e57e0 con 0x7f7930102ae0 2024-09-21T06:03:32.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.842+0000 7f793558e640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f791c0b6600 0x7f791c0b8b10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7920031440 tx=0x7f7920005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:32.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:32.844+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f792c014030 con 0x7f7930102ae0 2024-09-21T06:03:33.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.039+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f793006bf50 con 0x7f7930102ae0 2024-09-21T06:03:33.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.043+0000 7f7926ffd640 1 -- 172.21.15.31:0/3395107418 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7f793006bf50 con 0x7f7930102ae0 2024-09-21T06:03:33.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f791c0b6600 msgr2=0x7f791c0b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:33.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f791c0b6600 0x7f791c0b8b10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7920031440 tx=0x7f7920005a20 comp rx=0 tx=0).stop 2024-09-21T06:03:33.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 msgr2=0x7f793019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 0x7f793019b2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f792c00c970 tx=0x7f792c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 shutdown_connections 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7930108d20 0x7f793019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79301083d0 0x7f793019b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f791c0b6600 0x7f791c0b8b10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 --2- 172.21.15.31:0/3395107418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7930102ae0 0x7f793019b2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 >> 172.21.15.31:0/3395107418 conn(0x7f7930079760 msgr2=0x7f7930104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 shutdown_connections 2024-09-21T06:03:33.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:33.046+0000 7f7937ff3640 1 -- 172.21.15.31:0/3395107418 wait complete. 2024-09-21T06:03:33.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/314241732' entity='client.admin' 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1425650259' entity='client.admin' 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3395107418' entity='client.admin' 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:33.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/314241732' entity='client.admin' 2024-09-21T06:03:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1425650259' entity='client.admin' 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3395107418' entity='client.admin' 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/314241732' entity='client.admin' 2024-09-21T06:03:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1425650259' entity='client.admin' 2024-09-21T06:03:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3395107418' entity='client.admin' 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:33.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 -- 172.21.15.31:0/2473961732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c102ae0 msgr2=0x7f695c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 --2- 172.21.15.31:0/2473961732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c102ae0 0x7f695c102ef0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f695002a410 comp rx=0 tx=0).stop 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 -- 172.21.15.31:0/2473961732 shutdown_connections 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 --2- 172.21.15.31:0/2473961732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f695c108d20 0x7f695c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 --2- 172.21.15.31:0/2473961732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c1083d0 0x7f695c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 --2- 172.21.15.31:0/2473961732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c102ae0 0x7f695c102ef0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 -- 172.21.15.31:0/2473961732 >> 172.21.15.31:0/2473961732 conn(0x7f695c079760 msgr2=0x7f695c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 -- 172.21.15.31:0/2473961732 shutdown_connections 2024-09-21T06:03:34.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.093+0000 7f696311d640 1 -- 172.21.15.31:0/2473961732 wait complete. 2024-09-21T06:03:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 Processor -- start 2024-09-21T06:03:34.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 -- start start 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f695c102ae0 0x7f695c19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 0x7f695c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c108d20 0x7f695c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f695c112990 con 0x7f695c1083d0 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f695c112810 con 0x7f695c108d20 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f696311d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f695c112b10 con 0x7f695c102ae0 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f695bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 0x7f695c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f695bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 0x7f695c19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47686/0 (socket says 172.21.15.31:47686) 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f695bfff640 1 -- 172.21.15.31:0/632184615 learned_addr learned my addr 172.21.15.31:0/632184615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:34.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.094+0000 7f6960eb9640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f695c102ae0 0x7f695c19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f69616ba640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c108d20 0x7f695c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f695bfff640 1 -- 172.21.15.31:0/632184615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f695c102ae0 msgr2=0x7f695c19b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f695bfff640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f695c102ae0 0x7f695c19b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f695bfff640 1 -- 172.21.15.31:0/632184615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c108d20 msgr2=0x7f695c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f695bfff640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c108d20 0x7f695c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f695bfff640 1 -- 172.21.15.31:0/632184615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c1a6530 con 0x7f695c1083d0 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f69616ba640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c108d20 0x7f695c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f695bfff640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 0x7f695c19b760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6948005fd0 tx=0x7f694800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948017070 con 0x7f695c1083d0 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6948009c30 con 0x7f695c1083d0 2024-09-21T06:03:34.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c1a6820 con 0x7f695c1083d0 2024-09-21T06:03:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948003d30 con 0x7f695c1083d0 2024-09-21T06:03:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.095+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1a6d60 con 0x7f695c1083d0 2024-09-21T06:03:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.096+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f695c103d10 con 0x7f695c1083d0 2024-09-21T06:03:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.098+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f6948003ed0 con 0x7f695c1083d0 2024-09-21T06:03:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.099+0000 7f6959ffb640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f694c0b64d0 0x7f694c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.099+0000 7f6960eb9640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f694c0b64d0 0x7f694c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.099+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f69480e54f0 con 0x7f695c1083d0 2024-09-21T06:03:34.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.099+0000 7f6960eb9640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f694c0b64d0 0x7f694c0b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6950000c00 tx=0x7f6950031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:34.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.103+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6948020030 con 0x7f695c1083d0 2024-09-21T06:03:34.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.300+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f695c02e640 con 0x7f695c1083d0 2024-09-21T06:03:34.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.305+0000 7f6959ffb640 1 -- 172.21.15.31:0/632184615 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v33)=0 v33) ==== 120+0+0 (secure 0 0 0) 0x7f6948009de0 con 0x7f695c1083d0 2024-09-21T06:03:34.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f694c0b64d0 msgr2=0x7f694c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f694c0b64d0 0x7f694c0b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6950000c00 tx=0x7f6950031400 comp rx=0 tx=0).stop 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 msgr2=0x7f695c19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 0x7f695c19b760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6948005fd0 tx=0x7f694800dde0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 shutdown_connections 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f695c108d20 0x7f695c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f694c0b64d0 0x7f694c0b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c1083d0 0x7f695c19b760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 --2- 172.21.15.31:0/632184615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f695c102ae0 0x7f695c19b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.307+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 >> 172.21.15.31:0/632184615 conn(0x7f695c079760 msgr2=0x7f695c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.308+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 shutdown_connections 2024-09-21T06:03:34.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:34.308+0000 7f696311d640 1 -- 172.21.15.31:0/632184615 wait complete. 2024-09-21T06:03:34.964 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:34.964 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:34.965 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:35.363 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:35 smithi031 ceph-mon[31237]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:35.363 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:35 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/632184615' entity='client.admin' 2024-09-21T06:03:35.395 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-21T06:03:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:35 smithi110 ceph-mon[34550]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:35 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/632184615' entity='client.admin' 2024-09-21T06:03:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:35 smithi107 ceph-mon[34380]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:35 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/632184615' entity='client.admin' 2024-09-21T06:03:36.269 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi031') 2024-09-21T06:03:37.560 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:37 smithi031 ceph-mon[31237]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:37.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 -- 172.21.15.31:0/2654068704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17401084c0 msgr2=0x7f174010ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 --2- 172.21.15.31:0/2654068704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17401084c0 0x7f174010ecf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1738009ca0 tx=0x7f173802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 -- 172.21.15.31:0/2654068704 shutdown_connections 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 --2- 172.21.15.31:0/2654068704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17401084c0 0x7f174010ecf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 --2- 172.21.15.31:0/2654068704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1740107b70 0x7f1740107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 --2- 172.21.15.31:0/2654068704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1740102230 0x7f1740102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 -- 172.21.15.31:0/2654068704 >> 172.21.15.31:0/2654068704 conn(0x7f1740079e40 msgr2=0x7f174007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.651+0000 7f1747d2d640 1 -- 172.21.15.31:0/2654068704 shutdown_connections 2024-09-21T06:03:37.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.652+0000 7f1747d2d640 1 -- 172.21.15.31:0/2654068704 wait complete. 2024-09-21T06:03:37.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.652+0000 7f1747d2d640 1 Processor -- start 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.652+0000 7f1747d2d640 1 -- start start 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1747d2d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1740102230 0x7f174019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1747d2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 0x7f174019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1747d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17401084c0 0x7f174019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1747d2d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1740113b00 con 0x7f1740107b70 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1747d2d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1740113980 con 0x7f17401084c0 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1747d2d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1740113c80 con 0x7f1740102230 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 0x7f174019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 0x7f174019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47700/0 (socket says 172.21.15.31:47700) 2024-09-21T06:03:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 -- 172.21.15.31:0/4180846391 learned_addr learned my addr 172.21.15.31:0/4180846391 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f1745ac9640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1740102230 0x7f174019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17462ca640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17401084c0 0x7f174019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 -- 172.21.15.31:0/4180846391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1740102230 msgr2=0x7f174019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1740102230 0x7f174019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 -- 172.21.15.31:0/4180846391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17401084c0 msgr2=0x7f174019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17401084c0 0x7f174019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17452c8640 1 -- 172.21.15.31:0/4180846391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401a0440 con 0x7f1740107b70 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.653+0000 7f17462ca640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17401084c0 0x7f174019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.654+0000 7f17452c8640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 0x7f174019aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f173000d740 tx=0x7f173000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:37.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.654+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730017070 con 0x7f1740107b70 2024-09-21T06:03:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.654+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173000de50 con 0x7f1740107b70 2024-09-21T06:03:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.654+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401a0730 con 0x7f1740107b70 2024-09-21T06:03:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.654+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173000f620 con 0x7f1740107b70 2024-09-21T06:03:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.654+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401a6e90 con 0x7f1740107b70 2024-09-21T06:03:37.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.655+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1740102640 con 0x7f1740107b70 2024-09-21T06:03:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.656+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f173001a070 con 0x7f1740107b70 2024-09-21T06:03:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.657+0000 7f1736ffd640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f172c0b6550 0x7f172c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.657+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f17300e5090 con 0x7f1740107b70 2024-09-21T06:03:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.657+0000 7f1745ac9640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f172c0b6550 0x7f172c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.658+0000 7f1745ac9640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f172c0b6550 0x7f172c0b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f173c005fd0 tx=0x7f173c005e20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:37.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.660+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f17300ad5a0 con 0x7f1740107b70 2024-09-21T06:03:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:37 smithi110 ceph-mon[34550]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:37 smithi107 ceph-mon[34380]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 -- 172.21.15.31:0/1885478946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c105300 msgr2=0x7fa08c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 --2- 172.21.15.31:0/1885478946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c105300 0x7fa08c10d850 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 -- 172.21.15.31:0/1885478946 shutdown_connections 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 --2- 172.21.15.31:0/1885478946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa08c10de10 0x7fa08c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 --2- 172.21.15.31:0/1885478946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c105300 0x7fa08c10d850 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 --2- 172.21.15.31:0/1885478946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa08c1049b0 0x7fa08c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.835+0000 7fa093714640 1 -- 172.21.15.31:0/1885478946 >> 172.21.15.31:0/1885478946 conn(0x7fa08c079730 msgr2=0x7fa08c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.836+0000 7fa093714640 1 -- 172.21.15.31:0/1885478946 shutdown_connections 2024-09-21T06:03:37.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.836+0000 7fa093714640 1 -- 172.21.15.31:0/1885478946 wait complete. 2024-09-21T06:03:37.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.836+0000 7fa093714640 1 Processor -- start 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.836+0000 7fa093714640 1 -- start start 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa093714640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa08c1049b0 0x7fa08c1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa093714640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa08c105300 0x7fa08c102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa093714640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 0x7fa08c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa093714640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c113a70 con 0x7fa08c10de10 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa093714640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c1138f0 con 0x7fa08c105300 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa093714640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c113bf0 con 0x7fa08c1049b0 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 0x7fa08c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 0x7fa08c103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47726/0 (socket says 172.21.15.31:47726) 2024-09-21T06:03:37.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 -- 172.21.15.31:0/4237137746 learned_addr learned my addr 172.21.15.31:0/4237137746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa0914b0640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa08c1049b0 0x7fa08c1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa090caf640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa08c105300 0x7fa08c102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 -- 172.21.15.31:0/4237137746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa08c1049b0 msgr2=0x7fa08c1026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa08c1049b0 0x7fa08c1026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 -- 172.21.15.31:0/4237137746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa08c105300 msgr2=0x7fa08c102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa08c105300 0x7fa08c102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.837+0000 7fa091cb1640 1 -- 172.21.15.31:0/4237137746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c1a87e0 con 0x7fa08c10de10 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.838+0000 7fa091cb1640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 0x7fa08c103170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa084009fd0 tx=0x7fa08400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.838+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084017070 con 0x7fa08c10de10 2024-09-21T06:03:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.838+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c1a8ad0 con 0x7fa08c10de10 2024-09-21T06:03:37.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.838+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa084003e20 con 0x7fa08c10de10 2024-09-21T06:03:37.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.838+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084010460 con 0x7fa08c10de10 2024-09-21T06:03:37.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.838+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c1a8f90 con 0x7fa08c10de10 2024-09-21T06:03:37.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.839+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c104dc0 con 0x7fa08c10de10 2024-09-21T06:03:37.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.841+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fa084003970 con 0x7fa08c10de10 2024-09-21T06:03:37.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.842+0000 7fa0827fc640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0780b64d0 0x7fa0780b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:37.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.842+0000 7fa0914b0640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0780b64d0 0x7fa0780b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:37.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.842+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa0840e5410 con 0x7fa08c10de10 2024-09-21T06:03:37.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.842+0000 7fa0914b0640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0780b64d0 0x7fa0780b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa08c104140 tx=0x7fa088005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:37.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.845+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa0840ad8a0 con 0x7fa08c10de10 2024-09-21T06:03:37.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.854+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f174010a310 con 0x7f1740107b70 2024-09-21T06:03:37.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.858+0000 7f1736ffd640 1 -- 172.21.15.31:0/4180846391 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f17300b1980 con 0x7f1740107b70 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f172c0b6550 msgr2=0x7f172c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f172c0b6550 0x7f172c0b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f173c005fd0 tx=0x7f173c005e20 comp rx=0 tx=0).stop 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 msgr2=0x7f174019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 0x7f174019aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f173000d740 tx=0x7f173000dc00 comp rx=0 tx=0).stop 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 shutdown_connections 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17401084c0 0x7f174019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f172c0b6550 0x7f172c0b8a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1740107b70 0x7f174019aa70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 --2- 172.21.15.31:0/4180846391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1740102230 0x7f174019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 >> 172.21.15.31:0/4180846391 conn(0x7f1740079e40 msgr2=0x7f1740103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 shutdown_connections 2024-09-21T06:03:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:37.861+0000 7f1747d2d640 1 -- 172.21.15.31:0/4180846391 wait complete. 2024-09-21T06:03:38.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.046+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa08c02e640 con 0x7fa08c10de10 2024-09-21T06:03:38.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.050+0000 7fa0827fc640 1 -- 172.21.15.31:0/4237137746 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v35)=0 v35) ==== 133+0+0 (secure 0 0 0) 0x7fa0840d00c0 con 0x7fa08c10de10 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0780b64d0 msgr2=0x7fa0780b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0780b64d0 0x7fa0780b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa08c104140 tx=0x7fa088005f90 comp rx=0 tx=0).stop 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 msgr2=0x7fa08c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 0x7fa08c103170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa084009fd0 tx=0x7fa08400b260 comp rx=0 tx=0).stop 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 shutdown_connections 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0780b64d0 0x7fa0780b89e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa08c10de10 0x7fa08c103170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa08c105300 0x7fa08c102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 --2- 172.21.15.31:0/4237137746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa08c1049b0 0x7fa08c1026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 >> 172.21.15.31:0/4237137746 conn(0x7fa08c079730 msgr2=0x7fa08c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 shutdown_connections 2024-09-21T06:03:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:38.053+0000 7fa093714640 1 -- 172.21.15.31:0/4237137746 wait complete. 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4180846391' entity='client.admin' 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4237137746' entity='client.admin' 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:38.912 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 -- 172.21.15.31:0/2426788108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 msgr2=0x7f39681098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 --2- 172.21.15.31:0/2426788108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f39681098e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3950009900 tx=0x7f395002a410 comp rx=0 tx=0).stop 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 -- 172.21.15.31:0/2426788108 shutdown_connections 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 --2- 172.21.15.31:0/2426788108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3968109e20 0x7f3968110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 --2- 172.21.15.31:0/2426788108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f39681098e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 --2- 172.21.15.31:0/2426788108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968103b90 0x7f3968103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 -- 172.21.15.31:0/2426788108 >> 172.21.15.31:0/2426788108 conn(0x7f39680ff600 msgr2=0x7f3968101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:39.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 -- 172.21.15.31:0/2426788108 shutdown_connections 2024-09-21T06:03:39.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 -- 172.21.15.31:0/2426788108 wait complete. 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 Processor -- start 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.242+0000 7f396e380640 1 -- start start 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396e380640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3968103b90 0x7f396819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396e380640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f396819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396e380640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968109e20 0x7f396819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396e380640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3968113b30 con 0x7f39681094d0 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396e380640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39681139b0 con 0x7f3968109e20 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396e380640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3968113cb0 con 0x7f3968103b90 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f39677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f396819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:39.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f39677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f396819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56308/0 (socket says 172.21.15.31:56308) 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f39677fe640 1 -- 172.21.15.31:0/1574544565 learned_addr learned my addr 172.21.15.31:0/1574544565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396c91d640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968109e20 0x7f396819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f3967fff640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3968103b90 0x7f396819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.243+0000 7f396c91d640 1 -- 172.21.15.31:0/1574544565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3968103b90 msgr2=0x7f396819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396c91d640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3968103b90 0x7f396819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396c91d640 1 -- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 msgr2=0x7f396819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396c91d640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f396819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396c91d640 1 -- 172.21.15.31:0/1574544565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39681a0410 con 0x7f3968109e20 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f39677fe640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f396819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396c91d640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968109e20 0x7f396819fce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f396000e940 tx=0x7f396000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:39.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960004d20 con 0x7f3968109e20 2024-09-21T06:03:39.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3960005020 con 0x7f3968109e20 2024-09-21T06:03:39.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39681a0700 con 0x7f3968109e20 2024-09-21T06:03:39.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39600116d0 con 0x7f3968109e20 2024-09-21T06:03:39.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.244+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39681a6e60 con 0x7f3968109e20 2024-09-21T06:03:39.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.245+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3968104e10 con 0x7f3968109e20 2024-09-21T06:03:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.248+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f3960005460 con 0x7f3968109e20 2024-09-21T06:03:39.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.249+0000 7f39657fa640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39540b6550 0x7f39540b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.249+0000 7f3967fff640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39540b6550 0x7f39540b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.249+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f39600e5710 con 0x7f3968109e20 2024-09-21T06:03:39.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.249+0000 7f3967fff640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39540b6550 0x7f39540b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f395800bbe0 tx=0x7f395800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:39.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.254+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3960016030 con 0x7f3968109e20 2024-09-21T06:03:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4180846391' entity='client.admin' 2024-09-21T06:03:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4237137746' entity='client.admin' 2024-09-21T06:03:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:39.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4180846391' entity='client.admin' 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4237137746' entity='client.admin' 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:39.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.469+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f396802e640 con 0x7f3968109e20 2024-09-21T06:03:39.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.477+0000 7f39657fa640 1 -- 172.21.15.31:0/1574544565 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v37)=0 v37) ==== 125+0+0 (secure 0 0 0) 0x7f39600b1f80 con 0x7f3968109e20 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39540b6550 msgr2=0x7f39540b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39540b6550 0x7f39540b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f395800bbe0 tx=0x7f395800b390 comp rx=0 tx=0).stop 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968109e20 msgr2=0x7f396819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968109e20 0x7f396819fce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f396000e940 tx=0x7f396000ee00 comp rx=0 tx=0).stop 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 shutdown_connections 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3968109e20 0x7f396819fce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39540b6550 0x7f39540b8a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39681094d0 0x7f396819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 --2- 172.21.15.31:0/1574544565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3968103b90 0x7f396819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:39.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 >> 172.21.15.31:0/1574544565 conn(0x7f39680ff600 msgr2=0x7f3968101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:39.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 shutdown_connections 2024-09-21T06:03:39.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.481+0000 7f396e380640 1 -- 172.21.15.31:0/1574544565 wait complete. 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 -- 172.21.15.31:0/2572633119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c109e10 msgr2=0x7f4c8c110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 --2- 172.21.15.31:0/2572633119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c109e10 0x7f4c8c110640 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009ca0 tx=0x7f4c8402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 -- 172.21.15.31:0/2572633119 shutdown_connections 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 --2- 172.21.15.31:0/2572633119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c109e10 0x7f4c8c110640 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 --2- 172.21.15.31:0/2572633119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 --2- 172.21.15.31:0/2572633119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c8c103b80 0x7f4c8c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 -- 172.21.15.31:0/2572633119 >> 172.21.15.31:0/2572633119 conn(0x7f4c8c0ff610 msgr2=0x7f4c8c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 -- 172.21.15.31:0/2572633119 shutdown_connections 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.137+0000 7f4c936de640 1 -- 172.21.15.31:0/2572633119 wait complete. 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 Processor -- start 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 -- start start 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c103b80 0x7f4c8c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c8c109e10 0x7f4c8c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113ad0 con 0x7f4c8c1094c0 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113950 con 0x7f4c8c103b80 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.138+0000 7f4c936de640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113c50 con 0x7f4c8c109e10 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56298/0 (socket says 172.21.15.31:56298) 2024-09-21T06:03:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 -- 172.21.15.31:0/3135218149 learned_addr learned my addr 172.21.15.31:0/3135218149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c91c7b640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c8c109e10 0x7f4c8c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c9147a640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c103b80 0x7f4c8c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 -- 172.21.15.31:0/3135218149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c8c109e10 msgr2=0x7f4c8c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c8c109e10 0x7f4c8c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 -- 172.21.15.31:0/3135218149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c103b80 msgr2=0x7f4c8c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c103b80 0x7f4c8c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 -- 172.21.15.31:0/3135218149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1a03f0 con 0x7f4c8c1094c0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c90c79640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c19aa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d740 tx=0x7f4c8800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88017070 con 0x7f4c8c1094c0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c8800de50 con 0x7f4c8c1094c0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800f620 con 0x7f4c8c1094c0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.139+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1a06e0 con 0x7f4c8c1094c0 2024-09-21T06:03:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.140+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a6e40 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.141+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c104e00 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.142+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f4c8801a070 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.143+0000 7f4c827fc640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b6550 0x7f4c780b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.143+0000 7f4c9147a640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b6550 0x7f4c780b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.143+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4c880e4ff0 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.144+0000 7f4c9147a640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b6550 0x7f4c780b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005fd0 tx=0x7f4c7c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.147+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c880b1860 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.393+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f4c8c10bc00 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.399+0000 7f4c827fc640 1 -- 172.21.15.31:0/3135218149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v36)=0 v36) ==== 119+0+0 (secure 0 0 0) 0x7f4c880b67a0 con 0x7f4c8c1094c0 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b6550 msgr2=0x7f4c780b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b6550 0x7f4c780b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005fd0 tx=0x7f4c7c00b390 comp rx=0 tx=0).stop 2024-09-21T06:03:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 msgr2=0x7f4c8c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c19aa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d740 tx=0x7f4c8800dc00 comp rx=0 tx=0).stop 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 shutdown_connections 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c8c109e10 0x7f4c8c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b6550 0x7f4c780b8a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8c1094c0 0x7f4c8c19aa20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 --2- 172.21.15.31:0/3135218149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c8c103b80 0x7f4c8c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 >> 172.21.15.31:0/3135218149 conn(0x7f4c8c0ff610 msgr2=0x7f4c8c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 shutdown_connections 2024-09-21T06:03:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:39.402+0000 7f4c936de640 1 -- 172.21.15.31:0/3135218149 wait complete. 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3135218149' entity='client.admin' 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: from='client.? ' entity='client.admin' 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:40.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:40 smithi031 ceph-mon[31237]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:40.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3135218149' entity='client.admin' 2024-09-21T06:03:40.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: from='client.? ' entity='client.admin' 2024-09-21T06:03:40.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:40.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:40.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:40.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:40.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:40 smithi110 ceph-mon[34550]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3135218149' entity='client.admin' 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: from='client.? ' entity='client.admin' 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:03:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:40 smithi107 ceph-mon[34380]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:41.022 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.022 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.022 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.022 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.022 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.022 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.023 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c04f7042-77de-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-21T06:03:41.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T06:03:41.283 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/51131642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 msgr2=0x7ff8cc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/51131642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc1098b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009900 tx=0x7ff8c002a410 comp rx=0 tx=0).stop 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/51131642 shutdown_connections 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/51131642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc109df0 0x7ff8cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/51131642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc1098b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/51131642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8cc103b60 0x7ff8cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/51131642 >> 172.21.15.31:0/51131642 conn(0x7ff8cc07b850 msgr2=0x7ff8cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:41.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/51131642 shutdown_connections 2024-09-21T06:03:41.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.704+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/51131642 wait complete. 2024-09-21T06:03:41.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 Processor -- start 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 -- start start 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc103b60 0x7ff8cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8cc109df0 0x7ff8cc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113ab0 con 0x7ff8cc1094a0 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113930 con 0x7ff8cc109df0 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8d0d4b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113c30 con 0x7ff8cc103b60 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8ca59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc103b60 0x7ff8cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8c9d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56318/0 (socket says 172.21.15.31:56318) 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.705+0000 7ff8c9d9b640 1 -- 172.21.15.31:0/10526769 learned_addr learned my addr 172.21.15.31:0/10526769 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8cad9d640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8cc109df0 0x7ff8cc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8c9d9b640 1 -- 172.21.15.31:0/10526769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc103b60 msgr2=0x7ff8cc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8c9d9b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc103b60 0x7ff8cc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8c9d9b640 1 -- 172.21.15.31:0/10526769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8cc109df0 msgr2=0x7ff8cc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8c9d9b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8cc109df0 0x7ff8cc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8c9d9b640 1 -- 172.21.15.31:0/10526769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1a8740 con 0x7ff8cc1094a0 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8ca59c640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc103b60 0x7ff8cc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8c9d9b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc075c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8c002a930 tx=0x7ff8c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c003f030 con 0x7ff8cc1094a0 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc1a89d0 con 0x7ff8cc1094a0 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0003aa0 con 0x7ff8cc1094a0 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0035070 con 0x7ff8cc1094a0 2024-09-21T06:03:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.706+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a8f00 con 0x7ff8cc1094a0 2024-09-21T06:03:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.708+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc103f70 con 0x7ff8cc1094a0 2024-09-21T06:03:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.709+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7ff8c0042030 con 0x7ff8cc1094a0 2024-09-21T06:03:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.710+0000 7ff8bb7fe640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8b40b64d0 0x7ff8b40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.710+0000 7ff8ca59c640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8b40b64d0 0x7ff8b40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.710+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff8c0106190 con 0x7ff8cc1094a0 2024-09-21T06:03:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.711+0000 7ff8ca59c640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8b40b64d0 0x7ff8b40b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc077160 tx=0x7ff8bc009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.714+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8c00d2a00 con 0x7ff8cc1094a0 2024-09-21T06:03:41.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.901+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff8cc10bc40 con 0x7ff8cc1094a0 2024-09-21T06:03:41.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.902+0000 7ff8bb7fe640 1 -- 172.21.15.31:0/10526769 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7ff8c00d7940 con 0x7ff8cc1094a0 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global dev auth_debug true 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global basic log_to_file true 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global basic log_to_journald false 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global basic log_to_stderr false 2024-09-21T06:03:41.904 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_allow_pool_delete true 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global dev ms_die_on_bug true 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global dev ms_die_on_old_message true 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global dev ms_die_on_skipped_message false 2024-09-21T06:03:41.905 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced debug_mon 20/20 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced debug_ms 1/1 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced debug_paxos 20/20 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T06:03:41.906 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_client 20/20 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:03:41.907 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev bdev_debug_aio true 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced bdev_enable_discard true 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced bluestore_allocator bitmap * 2024-09-21T06:03:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_bluefs 1/20 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_bluestore 1/20 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_ms 1/1 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_osd 20/20 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced debug_rocksdb 4/10 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_op_order true 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_shutdown true 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T06:03:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T06:03:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced debug_mds 20/20 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced debug_ms 2/2 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_bal_split_size 100 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds dev mds_debug_frag true 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:03:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:mds dev mds_verify_scatter true 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced client_mount_timeout 600 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced debug_client 20/20 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced debug_ms 1/1 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T06:03:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8b40b64d0 msgr2=0x7ff8b40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8b40b64d0 0x7ff8b40b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc077160 tx=0x7ff8bc009760 comp rx=0 tx=0).stop 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 msgr2=0x7ff8cc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc075c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8c002a930 tx=0x7ff8c0004270 comp rx=0 tx=0).stop 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 shutdown_connections 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8cc109df0 0x7ff8cc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8b40b64d0 0x7ff8b40b89e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc1094a0 0x7ff8cc075c50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 --2- 172.21.15.31:0/10526769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc103b60 0x7ff8cc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 >> 172.21.15.31:0/10526769 conn(0x7ff8cc07b850 msgr2=0x7ff8cc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 shutdown_connections 2024-09-21T06:03:41.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:41.904+0000 7ff8d0d4b640 1 -- 172.21.15.31:0/10526769 wait complete. 2024-09-21T06:03:42.541 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T06:03:42.541 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:03:42.541 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:03:42.559 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:03:42.559 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-21T06:03:42.618 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 06:02:15.827335693 +0000 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 05:57:46.966464195 +0000 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 05:57:46.966464195 +0000 2024-09-21T06:03:42.619 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 05:57:46.966464195 +0000 2024-09-21T06:03:42.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:03:42.682 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:42 smithi031 ceph-mon[31237]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:42.683 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:42 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/10526769' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:03:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T06:03:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T06:03:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000297038 s, 1.7 MB/s 2024-09-21T06:03:42.687 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:03:42.746 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-21T06:03:42.802 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:03:42.802 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T06:03:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 06:02:15.867335014 +0000 2024-09-21T06:03:42.803 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 05:57:47.302458443 +0000 2024-09-21T06:03:42.803 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 05:57:47.302458443 +0000 2024-09-21T06:03:42.803 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 05:57:47.302458443 +0000 2024-09-21T06:03:42.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:03:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T06:03:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T06:03:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000204491 s, 2.5 MB/s 2024-09-21T06:03:42.869 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:03:42.927 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 06:02:15.905334370 +0000 2024-09-21T06:03:42.983 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 05:57:47.641452640 +0000 2024-09-21T06:03:42.984 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 05:57:47.641452640 +0000 2024-09-21T06:03:42.984 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 05:57:47.641452640 +0000 2024-09-21T06:03:42.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:03:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:42 smithi110 ceph-mon[34550]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:42 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/10526769' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:03:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:42 smithi107 ceph-mon[34380]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:42 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/10526769' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:03:43.047 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T06:03:43.047 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T06:03:43.047 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00019626 s, 2.6 MB/s 2024-09-21T06:03:43.048 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:03:43.105 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-21T06:03:43.161 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-21 06:02:15.940333776 +0000 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-21 05:57:47.994446598 +0000 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-21 05:57:47.994446598 +0000 2024-09-21T06:03:43.162 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-21 05:57:47.994446598 +0000 2024-09-21T06:03:43.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:03:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-21T06:03:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-21T06:03:43.227 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000208056 s, 2.5 MB/s 2024-09-21T06:03:43.227 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:03:43.285 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-21T06:03:43.285 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:03:43.304 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:03:43.305 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-21T06:03:43.363 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:03:43.363 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 06:03:15.017404729 +0000 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 05:57:51.834501226 +0000 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 05:57:51.834501226 +0000 2024-09-21T06:03:43.364 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 05:57:51.834501226 +0000 2024-09-21T06:03:43.364 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:03:43.433 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T06:03:43.434 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T06:03:43.434 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000308276 s, 1.7 MB/s 2024-09-21T06:03:43.435 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:03:43.494 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-21T06:03:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:03:43.551 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 06:03:15.018404713 +0000 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 05:57:52.157495909 +0000 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 05:57:52.157495909 +0000 2024-09-21T06:03:43.552 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 05:57:52.157495909 +0000 2024-09-21T06:03:43.552 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:03:43.616 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T06:03:43.617 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T06:03:43.617 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000281871 s, 1.8 MB/s 2024-09-21T06:03:43.618 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:03:43.675 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-21T06:03:43.732 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:03:43.732 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:43.732 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T06:03:43.732 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:43.732 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:43.732 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 06:03:15.018404713 +0000 2024-09-21T06:03:43.733 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 05:57:52.466490822 +0000 2024-09-21T06:03:43.733 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 05:57:52.466490822 +0000 2024-09-21T06:03:43.733 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 05:57:52.466490822 +0000 2024-09-21T06:03:43.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:03:43.797 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T06:03:43.797 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T06:03:43.797 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000181357 s, 2.8 MB/s 2024-09-21T06:03:43.798 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:03:43.855 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-21 06:03:15.018404713 +0000 2024-09-21T06:03:43.911 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-21 05:57:52.784485588 +0000 2024-09-21T06:03:43.912 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-21 05:57:52.784485588 +0000 2024-09-21T06:03:43.912 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-21 05:57:52.784485588 +0000 2024-09-21T06:03:43.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:03:43.975 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-21T06:03:43.975 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-21T06:03:43.975 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000200085 s, 2.6 MB/s 2024-09-21T06:03:43.976 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:03:44.033 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-21T06:03:44.033 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:03:44.053 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:03:44.053 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-21T06:03:44.111 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:03:44.111 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:44.111 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-21T06:03:44.111 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:44.112 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:44.112 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 06:03:12.798002761 +0000 2024-09-21T06:03:44.112 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 05:58:01.522235097 +0000 2024-09-21T06:03:44.112 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 05:58:01.522235097 +0000 2024-09-21T06:03:44.112 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 05:58:01.522235097 +0000 2024-09-21T06:03:44.112 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:03:44.181 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T06:03:44.182 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T06:03:44.182 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.00014226 s, 3.6 MB/s 2024-09-21T06:03:44.183 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:03:44.242 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-21T06:03:44.299 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 06:03:12.798002761 +0000 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 05:58:01.832229746 +0000 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 05:58:01.832229746 +0000 2024-09-21T06:03:44.300 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 05:58:01.832229746 +0000 2024-09-21T06:03:44.300 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:03:44.365 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T06:03:44.365 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T06:03:44.365 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000112887 s, 4.5 MB/s 2024-09-21T06:03:44.366 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:03:44.424 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 06:03:12.799002744 +0000 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 05:58:02.147224309 +0000 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 05:58:02.147224309 +0000 2024-09-21T06:03:44.480 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 05:58:02.147224309 +0000 2024-09-21T06:03:44.481 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:03:44.544 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T06:03:44.544 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T06:03:44.544 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000117196 s, 4.4 MB/s 2024-09-21T06:03:44.545 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:03:44.601 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-21 06:03:12.799002744 +0000 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-21 05:58:02.466218802 +0000 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-21 05:58:02.466218802 +0000 2024-09-21T06:03:44.657 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-21 05:58:02.466218802 +0000 2024-09-21T06:03:44.658 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:03:44.719 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-21T06:03:44.720 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-21T06:03:44.720 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000111064 s, 4.6 MB/s 2024-09-21T06:03:44.721 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:03:44.777 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-09-21T06:03:44.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:03:44.795 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:44 smithi110 ceph-mon[34550]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:44.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:44 smithi031 ceph-mon[31237]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:45.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:44 smithi107 ceph-mon[34380]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:45.042 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:46.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:46 smithi031 ceph-mon[31237]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:47.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:46 smithi110 ceph-mon[34550]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:46 smithi107 ceph-mon[34380]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:47.401 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:03:47.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-09-21T06:03:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.113+0000 7ff85ac86640 1 -- 172.21.15.31:0/2191363735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854102a60 msgr2=0x7ff854102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.113+0000 7ff85ac86640 1 --2- 172.21.15.31:0/2191363735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854102a60 0x7ff854102e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009900 tx=0x7ff83c02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 -- 172.21.15.31:0/2191363735 shutdown_connections 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 --2- 172.21.15.31:0/2191363735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff854108cf0 0x7ff85410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 --2- 172.21.15.31:0/2191363735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 0x7ff8541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 --2- 172.21.15.31:0/2191363735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854102a60 0x7ff854102e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 -- 172.21.15.31:0/2191363735 >> 172.21.15.31:0/2191363735 conn(0x7ff854079730 msgr2=0x7ff854079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 -- 172.21.15.31:0/2191363735 shutdown_connections 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 -- 172.21.15.31:0/2191363735 wait complete. 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 Processor -- start 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.114+0000 7ff85ac86640 1 -- start start 2024-09-21T06:03:48.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff85ac86640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff854102a60 0x7ff85419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff85ac86640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 0x7ff85419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff85ac86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 0x7ff85419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff85ac86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff854112960 con 0x7ff854108cf0 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff85ac86640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8541127e0 con 0x7ff8541083a0 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff85ac86640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff854112ae0 con 0x7ff854102a60 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 0x7ff85419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 0x7ff85419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56334/0 (socket says 172.21.15.31:56334) 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 -- 172.21.15.31:0/1593323685 learned_addr learned my addr 172.21.15.31:0/1593323685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff858a22640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff854102a60 0x7ff85419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff853fff640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 0x7ff85419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 -- 172.21.15.31:0/1593323685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff854102a60 msgr2=0x7ff85419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff854102a60 0x7ff85419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:48.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 -- 172.21.15.31:0/1593323685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 msgr2=0x7ff85419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 0x7ff85419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 -- 172.21.15.31:0/1593323685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8541a6520 con 0x7ff854108cf0 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff853fff640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 0x7ff85419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff859223640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 0x7ff85419bc90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00d950 tx=0x7ff84c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.115+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c017030 con 0x7ff854108cf0 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.116+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8541a6810 con 0x7ff854108cf0 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.116+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff84c004020 con 0x7ff854108cf0 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.116+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c003b50 con 0x7ff854108cf0 2024-09-21T06:03:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.116+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8541a6d50 con 0x7ff854108cf0 2024-09-21T06:03:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.117+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff854102e70 con 0x7ff854108cf0 2024-09-21T06:03:48.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.119+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7ff84c01a070 con 0x7ff854108cf0 2024-09-21T06:03:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.120+0000 7ff851ffb640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8400b64d0 0x7ff8400b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.120+0000 7ff858a22640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8400b64d0 0x7ff8400b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.120+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff84c0e51e0 con 0x7ff854108cf0 2024-09-21T06:03:48.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.120+0000 7ff858a22640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8400b64d0 0x7ff8400b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff83c000c00 tx=0x7ff83c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:48.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.123+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff84c021070 con 0x7ff854108cf0 2024-09-21T06:03:48.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:03:48.322+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff854063800 con 0x7ff8400b64d0 2024-09-21T06:03:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:48 smithi031 ceph-mon[31237]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:48 smithi031 ceph-mon[31237]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:03:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:03:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:03:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:48 smithi110 ceph-mon[34550]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:48 smithi110 ceph-mon[34550]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:03:49.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:03:49.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:03:49.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:48 smithi107 ceph-mon[34380]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:48 smithi107 ceph-mon[34380]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:03:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:50.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:50 smithi031 ceph-mon[31237]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:50.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:50 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/568824743' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "80675501-db5a-4326-bf65-f570e2ef7dcf"} : dispatch 2024-09-21T06:03:50.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:50 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/568824743' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80675501-db5a-4326-bf65-f570e2ef7dcf"}]': finished 2024-09-21T06:03:50.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:50 smithi031 ceph-mon[31237]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:03:50.844 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:03:51.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:50 smithi110 ceph-mon[34550]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:51.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:50 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/568824743' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "80675501-db5a-4326-bf65-f570e2ef7dcf"} : dispatch 2024-09-21T06:03:51.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:50 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/568824743' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80675501-db5a-4326-bf65-f570e2ef7dcf"}]': finished 2024-09-21T06:03:51.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:50 smithi110 ceph-mon[34550]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:03:51.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:03:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:50 smithi107 ceph-mon[34380]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:50 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/568824743' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "80675501-db5a-4326-bf65-f570e2ef7dcf"} : dispatch 2024-09-21T06:03:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:50 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/568824743' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80675501-db5a-4326-bf65-f570e2ef7dcf"}]': finished 2024-09-21T06:03:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:50 smithi107 ceph-mon[34380]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:03:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:03:52.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:51 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3549460386' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:03:52.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:51 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3549460386' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:03:52.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:51 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3549460386' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:03:53.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:52 smithi110 ceph-mon[34550]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:53.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:52 smithi107 ceph-mon[34380]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:53.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:52 smithi031 ceph-mon[31237]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:55.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:54 smithi110 ceph-mon[34550]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:55.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:54 smithi107 ceph-mon[34380]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:55.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:54 smithi031 ceph-mon[31237]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:56.864 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:56 smithi031 ceph-mon[31237]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:57.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:56 smithi110 ceph-mon[34550]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:57.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:56 smithi107 ceph-mon[34380]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:58.776 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:58 smithi031 ceph-mon[31237]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:59.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:58 smithi110 ceph-mon[34550]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:03:59.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:58 smithi107 ceph-mon[34380]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:00.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:04:00.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:00.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:03:59 smithi110 ceph-mon[34550]: Deploying daemon osd.0 on smithi031 2024-09-21T06:04:00.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:04:00.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:00.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:03:59 smithi107 ceph-mon[34380]: Deploying daemon osd.0 on smithi031 2024-09-21T06:04:00.132 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:04:00.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:00.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:03:59 smithi031 ceph-mon[31237]: Deploying daemon osd.0 on smithi031 2024-09-21T06:04:00.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:00 smithi031 ceph-mon[31237]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:01.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:00 smithi110 ceph-mon[34550]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:00 smithi107 ceph-mon[34380]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:02.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:02 smithi031 ceph-mon[31237]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:03.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:02 smithi110 ceph-mon[34550]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:03.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:02 smithi107 ceph-mon[34380]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:05.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:04 smithi031 ceph-mon[31237]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:05.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:04 smithi110 ceph-mon[34550]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:05.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:04 smithi107 ceph-mon[34380]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:05.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:05.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:05.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:06.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:06.957 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:06 smithi031 ceph-mon[31237]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:07.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:06 smithi110 ceph-mon[34550]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:07.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:06 smithi107 ceph-mon[34380]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:08 smithi110 ceph-mon[34550]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:09.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:09.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:08 smithi107 ceph-mon[34380]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:08 smithi031 ceph-mon[31237]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:09.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:09.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:09.085 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:09.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.741+0000 7ff851ffb640 1 -- 172.21.15.31:0/1593323685 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff854063800 con 0x7ff8400b64d0 2024-09-21T06:04:09.743 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8400b64d0 msgr2=0x7ff8400b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8400b64d0 0x7ff8400b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff83c000c00 tx=0x7ff83c031400 comp rx=0 tx=0).stop 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 msgr2=0x7ff85419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 0x7ff85419bc90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00d950 tx=0x7ff84c00de10 comp rx=0 tx=0).stop 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 shutdown_connections 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8400b64d0 0x7ff8400b89e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff854108cf0 0x7ff85419bc90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8541083a0 0x7ff85419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 --2- 172.21.15.31:0/1593323685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff854102a60 0x7ff85419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 >> 172.21.15.31:0/1593323685 conn(0x7ff854079730 msgr2=0x7ff8541013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:09.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 shutdown_connections 2024-09-21T06:04:09.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:04:09.744+0000 7ff85ac86640 1 -- 172.21.15.31:0/1593323685 wait complete. 2024-09-21T06:04:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:10.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:10.119 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:10.119 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:10.448 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T06:04:10.450 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_4... 2024-09-21T06:04:10.450 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:04:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:04:11.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:10 smithi107 ceph-mon[34380]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:11.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:10 smithi031 ceph-mon[31237]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:11.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:10 smithi110 ceph-mon[34550]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:12.135 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:04:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0[48759]: 2024-09-21T06:04:11.903+0000 7f9a8517c640 -1 osd.0 0 log_to_monitors true 2024-09-21T06:04:12.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:12 smithi031 ceph-mon[31237]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:12.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:12 smithi031 ceph-mon[31237]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:04:12.887 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:12 smithi107 ceph-mon[34380]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:12.887 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:12 smithi107 ceph-mon[34380]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:04:12.890 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T06:04:12.925 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-09-21T06:04:13.172 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:04:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:12 smithi110 ceph-mon[34550]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:12 smithi110 ceph-mon[34550]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:04:13.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 -- 172.21.15.107:0/1835161275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd950104880 msgr2=0x7fd950104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 --2- 172.21.15.107:0/1835161275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd950104880 0x7fd950104c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd940009900 tx=0x7fd94002a410 comp rx=0 tx=0).stop 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 -- 172.21.15.107:0/1835161275 shutdown_connections 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 --2- 172.21.15.107:0/1835161275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd95010de30 0x7fd950110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 --2- 172.21.15.107:0/1835161275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9501052b0 0x7fd95010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 --2- 172.21.15.107:0/1835161275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd950104880 0x7fd950104c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 -- 172.21.15.107:0/1835161275 >> 172.21.15.107:0/1835161275 conn(0x7fd950079730 msgr2=0x7fd950079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 -- 172.21.15.107:0/1835161275 shutdown_connections 2024-09-21T06:04:13.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.624+0000 7fd955ff0640 1 -- 172.21.15.107:0/1835161275 wait complete. 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 Processor -- start 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 -- start start 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd950104880 0x7fd950102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9501052b0 0x7fd950102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 0x7fd950103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd950113a70 con 0x7fd9501052b0 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501138f0 con 0x7fd95010de30 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd955ff0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd950113bf0 con 0x7fd950104880 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd94ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 0x7fd950103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd94ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 0x7fd950103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46810/0 (socket says 172.21.15.107:46810) 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd94ffff640 1 -- 172.21.15.107:0/2072348204 learned_addr learned my addr 172.21.15.107:0/2072348204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T06:04:13.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.625+0000 7fd94f7fe640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd950104880 0x7fd950102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd947fff640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9501052b0 0x7fd950102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94ffff640 1 -- 172.21.15.107:0/2072348204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd950104880 msgr2=0x7fd950102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94ffff640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd950104880 0x7fd950102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94ffff640 1 -- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9501052b0 msgr2=0x7fd950102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94ffff640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9501052b0 0x7fd950102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94ffff640 1 -- 172.21.15.107:0/2072348204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501a8710 con 0x7fd95010de30 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94ffff640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 0x7fd950103210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd94800cce0 tx=0x7fd94800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94800f450 con 0x7fd95010de30 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94800faa0 con 0x7fd95010de30 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948018b70 con 0x7fd95010de30 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501a8a00 con 0x7fd95010de30 2024-09-21T06:04:13.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.626+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a8ef0 con 0x7fd95010de30 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.627+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950104c90 con 0x7fd95010de30 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.629+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fd948018d10 con 0x7fd95010de30 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.630+0000 7fd94d7fa640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd93c0b64d0 0x7fd93c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.630+0000 7fd94f7fe640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd93c0b64d0 0x7fd93c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.630+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1576+0+0 (secure 0 0 0) 0x7fd9480e5db0 con 0x7fd95010de30 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.630+0000 7fd94f7fe640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd93c0b64d0 0x7fd93c0b89e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd940009230 tx=0x7fd940031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:13.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.633+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd948014030 con 0x7fd95010de30 2024-09-21T06:04:13.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:13.819+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd950063800 con 0x7fd93c0b64d0 2024-09-21T06:04:13.821 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi031 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:13.822 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:13 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.136 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0[48759]: 2024-09-21T06:04:13.802+0000 7f9a810fe640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T06:04:14.136 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0[48759]: 2024-09-21T06:04:13.806+0000 7f9a7cefc640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:04:14.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:04:14.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:04:14.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:04:14.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:14.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi031 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:14.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:13 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi031 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi031 to 17092M 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:14.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:13 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:15.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:15.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:04:15.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:04:15.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:15.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:15.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:04:15.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:04:15.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:04:15.046 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:15.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:04:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:15.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:04:15.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:16.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:15 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:04:16.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:15 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:04:16.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:15 smithi107 ceph-mon[34380]: osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] boot 2024-09-21T06:04:16.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:15 smithi107 ceph-mon[34380]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:04:16.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:16.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:15 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:04:16.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:15 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:04:16.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:15 smithi031 ceph-mon[31237]: osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] boot 2024-09-21T06:04:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:15 smithi031 ceph-mon[31237]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:04:16.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:16.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:15 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:04:16.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:15 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:04:16.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:15 smithi110 ceph-mon[34550]: osd.0 [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] boot 2024-09-21T06:04:16.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:15 smithi110 ceph-mon[34550]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:04:16.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:04:17.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:17.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:04:17.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/4094644573' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"} : dispatch 2024-09-21T06:04:17.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"} : dispatch 2024-09-21T06:04:17.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"}]': finished 2024-09-21T06:04:17.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:04:17.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:17.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:16 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/1893843380' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:04:17.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/4094644573' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"} : dispatch 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"} : dispatch 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"}]': finished 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:17.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:16 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/1893843380' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:04:17.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:04:17.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:04:17.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/4094644573' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"} : dispatch 2024-09-21T06:04:17.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"} : dispatch 2024-09-21T06:04:17.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0e6db75-1424-4686-b6d1-f889a7cf426c"}]': finished 2024-09-21T06:04:17.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:04:17.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:17.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:16 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/1893843380' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:04:19.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:18 smithi031 ceph-mon[31237]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:19.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:18 smithi110 ceph-mon[34550]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:19.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:18 smithi107 ceph-mon[34380]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:21.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:20 smithi031 ceph-mon[31237]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:21.195 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:20 smithi107 ceph-mon[34380]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:21.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:20 smithi110 ceph-mon[34550]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:23.091 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:22 smithi107 ceph-mon[34380]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:23.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:22 smithi031 ceph-mon[31237]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:22 smithi110 ceph-mon[34550]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:25.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:24 smithi031 ceph-mon[31237]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:04:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:24 smithi031 ceph-mon[31237]: Deploying daemon osd.1 on smithi107 2024-09-21T06:04:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:24 smithi107 ceph-mon[34380]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:04:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:24 smithi107 ceph-mon[34380]: Deploying daemon osd.1 on smithi107 2024-09-21T06:04:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:24 smithi110 ceph-mon[34550]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:04:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:24 smithi110 ceph-mon[34550]: Deploying daemon osd.1 on smithi107 2024-09-21T06:04:27.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:26 smithi107 ceph-mon[34380]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:27.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:26 smithi031 ceph-mon[31237]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:27.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:26 smithi110 ceph-mon[34550]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:29.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:28 smithi031 ceph-mon[31237]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:29.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:28 smithi110 ceph-mon[34550]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:29.284 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:28 smithi107 ceph-mon[34380]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:31.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:30 smithi031 ceph-mon[31237]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:31.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:31.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:31.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:31.154 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:30 smithi107 ceph-mon[34380]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:31.154 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:31.154 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:31.154 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:30 smithi110 ceph-mon[34550]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:32 smithi031 ceph-mon[31237]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:32 smithi110 ceph-mon[34550]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:33.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:33.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:32 smithi107 ceph-mon[34380]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.714+0000 7fd94d7fa640 1 -- 172.21.15.107:0/2072348204 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd950063800 con 0x7fd93c0b64d0 2024-09-21T06:04:34.716 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd93c0b64d0 msgr2=0x7fd93c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd93c0b64d0 0x7fd93c0b89e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd940009230 tx=0x7fd940031400 comp rx=0 tx=0).stop 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 msgr2=0x7fd950103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 0x7fd950103210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd94800cce0 tx=0x7fd94800a620 comp rx=0 tx=0).stop 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 shutdown_connections 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95010de30 0x7fd950103210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd93c0b64d0 0x7fd93c0b89e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9501052b0 0x7fd950102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 --2- 172.21.15.107:0/2072348204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd950104880 0x7fd950102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 >> 172.21.15.107:0/2072348204 conn(0x7fd950079730 msgr2=0x7fd9500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 shutdown_connections 2024-09-21T06:04:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:04:34.716+0000 7fd955ff0640 1 -- 172.21.15.107:0/2072348204 wait complete. 2024-09-21T06:04:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:34 smithi107 ceph-mon[34380]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:35.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:34 smithi031 ceph-mon[31237]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:35.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:35.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:35.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:34 smithi110 ceph-mon[34550]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:35.329 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T06:04:35.331 INFO:tasks.cephadm:Deploying osd.2 on smithi110 with /dev/vg_nvme/lv_4... 2024-09-21T06:04:35.331 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:04:35.582 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:04:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:36 smithi107 ceph-mon[34380]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:36 smithi107 ceph-mon[34380]: from='osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:04:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:36 smithi107 ceph-mon[34380]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:04:37.040 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:04:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1[39074]: 2024-09-21T06:04:36.605+0000 7f20ce171640 -1 osd.1 0 log_to_monitors true 2024-09-21T06:04:37.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:36 smithi031 ceph-mon[31237]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:37.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:36 smithi031 ceph-mon[31237]: from='osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:04:37.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:36 smithi031 ceph-mon[31237]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:04:37.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:36 smithi110 ceph-mon[34550]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:37.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:36 smithi110 ceph-mon[34550]: from='osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:04:37.221 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:36 smithi110 ceph-mon[34550]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:04:37.909 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:37 smithi110 ceph-mon[34550]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:04:37.909 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:37 smithi110 ceph-mon[34550]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:04:37.909 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:37 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:37.909 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:37 smithi110 ceph-mon[34550]: from='osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:04:37.909 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:37 smithi110 ceph-mon[34550]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:04:37.912 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:04:37.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-09-21T06:04:38.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:37 smithi031 ceph-mon[31237]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:04:38.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:37 smithi031 ceph-mon[31237]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:04:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:37 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:37 smithi031 ceph-mon[31237]: from='osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:04:38.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:37 smithi031 ceph-mon[31237]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:04:38.214 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:04:38.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:37 smithi107 ceph-mon[34380]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:04:38.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:37 smithi107 ceph-mon[34380]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:04:38.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:37 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:38.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:37 smithi107 ceph-mon[34380]: from='osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:04:38.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:37 smithi107 ceph-mon[34380]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:04:38.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 -- 172.21.15.110:0/2978483609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71410de40 msgr2=0x7fc714110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 --2- 172.21.15.110:0/2978483609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71410de40 0x7fc714110330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009ca0 tx=0x7fc70c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 -- 172.21.15.110:0/2978483609 shutdown_connections 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 --2- 172.21.15.110:0/2978483609 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc71410de40 0x7fc714110330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 --2- 172.21.15.110:0/2978483609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc714105330 0x7fc71410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 --2- 172.21.15.110:0/2978483609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 0x7fc714104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 -- 172.21.15.110:0/2978483609 >> 172.21.15.110:0/2978483609 conn(0x7fc714079750 msgr2=0x7fc714079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 -- 172.21.15.110:0/2978483609 shutdown_connections 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.664+0000 7fc718cbb640 1 -- 172.21.15.110:0/2978483609 wait complete. 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 Processor -- start 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 -- start start 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 0x7fc71419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 0x7fc71419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71410de40 0x7fc71419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc714113ac0 con 0x7fc71410de40 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc714113940 con 0x7fc7141049e0 2024-09-21T06:04:38.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc718cbb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc714113c40 con 0x7fc714105330 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 0x7fc71419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 0x7fc71419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59830/0 (socket says 172.21.15.110:59830) 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 -- 172.21.15.110:0/1593074489 learned_addr learned my addr 172.21.15.110:0/1593074489 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc71259c640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 0x7fc71419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc712d9d640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71410de40 0x7fc71419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 -- 172.21.15.110:0/1593074489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 msgr2=0x7fc71419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 0x7fc71419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 -- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71410de40 msgr2=0x7fc71419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71410de40 0x7fc71419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc711d9b640 1 -- 172.21.15.110:0/1593074489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141a03a0 con 0x7fc714105330 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc71259c640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 0x7fc71419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.665+0000 7fc712d9d640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71410de40 0x7fc71419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:38.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.666+0000 7fc711d9b640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 0x7fc71419a9d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c970 tx=0x7fc70800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.666+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708007bb0 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.666+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7141a0690 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.666+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708007d50 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.666+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70800f450 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.666+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7141a6df0 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.667+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71406c280 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.668+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fc708004eb0 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.669+0000 7fc7037fe640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6fc0b64d0 0x7fc6fc0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.669+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7fc7080e5bc0 con 0x7fc714105330 2024-09-21T06:04:38.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.669+0000 7fc71259c640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6fc0b64d0 0x7fc6fc0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:38.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.670+0000 7fc71259c640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6fc0b64d0 0x7fc6fc0b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc71406c980 tx=0x7fc704005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:38.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.673+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc708014030 con 0x7fc714105330 2024-09-21T06:04:38.860 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:04:38.858+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc714106f40 con 0x7fc6fc0b64d0 2024-09-21T06:04:39.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:39.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:04:39.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi107 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:38 smithi110 ceph-mon[34550]: from='osd.1 ' entity='osd.1' 2024-09-21T06:04:39.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi107 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:39.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:39.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:38 smithi031 ceph-mon[31237]: from='osd.1 ' entity='osd.1' 2024-09-21T06:04:39.289 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1[39074]: 2024-09-21T06:04:38.829+0000 7f20ca0f3640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T06:04:39.289 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1[39074]: 2024-09-21T06:04:38.958+0000 7f20c1ee9640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:04:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:04:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi107 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi107 to 17087M 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:39.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:38 smithi107 ceph-mon[34380]: from='osd.1 ' entity='osd.1' 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:39 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:40.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:04:40.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:04:40.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:04:40.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:04:40.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:04:40.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:40.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:39 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:04:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:04:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:04:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:04:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:04:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:40.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:39 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:41.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:40 smithi031 ceph-mon[31237]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:41.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:40 smithi031 ceph-mon[31237]: osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] boot 2024-09-21T06:04:41.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:40 smithi031 ceph-mon[31237]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:04:41.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:40 smithi110 ceph-mon[34550]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:40 smithi110 ceph-mon[34550]: osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] boot 2024-09-21T06:04:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:40 smithi110 ceph-mon[34550]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:04:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:40 smithi107 ceph-mon[34380]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:04:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:40 smithi107 ceph-mon[34380]: osd.1 [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] boot 2024-09-21T06:04:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:40 smithi107 ceph-mon[34380]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:04:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:04:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:04:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/3337235619' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"} : dispatch 2024-09-21T06:04:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"} : dispatch 2024-09-21T06:04:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"}]': finished 2024-09-21T06:04:42.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:04:42.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:04:42.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:41 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/2855828664' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:04:42.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:04:42.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/3337235619' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"} : dispatch 2024-09-21T06:04:42.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"} : dispatch 2024-09-21T06:04:42.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"}]': finished 2024-09-21T06:04:42.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:04:42.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:04:42.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:41 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/2855828664' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:04:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:04:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/3337235619' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"} : dispatch 2024-09-21T06:04:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"} : dispatch 2024-09-21T06:04:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612"}]': finished 2024-09-21T06:04:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:04:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:04:42.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:41 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/2855828664' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:04:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:43 smithi110 ceph-mon[34550]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:43 smithi107 ceph-mon[34380]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:43 smithi031 ceph-mon[31237]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:45 smithi110 ceph-mon[34550]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:45 smithi107 ceph-mon[34380]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:45 smithi031 ceph-mon[31237]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:47.486 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:47 smithi110 ceph-mon[34550]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:47 smithi107 ceph-mon[34380]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:47 smithi031 ceph-mon[31237]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:49.474 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:49 smithi110 ceph-mon[34550]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:50.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:49 smithi031 ceph-mon[31237]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:50.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:49 smithi107 ceph-mon[34380]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:50.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:04:50.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:50.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:04:50.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:50.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:04:50.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:51 smithi110 ceph-mon[34550]: Deploying daemon osd.2 on smithi110 2024-09-21T06:04:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:51 smithi110 ceph-mon[34550]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:51 smithi107 ceph-mon[34380]: Deploying daemon osd.2 on smithi110 2024-09-21T06:04:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:51 smithi107 ceph-mon[34380]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:51 smithi031 ceph-mon[31237]: Deploying daemon osd.2 on smithi110 2024-09-21T06:04:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:51 smithi031 ceph-mon[31237]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:53.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:53 smithi110 ceph-mon[34550]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:53.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:53 smithi107 ceph-mon[34380]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:53 smithi031 ceph-mon[31237]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:55.375 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:55 smithi110 ceph-mon[34550]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:55 smithi107 ceph-mon[34380]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:55 smithi031 ceph-mon[31237]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:56.486 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:56.486 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:56.486 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:56.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:56.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:56.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:56.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:57.503 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:57 smithi110 ceph-mon[34550]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:57.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:57 smithi107 ceph-mon[34380]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:57 smithi031 ceph-mon[31237]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:58 smithi110 ceph-mon[34550]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:59.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:59.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:04:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:58 smithi107 ceph-mon[34380]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:04:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:58 smithi031 ceph-mon[31237]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:04:59.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:04:59.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:59.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:59.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:04:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:00.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.122+0000 7fc7037fe640 1 -- 172.21.15.110:0/1593074489 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc714106f40 con 0x7fc6fc0b64d0 2024-09-21T06:05:00.124 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 2 on host 'smithi110' 2024-09-21T06:05:00.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6fc0b64d0 msgr2=0x7fc6fc0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6fc0b64d0 0x7fc6fc0b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc71406c980 tx=0x7fc704005f90 comp rx=0 tx=0).stop 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 msgr2=0x7fc71419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 0x7fc71419a9d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c970 tx=0x7fc70800ce30 comp rx=0 tx=0).stop 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 shutdown_connections 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6fc0b64d0 0x7fc6fc0b89e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc71410de40 0x7fc71419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc714105330 0x7fc71419a9d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 --2- 172.21.15.110:0/1593074489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7141049e0 0x7fc71419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.124+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 >> 172.21.15.110:0/1593074489 conn(0x7fc714079750 msgr2=0x7fc714105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.125+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 shutdown_connections 2024-09-21T06:05:00.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:00.125+0000 7fc718cbb640 1 -- 172.21.15.110:0/1593074489 wait complete. 2024-09-21T06:05:00.828 DEBUG:teuthology.orchestra.run.smithi110:osd.2> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T06:05:00.830 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2024-09-21T06:05:00.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:05:01.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:00 smithi110 ceph-mon[34550]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:01.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:01.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:01.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:01.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:00 smithi031 ceph-mon[31237]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:01.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:01.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:01.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:00 smithi107 ceph-mon[34380]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:02.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:01 smithi110 ceph-mon[34550]: from='osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:05:02.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:01 smithi110 ceph-mon[34550]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:05:02.061 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:01 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T06:05:01.794+0000 7fe649b65640 -1 osd.2 0 log_to_monitors true 2024-09-21T06:05:02.269 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:01 smithi031 ceph-mon[31237]: from='osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:05:02.269 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:01 smithi031 ceph-mon[31237]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:05:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:01 smithi107 ceph-mon[34380]: from='osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:05:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:01 smithi107 ceph-mon[34380]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:05:03.139 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:02 smithi110 ceph-mon[34550]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:03.140 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:02 smithi110 ceph-mon[34550]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:05:03.140 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:02 smithi110 ceph-mon[34550]: from='osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:05:03.140 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:02 smithi110 ceph-mon[34550]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:05:03.140 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:02 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:03.140 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:02 smithi110 ceph-mon[34550]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:05:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:02 smithi107 ceph-mon[34380]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:02 smithi107 ceph-mon[34380]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:05:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:02 smithi107 ceph-mon[34380]: from='osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:05:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:02 smithi107 ceph-mon[34380]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:05:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:02 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:02 smithi107 ceph-mon[34380]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:05:03.364 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:02 smithi031 ceph-mon[31237]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:02 smithi031 ceph-mon[31237]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:05:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:02 smithi031 ceph-mon[31237]: from='osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:05:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:02 smithi031 ceph-mon[31237]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:05:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:02 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:03.365 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:02 smithi031 ceph-mon[31237]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:05:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:05:03.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-09-21T06:05:03.950 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T06:05:03.690+0000 7fe645ae7640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T06:05:03.950 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T06:05:03.695+0000 7fe63d8dd640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:05:03.984 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:04.203 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:04.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:04.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.204 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:03 smithi031 ceph-mon[31237]: from='osd.2 ' entity='osd.2' 2024-09-21T06:05:04.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:05:04.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:05:04.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:03 smithi110 ceph-mon[34550]: from='osd.2 ' entity='osd.2' 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:04.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:04.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:04.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:03 smithi107 ceph-mon[34380]: from='osd.2 ' entity='osd.2' 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 -- 172.21.15.31:0/1291248171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3641094a0 msgr2=0x7fe3641098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/1291248171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3641094a0 0x7fe3641098b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe360009900 tx=0x7fe36002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 -- 172.21.15.31:0/1291248171 shutdown_connections 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/1291248171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe364109df0 0x7fe364110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/1291248171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3641094a0 0x7fe3641098b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/1291248171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe364103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 -- 172.21.15.31:0/1291248171 >> 172.21.15.31:0/1291248171 conn(0x7fe36407b850 msgr2=0x7fe36407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 -- 172.21.15.31:0/1291248171 shutdown_connections 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.433+0000 7fe36c7cd640 1 -- 172.21.15.31:0/1291248171 wait complete. 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 Processor -- start 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 -- start start 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641094a0 0x7fe36419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 0x7fe36419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe364113ab0 con 0x7fe364109df0 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe364113930 con 0x7fe3641094a0 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36c7cd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe364113c30 con 0x7fe364103b60 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 0x7fe36419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 0x7fe36419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54340/0 (socket says 172.21.15.31:54340) 2024-09-21T06:05:04.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 -- 172.21.15.31:0/2180525677 learned_addr learned my addr 172.21.15.31:0/2180525677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36a569640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe369d68640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641094a0 0x7fe36419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 -- 172.21.15.31:0/2180525677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 msgr2=0x7fe36419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 -- 172.21.15.31:0/2180525677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641094a0 msgr2=0x7fe36419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641094a0 0x7fe36419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.434+0000 7fe36ad6a640 1 -- 172.21.15.31:0/2180525677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3641a0410 con 0x7fe364109df0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.435+0000 7fe36ad6a640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 0x7fe36419fce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00eaa0 tx=0x7fe35c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.435+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00cdb0 con 0x7fe364109df0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.435+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3641a0700 con 0x7fe364109df0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.435+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35c005020 con 0x7fe364109df0 2024-09-21T06:05:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.435+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0116d0 con 0x7fe364109df0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.435+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3641a6e60 con 0x7fe364109df0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.436+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe364103f70 con 0x7fe364109df0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.437+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fe35c0051d0 con 0x7fe364109df0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.438+0000 7fe3577fe640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3500b64d0 0x7fe3500b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.438+0000 7fe36a569640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3500b64d0 0x7fe3500b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.438+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7fe35c0e61d0 con 0x7fe364109df0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.439+0000 7fe36a569640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3500b64d0 0x7fe3500b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe358009bd0 tx=0x7fe358005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.441+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe35c016030 con 0x7fe364109df0 2024-09-21T06:05:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:04.647+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe364063800 con 0x7fe3500b64d0 2024-09-21T06:05:05.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:05:05.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:05:05.256 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi110 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] boot 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:05:05.257 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:05:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:05:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi110 2024-09-21T06:05:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] boot 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:05:05.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi110 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi110 to 17088M 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: osd.2 [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] boot 2024-09-21T06:05:05.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:05:05.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:05:06.001 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:05 smithi031 ceph-mon[31237]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:06.002 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:05 smithi031 ceph-mon[31237]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:05:06.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:05 smithi110 ceph-mon[34550]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:06.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:05 smithi110 ceph-mon[34550]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:05:06.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:05 smithi107 ceph-mon[34380]: from='client.14511 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:06.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:05 smithi107 ceph-mon[34380]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:05:07.220 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:06 smithi107 ceph-mon[34380]: pgmap v67: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:07.220 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:05:07.220 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42917]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:05:07.220 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42917]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.221 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42917]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.221 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42917]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.275 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:06 smithi110 ceph-mon[34550]: pgmap v67: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:07.275 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:05:07.343 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:06 smithi031 ceph-mon[31237]: pgmap v67: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:07.343 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53343]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53343]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53343]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53346]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.343 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53346]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.344 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53346]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.528 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:05:07.528 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.528 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.529 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43235]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.529 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:05:07.529 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.529 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43238]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.529 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43238]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:05:07.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43241]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43241]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:05:07.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43244]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 sudo[43244]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.534 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:05:07.534 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.534 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.535 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42920]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42923]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42923]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42923]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.535 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42926]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 sudo[42926]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53350]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:05:07.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53350]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53350]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53350]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:07.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53358]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:05:07.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53358]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:05:07.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53358]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:05:07.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 sudo[53358]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/680681194' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24b23ec1-a44d-4d63-9a6e-fab54498c7db"} : dispatch 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/680681194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24b23ec1-a44d-4d63-9a6e-fab54498c7db"}]': finished 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.281 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:07 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3753153756' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:08.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:05:08.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:05:08.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:05:08.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/680681194' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24b23ec1-a44d-4d63-9a6e-fab54498c7db"} : dispatch 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/680681194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24b23ec1-a44d-4d63-9a6e-fab54498c7db"}]': finished 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:07 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3753153756' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:08.306 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:05:08.306 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:05:08.306 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:05:08.306 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/680681194' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24b23ec1-a44d-4d63-9a6e-fab54498c7db"} : dispatch 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/680681194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24b23ec1-a44d-4d63-9a6e-fab54498c7db"}]': finished 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.307 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:05:08.308 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:07 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3753153756' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:08.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:08.423+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe35c0a9cf0 con 0x7fe364109df0 2024-09-21T06:05:09.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:09 smithi031 ceph-mon[31237]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:09.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:09 smithi031 ceph-mon[31237]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:05:09.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:09.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:09 smithi031 ceph-mon[31237]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T06:05:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:09 smithi110 ceph-mon[34550]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:09 smithi110 ceph-mon[34550]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:05:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:09 smithi110 ceph-mon[34550]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T06:05:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:09 smithi107 ceph-mon[34380]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:09 smithi107 ceph-mon[34380]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:05:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:09 smithi107 ceph-mon[34380]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T06:05:11.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:11 smithi031 ceph-mon[31237]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:11 smithi110 ceph-mon[34550]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:11 smithi107 ceph-mon[34380]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:13.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:13 smithi031 ceph-mon[31237]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:13.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:13 smithi110 ceph-mon[34550]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:13 smithi107 ceph-mon[34380]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:15.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:15 smithi031 ceph-mon[31237]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:15.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:15 smithi110 ceph-mon[34550]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:15.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:15 smithi107 ceph-mon[34380]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:16.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:05:16.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:16 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:16.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:05:16.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:16 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:16.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:05:16.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:16 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:17.382 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:17 smithi031 ceph-mon[31237]: Deploying daemon osd.3 on smithi031 2024-09-21T06:05:17.382 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:17 smithi031 ceph-mon[31237]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:17.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:17 smithi110 ceph-mon[34550]: Deploying daemon osd.3 on smithi031 2024-09-21T06:05:17.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:17 smithi110 ceph-mon[34550]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:17 smithi107 ceph-mon[34380]: Deploying daemon osd.3 on smithi031 2024-09-21T06:05:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:17 smithi107 ceph-mon[34380]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:19.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:19 smithi031 ceph-mon[31237]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:19.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:19 smithi110 ceph-mon[34550]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:19 smithi107 ceph-mon[34380]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:21.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:21 smithi031 ceph-mon[31237]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:21.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:21 smithi110 ceph-mon[34550]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:21.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:21 smithi107 ceph-mon[34380]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:22.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:22.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:22.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:22.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:22.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:22.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:22.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:22.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:22.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:23.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:23 smithi031 ceph-mon[31237]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:23.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:23 smithi110 ceph-mon[34550]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:23.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:23 smithi107 ceph-mon[34380]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:24.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.933+0000 7fe3577fe640 1 -- 172.21.15.31:0/2180525677 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe364063800 con 0x7fe3500b64d0 2024-09-21T06:05:24.935 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3500b64d0 msgr2=0x7fe3500b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3500b64d0 0x7fe3500b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe358009bd0 tx=0x7fe358005f90 comp rx=0 tx=0).stop 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 msgr2=0x7fe36419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 0x7fe36419fce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00eaa0 tx=0x7fe35c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 shutdown_connections 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3500b64d0 0x7fe3500b89e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe364109df0 0x7fe36419fce0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3641094a0 0x7fe36419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 --2- 172.21.15.31:0/2180525677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe364103b60 0x7fe36419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:24.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 >> 172.21.15.31:0/2180525677 conn(0x7fe36407b850 msgr2=0x7fe364107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:24.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 shutdown_connections 2024-09-21T06:05:24.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:05:24.936+0000 7fe36c7cd640 1 -- 172.21.15.31:0/2180525677 wait complete. 2024-09-21T06:05:25.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:25.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:25.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:25.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:25.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:25.549 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.550 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:25.577 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T06:05:25.579 INFO:tasks.cephadm:Deploying osd.4 on smithi107 with /dev/vg_nvme/lv_3... 2024-09-21T06:05:25.579 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:05:25.833 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:05:27.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:27 smithi107 ceph-mon[34380]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:27.363 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:27 smithi031 ceph-mon[31237]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:27 smithi110 ceph-mon[34550]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:28.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T06:05:28.136 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-09-21T06:05:28.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:28 smithi107 ceph-mon[34380]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:05:28.380 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:05:28.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:28 smithi031 ceph-mon[31237]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:05:28.386 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:05:28 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:05:28.053+0000 7f154e92f640 -1 osd.3 0 log_to_monitors true 2024-09-21T06:05:28.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:28 smithi110 ceph-mon[34550]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 -- 172.21.15.107:0/1596014025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3901094a0 msgr2=0x7fc3901098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 --2- 172.21.15.107:0/1596014025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3901094a0 0x7fc3901098b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc38402a410 comp rx=0 tx=0).stop 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 -- 172.21.15.107:0/1596014025 shutdown_connections 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 --2- 172.21.15.107:0/1596014025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc390109df0 0x7fc390110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 --2- 172.21.15.107:0/1596014025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3901094a0 0x7fc3901098b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 --2- 172.21.15.107:0/1596014025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 0x7fc390103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 -- 172.21.15.107:0/1596014025 >> 172.21.15.107:0/1596014025 conn(0x7fc39007b850 msgr2=0x7fc39007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 -- 172.21.15.107:0/1596014025 shutdown_connections 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.849+0000 7fc398338640 1 -- 172.21.15.107:0/1596014025 wait complete. 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 Processor -- start 2024-09-21T06:05:28.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 -- start start 2024-09-21T06:05:28.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 0x7fc390075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:28.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3901094a0 0x7fc390075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:28.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 0x7fc390076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:28.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc390113ab0 con 0x7fc390103b60 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc390113930 con 0x7fc390109df0 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.850+0000 7fc398338640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc390113c30 con 0x7fc3901094a0 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 0x7fc390076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 0x7fc390076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60126/0 (socket says 172.21.15.107:60126) 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 -- 172.21.15.107:0/2264059556 learned_addr learned my addr 172.21.15.107:0/2264059556 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3960d4640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 0x7fc390075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3958d3640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3901094a0 0x7fc390075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 -- 172.21.15.107:0/2264059556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3901094a0 msgr2=0x7fc390075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3901094a0 0x7fc390075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 -- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 msgr2=0x7fc390075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:28.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 0x7fc390075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 -- 172.21.15.107:0/2264059556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3901a8710 con 0x7fc390109df0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3960d4640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 0x7fc390075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3958d3640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3901094a0 0x7fc390075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc3968d5640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 0x7fc390076190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc38800cce0 tx=0x7fc38800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38800f450 con 0x7fc390109df0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc38800faa0 con 0x7fc390109df0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3901a8a00 con 0x7fc390109df0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388018b70 con 0x7fc390109df0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.851+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901a8ef0 con 0x7fc390109df0 2024-09-21T06:05:28.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.852+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc390103f70 con 0x7fc390109df0 2024-09-21T06:05:28.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.854+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc388018d10 con 0x7fc390109df0 2024-09-21T06:05:28.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.854+0000 7fc38f7fe640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3780b6560 0x7fc3780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:28.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.855+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3588+0+0 (secure 0 0 0) 0x7fc3880e65e0 con 0x7fc390109df0 2024-09-21T06:05:28.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.855+0000 7fc3960d4640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3780b6560 0x7fc3780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:28.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.855+0000 7fc3960d4640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3780b6560 0x7fc3780b8a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc390077160 tx=0x7fc380005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:28.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:28.858+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc388014030 con 0x7fc390109df0 2024-09-21T06:05:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:29.048+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc390063800 con 0x7fc3780b6560 2024-09-21T06:05:29.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:29.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:05:29.418 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:29.419 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:29.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:29.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:29.420 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:29.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:29.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:29.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:05:29.183+0000 7f154a8b1640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T06:05:29.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:05:29.189+0000 7f15466af640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:29.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:30.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi031 2024-09-21T06:05:30.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-21T06:05:30.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:30.524 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:05:30.524 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:05:30.524 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:30.524 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:30.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi031 2024-09-21T06:05:30.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-21T06:05:30.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:30.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:05:30.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:05:30.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:30.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:30.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi031 2024-09-21T06:05:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi031 to 8546M 2024-09-21T06:05:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: from='osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:05:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:05:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] boot 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:31.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:31 smithi107 ceph-mon[34380]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:05:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:05:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:05:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] boot 2024-09-21T06:05:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:05:31.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:31.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:31 smithi110 ceph-mon[34550]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:05:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:05:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:05:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:05:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: osd.3 [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] boot 2024-09-21T06:05:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:05:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:05:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:31 smithi031 ceph-mon[31237]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:05:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:32 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/993652184' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"} : dispatch 2024-09-21T06:05:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:32 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"} : dispatch 2024-09-21T06:05:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:32 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"}]': finished 2024-09-21T06:05:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:32 smithi110 ceph-mon[34550]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T06:05:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:32 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/4060065132' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:32 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/993652184' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"} : dispatch 2024-09-21T06:05:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:32 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"} : dispatch 2024-09-21T06:05:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:32 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"}]': finished 2024-09-21T06:05:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:32 smithi107 ceph-mon[34380]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T06:05:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:32 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/4060065132' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:32.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:32 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/993652184' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"} : dispatch 2024-09-21T06:05:32.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:32 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"} : dispatch 2024-09-21T06:05:32.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:32 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5c6b1c9-2f76-4407-a87a-abcddb48fba3"}]': finished 2024-09-21T06:05:32.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:32 smithi031 ceph-mon[31237]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T06:05:32.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:32.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:32 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/4060065132' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:33 smithi031 ceph-mon[31237]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:33 smithi110 ceph-mon[34550]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:33 smithi107 ceph-mon[34380]: pgmap v89: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:35 smithi031 ceph-mon[31237]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:35 smithi110 ceph-mon[34550]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:35 smithi107 ceph-mon[34380]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:37.348 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:37 smithi107 ceph-mon[34380]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:05:37.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:37 smithi031 ceph-mon[31237]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:05:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:37 smithi110 ceph-mon[34550]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:05:39.518 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:39 smithi107 ceph-mon[34380]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T06:05:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:39 smithi031 ceph-mon[31237]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T06:05:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:39 smithi110 ceph-mon[34550]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T06:05:40.487 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:05:40.487 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:40.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:05:40.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:40.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:05:40.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:41.437 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:41 smithi107 ceph-mon[34380]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T06:05:41.437 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:41 smithi107 ceph-mon[34380]: Deploying daemon osd.4 on smithi107 2024-09-21T06:05:41.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:41 smithi031 ceph-mon[31237]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T06:05:41.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:41 smithi031 ceph-mon[31237]: Deploying daemon osd.4 on smithi107 2024-09-21T06:05:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:41 smithi110 ceph-mon[34550]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T06:05:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:41 smithi110 ceph-mon[34550]: Deploying daemon osd.4 on smithi107 2024-09-21T06:05:43.456 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:43 smithi107 ceph-mon[34380]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:05:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:43 smithi031 ceph-mon[31237]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:05:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:43 smithi110 ceph-mon[34550]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:05:45.471 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:45 smithi107 ceph-mon[34380]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:05:45.580 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:45 smithi110 ceph-mon[34550]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:05:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:45 smithi031 ceph-mon[31237]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:05:46.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:46.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:46.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:46.652 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:46.652 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:46.652 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:47 smithi031 ceph-mon[31237]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:05:47.637 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:47 smithi107 ceph-mon[34380]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:05:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:47 smithi110 ceph-mon[34550]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:05:49.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:49 smithi107 ceph-mon[34380]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:49.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:49 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:49 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:49 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:49.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:49 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:49.523 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:49 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:49 smithi031 ceph-mon[31237]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:49.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:49 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:49 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:49 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:49.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:49 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:49.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:49 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:49 smithi110 ceph-mon[34550]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:49 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:49 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:49 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:49.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:49 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:49.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:49 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.825+0000 7fc38f7fe640 1 -- 172.21.15.107:0/2264059556 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc390063800 con 0x7fc3780b6560 2024-09-21T06:05:49.828 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 4 on host 'smithi107' 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3780b6560 msgr2=0x7fc3780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3780b6560 0x7fc3780b8a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc390077160 tx=0x7fc380005f90 comp rx=0 tx=0).stop 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 msgr2=0x7fc390076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 0x7fc390076190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc38800cce0 tx=0x7fc38800a620 comp rx=0 tx=0).stop 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 shutdown_connections 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc390109df0 0x7fc390076190 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3901094a0 0x7fc390075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3780b6560 0x7fc3780b8a70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 --2- 172.21.15.107:0/2264059556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc390103b60 0x7fc390075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.827+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 >> 172.21.15.107:0/2264059556 conn(0x7fc39007b850 msgr2=0x7fc390105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.828+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 shutdown_connections 2024-09-21T06:05:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:05:49.828+0000 7fc398338640 1 -- 172.21.15.107:0/2264059556 wait complete. 2024-09-21T06:05:50.402 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:50.402 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:50.402 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:50.429 DEBUG:teuthology.orchestra.run.smithi107:osd.4> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T06:05:50.431 INFO:tasks.cephadm:Deploying osd.5 on smithi110 with /dev/vg_nvme/lv_3... 2024-09-21T06:05:50.431 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:05:50.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:50.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:50.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:50.690 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:05:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:51.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:51 smithi110 ceph-mon[34550]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:51.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:51 smithi031 ceph-mon[31237]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:51.644 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:51 smithi107 ceph-mon[34380]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:51.970 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:05:51 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4[45525]: 2024-09-21T06:05:51.860+0000 7fc526814640 -1 osd.4 0 log_to_monitors true 2024-09-21T06:05:52.460 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:52 smithi110 ceph-mon[34550]: from='osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:05:52.460 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:52 smithi110 ceph-mon[34550]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:05:52.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:52 smithi031 ceph-mon[31237]: from='osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:05:52.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:52 smithi031 ceph-mon[31237]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:05:52.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:52 smithi107 ceph-mon[34380]: from='osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:05:52.782 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:52 smithi107 ceph-mon[34380]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:05:53.213 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:05:53.242 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-09-21T06:05:53.444 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:53.445 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:53.446 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.493 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:53.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:53.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:53.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.789 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4[45525]: 2024-09-21T06:05:53.356+0000 7fc522796640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T06:05:53.789 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4[45525]: 2024-09-21T06:05:53.363+0000 7fc51e594640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:05:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 -- 172.21.15.110:0/1119670099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94001083d0 msgr2=0x7f94001087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 --2- 172.21.15.110:0/1119670099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94001083d0 0x7f94001087e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 -- 172.21.15.110:0/1119670099 shutdown_connections 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 --2- 172.21.15.110:0/1119670099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400108d20 0x7f940010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 --2- 172.21.15.110:0/1119670099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94001083d0 0x7f94001087e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 --2- 172.21.15.110:0/1119670099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9400102ae0 0x7f9400102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 -- 172.21.15.110:0/1119670099 >> 172.21.15.110:0/1119670099 conn(0x7f9400079760 msgr2=0x7f9400079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 -- 172.21.15.110:0/1119670099 shutdown_connections 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.988+0000 7f94054ba640 1 -- 172.21.15.110:0/1119670099 wait complete. 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 Processor -- start 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 -- start start 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400102ae0 0x7f940010ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94001083d0 0x7f94001104e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 0x7f9400110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113ab0 con 0x7f94001083d0 2024-09-21T06:05:53.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113930 con 0x7f9400102ae0 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f94054ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113c30 con 0x7f9400108d20 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 0x7f9400110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f93ff7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 0x7f9400110a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52138/0 (socket says 172.21.15.110:52138) 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f93ff7fe640 1 -- 172.21.15.110:0/1339695923 learned_addr learned my addr 172.21.15.110:0/1339695923 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.989+0000 7f93feffd640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400102ae0 0x7f940010ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93fe7fc640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94001083d0 0x7f94001104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93ff7fe640 1 -- 172.21.15.110:0/1339695923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400102ae0 msgr2=0x7f940010ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93ff7fe640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400102ae0 0x7f940010ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93ff7fe640 1 -- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94001083d0 msgr2=0x7f94001104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93ff7fe640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94001083d0 0x7f94001104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93ff7fe640 1 -- 172.21.15.110:0/1339695923 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94001a86c0 con 0x7f9400108d20 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93fe7fc640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94001083d0 0x7f94001104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:53.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93feffd640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400102ae0 0x7f940010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:53.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93ff7fe640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 0x7f9400110a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f93f800e970 tx=0x7f93f800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800ccc0 con 0x7f9400108d20 2024-09-21T06:05:53.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94001a89b0 con 0x7f9400108d20 2024-09-21T06:05:53.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f800ce60 con 0x7f9400108d20 2024-09-21T06:05:53.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8010430 con 0x7f9400108d20 2024-09-21T06:05:53.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.990+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001a8f40 con 0x7f9400108d20 2024-09-21T06:05:53.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.991+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9400102ef0 con 0x7f9400108d20 2024-09-21T06:05:53.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.993+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f93f8004f90 con 0x7f9400108d20 2024-09-21T06:05:53.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.994+0000 7f93e7fff640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93e80b6560 0x7f93e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.994+0000 7f93feffd640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93e80b6560 0x7f93e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.994+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4036+0+0 (secure 0 0 0) 0x7f93f80e6830 con 0x7f9400108d20 2024-09-21T06:05:53.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.994+0000 7f93feffd640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93e80b6560 0x7f93e80b8a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94001119f0 tx=0x7f93f4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:54.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:53.998+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93f80ae170 con 0x7f9400108d20 2024-09-21T06:05:54.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:05:54.203+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9400063800 con 0x7f93e80b6560 2024-09-21T06:05:54.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi107 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:54.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:54.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi107 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi107 2024-09-21T06:05:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi107 to 8543M 2024-09-21T06:05:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:05:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:05:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:54.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:05:54.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:05:54.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:55.544 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:05:55.544 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:05:55.544 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:55.544 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:55.545 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] boot 2024-09-21T06:05:55.545 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T06:05:55.545 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:05:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:05:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] boot 2024-09-21T06:05:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T06:05:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:05:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:05:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:05:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:05:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: osd.4 [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] boot 2024-09-21T06:05:55.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T06:05:55.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:05:56.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:56 smithi110 ceph-mon[34550]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:05:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:56 smithi031 ceph-mon[31237]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:05:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:56 smithi107 ceph-mon[34380]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:05:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:05:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/3734692668' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"} : dispatch 2024-09-21T06:05:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"} : dispatch 2024-09-21T06:05:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"}]': finished 2024-09-21T06:05:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:05:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:05:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:57 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/383357379' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/3734692668' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"} : dispatch 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"} : dispatch 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"}]': finished 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:05:57.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:57 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/383357379' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/3734692668' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"} : dispatch 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"} : dispatch 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44342172-be5e-4e39-ad8e-0aed23c0efb1"}]': finished 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:05:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:57 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/383357379' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:05:58.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:05:58 smithi031 ceph-mon[31237]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:05:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:05:58 smithi110 ceph-mon[34550]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:05:58.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:05:58 smithi107 ceph-mon[34380]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:00.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:00 smithi031 ceph-mon[31237]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:01.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:00 smithi110 ceph-mon[34550]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:00 smithi107 ceph-mon[34380]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:02.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:02 smithi110 ceph-mon[34550]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:02.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:02 smithi031 ceph-mon[31237]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:03.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:02 smithi107 ceph-mon[34380]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:04.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:04 smithi110 ceph-mon[34550]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:04.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:04 smithi031 ceph-mon[31237]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:05.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:04 smithi107 ceph-mon[34380]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:06:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:05.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:05 smithi031 ceph-mon[31237]: Deploying daemon osd.5 on smithi110 2024-09-21T06:06:05.930 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:06:05.930 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:05.930 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:05 smithi110 ceph-mon[34550]: Deploying daemon osd.5 on smithi110 2024-09-21T06:06:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:06:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:05 smithi107 ceph-mon[34380]: Deploying daemon osd.5 on smithi110 2024-09-21T06:06:06.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:06 smithi110 ceph-mon[34550]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:06.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:06 smithi031 ceph-mon[31237]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:07.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:06 smithi107 ceph-mon[34380]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:08.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:08 smithi031 ceph-mon[31237]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:08 smithi110 ceph-mon[34550]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:08 smithi107 ceph-mon[34380]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:10 smithi110 ceph-mon[34550]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:10.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:10 smithi031 ceph-mon[31237]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:11.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:10 smithi107 ceph-mon[34380]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:11.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:11.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:11.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:12.810 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:12 smithi110 ceph-mon[34550]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:13.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:12 smithi031 ceph-mon[31237]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:12 smithi107 ceph-mon[34380]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:14 smithi110 ceph-mon[34550]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:14.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:14.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:14.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:14 smithi107 ceph-mon[34380]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:14.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:14.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:14.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:14.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:14 smithi031 ceph-mon[31237]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:14.908 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 5 on host 'smithi110' 2024-09-21T06:06:14.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.906+0000 7f93e7fff640 1 -- 172.21.15.110:0/1339695923 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9400063800 con 0x7f93e80b6560 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93e80b6560 msgr2=0x7f93e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93e80b6560 0x7f93e80b8a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f94001119f0 tx=0x7f93f4009500 comp rx=0 tx=0).stop 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 msgr2=0x7f9400110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 0x7f9400110a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f93f800e970 tx=0x7f93f800ed70 comp rx=0 tx=0).stop 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 shutdown_connections 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9400108d20 0x7f9400110a20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93e80b6560 0x7f93e80b8a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94001083d0 0x7f94001104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 --2- 172.21.15.110:0/1339695923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400102ae0 0x7f940010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 >> 172.21.15.110:0/1339695923 conn(0x7f9400079760 msgr2=0x7f9400105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 shutdown_connections 2024-09-21T06:06:14.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:06:14.908+0000 7f94054ba640 1 -- 172.21.15.110:0/1339695923 wait complete. 2024-09-21T06:06:15.589 DEBUG:teuthology.orchestra.run.smithi110:osd.5> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T06:06:15.591 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2024-09-21T06:06:15.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:06:15.787 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:15.787 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:15.787 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:15.804 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:15.804 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:15.804 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:16.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:16 smithi110 ceph-mon[34550]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:16.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:16 smithi107 ceph-mon[34380]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:16.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:16 smithi031 ceph-mon[31237]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:17.130 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:06:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:06:16.877+0000 7f0e5bbfc640 -1 osd.5 0 log_to_monitors true 2024-09-21T06:06:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:17 smithi031 ceph-mon[31237]: from='osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:06:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:17 smithi031 ceph-mon[31237]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:06:17.919 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:17 smithi107 ceph-mon[34380]: from='osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:06:17.919 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:17 smithi107 ceph-mon[34380]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:06:18.017 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:17 smithi110 ceph-mon[34550]: from='osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:06:18.017 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:17 smithi110 ceph-mon[34550]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:06:18.360 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:06:18.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-09-21T06:06:18.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:18.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:18.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:18.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:18.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:18.780 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:18.780 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:06:18.553+0000 7f0e57b7e640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T06:06:18.780 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:06:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:06:18.562+0000 7f0e5397c640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:06:19.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:06:19.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:19.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:06:19.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:06:19.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:19.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:19.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 -- 172.21.15.31:0/4231810351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 msgr2=0x7fdf641029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/4231810351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf641029e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009900 tx=0x7fdf6002a480 comp rx=0 tx=0).stop 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 -- 172.21.15.31:0/4231810351 shutdown_connections 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/4231810351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf64110180 0x7fdf641125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/4231810351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf64102f20 0x7fdf6410fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/4231810351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf641029e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 -- 172.21.15.31:0/4231810351 >> 172.21.15.31:0/4231810351 conn(0x7fdf64079730 msgr2=0x7fdf64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 -- 172.21.15.31:0/4231810351 shutdown_connections 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.097+0000 7fdf6c14b640 1 -- 172.21.15.31:0/4231810351 wait complete. 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 Processor -- start 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 -- start start 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf6419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf64102f20 0x7fdf6419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf64110180 0x7fdf641a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64115c70 con 0x7fdf641025d0 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64115af0 con 0x7fdf64110180 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf6c14b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64115df0 con 0x7fdf64102f20 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.098+0000 7fdf69ee7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf6419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf6419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56248/0 (socket says 172.21.15.31:56248) 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 -- 172.21.15.31:0/1006774031 learned_addr learned my addr 172.21.15.31:0/1006774031 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf6a6e8640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf64110180 0x7fdf641a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf696e6640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf64102f20 0x7fdf6419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 -- 172.21.15.31:0/1006774031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf64102f20 msgr2=0x7fdf6419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf64102f20 0x7fdf6419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 -- 172.21.15.31:0/1006774031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf64110180 msgr2=0x7fdf641a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf64110180 0x7fdf641a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 -- 172.21.15.31:0/1006774031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf641a25d0 con 0x7fdf641025d0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf69ee7640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf6419c660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005940 tx=0x7fdf600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6003f030 con 0x7fdf641025d0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf600044c0 con 0x7fdf641025d0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf641a2860 con 0x7fdf641025d0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60035070 con 0x7fdf641025d0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.099+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf641a9020 con 0x7fdf641025d0 2024-09-21T06:06:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.101+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf641029e0 con 0x7fdf641025d0 2024-09-21T06:06:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.102+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fdf60042030 con 0x7fdf641025d0 2024-09-21T06:06:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.103+0000 7fdf5affd640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf500b6560 0x7fdf500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.103+0000 7fdf696e6640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf500b6560 0x7fdf500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.103+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4468+0+0 (secure 0 0 0) 0x7fdf60106a30 con 0x7fdf641025d0 2024-09-21T06:06:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.104+0000 7fdf696e6640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf500b6560 0x7fdf500b8a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf64079fa0 tx=0x7fdf54005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.106+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdf600d25a0 con 0x7fdf641025d0 2024-09-21T06:06:19.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:19.308+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdf6410b340 con 0x7fdf500b6560 2024-09-21T06:06:19.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi110 2024-09-21T06:06:19.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-21T06:06:19.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:06:19.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:06:19.817 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:19.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:19.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:19.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:06:19.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:06:19.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi110 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:06:20.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:06:20.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi110 2024-09-21T06:06:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi110 to 8544M 2024-09-21T06:06:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:06:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:06:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:20.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:20.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:06:20.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:06:20.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] boot 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:20.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:20 smithi031 ceph-mon[31237]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:06:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:06:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:06:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] boot 2024-09-21T06:06:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:06:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:21.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:20 smithi110 ceph-mon[34550]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: osd.5 [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] boot 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:06:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:20 smithi107 ceph-mon[34380]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:06:22.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:22.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:06:22.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3945200769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f9f50053-37df-4668-b5a6-c186af35a2b8"} : dispatch 2024-09-21T06:06:22.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3945200769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9f50053-37df-4668-b5a6-c186af35a2b8"}]': finished 2024-09-21T06:06:22.890 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:06:22.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:22.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:22 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1526140414' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:06:23.037 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:23.038 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:06:23.038 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3945200769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f9f50053-37df-4668-b5a6-c186af35a2b8"} : dispatch 2024-09-21T06:06:23.038 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3945200769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9f50053-37df-4668-b5a6-c186af35a2b8"}]': finished 2024-09-21T06:06:23.038 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:06:23.038 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:23.038 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:22 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1526140414' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:06:23.044 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:23.044 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:06:23.044 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3945200769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f9f50053-37df-4668-b5a6-c186af35a2b8"} : dispatch 2024-09-21T06:06:23.044 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3945200769' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f9f50053-37df-4668-b5a6-c186af35a2b8"}]': finished 2024-09-21T06:06:23.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:06:23.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:23.045 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:22 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1526140414' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:06:24.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:24 smithi031 ceph-mon[31237]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:25.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:24 smithi110 ceph-mon[34550]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:24 smithi107 ceph-mon[34380]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:26.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:26 smithi031 ceph-mon[31237]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:06:27.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:26 smithi110 ceph-mon[34550]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:06:27.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:26 smithi107 ceph-mon[34380]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:06:28.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:28 smithi031 ceph-mon[31237]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T06:06:29.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:28 smithi110 ceph-mon[34550]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T06:06:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:28 smithi107 ceph-mon[34380]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-21T06:06:30.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:30 smithi031 ceph-mon[31237]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:06:30.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:06:30.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:30.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:30 smithi031 ceph-mon[31237]: Deploying daemon osd.6 on smithi031 2024-09-21T06:06:31.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:30 smithi110 ceph-mon[34550]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:06:31.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:06:31.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:31.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:30 smithi110 ceph-mon[34550]: Deploying daemon osd.6 on smithi031 2024-09-21T06:06:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:30 smithi107 ceph-mon[34380]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:06:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:06:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:30 smithi107 ceph-mon[34380]: Deploying daemon osd.6 on smithi031 2024-09-21T06:06:32.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:32 smithi031 ceph-mon[31237]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:06:33.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:32 smithi110 ceph-mon[34550]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:06:33.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:32 smithi107 ceph-mon[34380]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:06:34.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:34 smithi031 ceph-mon[31237]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T06:06:35.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:34 smithi110 ceph-mon[34550]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T06:06:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:34 smithi107 ceph-mon[34380]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T06:06:36.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:36 smithi031 ceph-mon[31237]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:06:36.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:36 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:36.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:36 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:36.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:36 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:37.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:36 smithi110 ceph-mon[34550]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:06:37.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:36 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:37.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:36 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:37.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:36 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:36 smithi107 ceph-mon[34380]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:06:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:36 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:36 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:36 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:38.860 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:38 smithi031 ceph-mon[31237]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:39.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:38 smithi110 ceph-mon[34550]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:39.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:38 smithi107 ceph-mon[34380]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:40.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.263 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:40.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:40.264 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.305+0000 7fdf5affd640 1 -- 172.21.15.31:0/1006774031 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf6410b340 con 0x7fdf500b6560 2024-09-21T06:06:40.307 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf500b6560 msgr2=0x7fdf500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf500b6560 0x7fdf500b8a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf64079fa0 tx=0x7fdf54005f90 comp rx=0 tx=0).stop 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 msgr2=0x7fdf6419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf6419c660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005940 tx=0x7fdf600040c0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 shutdown_connections 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf64110180 0x7fdf641a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf64102f20 0x7fdf6419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf500b6560 0x7fdf500b8a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 --2- 172.21.15.31:0/1006774031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf641025d0 0x7fdf6419c660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 >> 172.21.15.31:0/1006774031 conn(0x7fdf64079730 msgr2=0x7fdf64111600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 shutdown_connections 2024-09-21T06:06:40.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:06:40.307+0000 7fdf6c14b640 1 -- 172.21.15.31:0/1006774031 wait complete. 2024-09-21T06:06:40.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:40.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:40.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:40.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:40.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:40.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:41.078 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T06:06:41.081 INFO:tasks.cephadm:Deploying osd.7 on smithi107 with /dev/vg_nvme/lv_2... 2024-09-21T06:06:41.081 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:06:41.325 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:41 smithi031 ceph-mon[31237]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:41.325 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:41 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:41.325 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:41 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:41.325 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:41 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:41.334 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:06:41.455 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:41 smithi107 ceph-mon[34380]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:41.455 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:41 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:41.455 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:41 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:41.455 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:41 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:41 smithi110 ceph-mon[34550]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:41 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:41 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:41 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:42.885 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:06:42 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:06:42.796+0000 7fce9fcfd640 -1 osd.6 0 log_to_monitors true 2024-09-21T06:06:43.340 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:43 smithi107 ceph-mon[34380]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:43.341 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:43 smithi107 ceph-mon[34380]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:06:43.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:43 smithi031 ceph-mon[31237]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:43.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:43 smithi031 ceph-mon[31237]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:06:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:43 smithi110 ceph-mon[34550]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:43 smithi110 ceph-mon[34550]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:06:43.733 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T06:06:43.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-09-21T06:06:44.018 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:06:44.385 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:06:44.128+0000 7fce9bc7f640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T06:06:44.386 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:06:44.140+0000 7fce97a7d640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:44.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:44.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:44 smithi031 ceph-mon[31237]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.442 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:44.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:44.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.443 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:44 smithi107 ceph-mon[34380]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:06:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 -- 172.21.15.107:0/3802455591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74103b80 msgr2=0x7f1f74103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3802455591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74103b80 0x7f1f74103f90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009900 tx=0x7f1f6402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 -- 172.21.15.107:0/3802455591 shutdown_connections 2024-09-21T06:06:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3802455591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74109e10 0x7f1f74110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3802455591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f741094c0 0x7f1f741098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3802455591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74103b80 0x7f1f74103f90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 -- 172.21.15.107:0/3802455591 >> 172.21.15.107:0/3802455591 conn(0x7f1f740ff5d0 msgr2=0x7f1f74101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 -- 172.21.15.107:0/3802455591 shutdown_connections 2024-09-21T06:06:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.440+0000 7f1f7926a640 1 -- 172.21.15.107:0/3802455591 wait complete. 2024-09-21T06:06:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.441+0000 7f1f7926a640 1 Processor -- start 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.441+0000 7f1f7926a640 1 -- start start 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74103b80 0x7f1f7419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f741094c0 0x7f1f7419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 0x7f1f7419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74113ad0 con 0x7f1f74103b80 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74113950 con 0x7f1f74109e10 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f74113c50 con 0x7f1f741094c0 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 0x7f1f7419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 0x7f1f7419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42652/0 (socket says 172.21.15.107:42652) 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 -- 172.21.15.107:0/3036127985 learned_addr learned my addr 172.21.15.107:0/3036127985 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T06:06:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f72ffd640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74103b80 0x7f1f7419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f727fc640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f741094c0 0x7f1f7419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 -- 172.21.15.107:0/3036127985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f741094c0 msgr2=0x7f1f7419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f741094c0 0x7f1f7419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 -- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74103b80 msgr2=0x7f1f7419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74103b80 0x7f1f7419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 -- 172.21.15.107:0/3036127985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f741a0330 con 0x7f1f74109e10 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f72ffd640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74103b80 0x7f1f7419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f737fe640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 0x7f1f7419fcc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00eb10 tx=0x7f1f6c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00ce20 con 0x7f1f74109e10 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f6c005020 con 0x7f1f74109e10 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.442+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f741a0620 con 0x7f1f74109e10 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.443+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c010690 con 0x7f1f74109e10 2024-09-21T06:06:44.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.443+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f741a6e40 con 0x7f1f74109e10 2024-09-21T06:06:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.444+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f74104e00 con 0x7f1f74109e10 2024-09-21T06:06:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.445+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1f6c005510 con 0x7f1f74109e10 2024-09-21T06:06:44.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.446+0000 7f1f5bfff640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f5c0b6590 0x7f1f5c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.446+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4900+0+0 (secure 0 0 0) 0x7f1f6c0e6d90 con 0x7f1f74109e10 2024-09-21T06:06:44.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.446+0000 7f1f72ffd640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f5c0b6590 0x7f1f5c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.447+0000 7f1f72ffd640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f5c0b6590 0x7f1f5c0b8aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f64031440 tx=0x7f1f64032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.449+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1f6c056710 con 0x7f1f74109e10 2024-09-21T06:06:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:06:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:06:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:06:44.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:44 smithi110 ceph-mon[34550]: from='osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:06:44.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:06:44.636+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1f7410bc10 con 0x7f1f5c0b6590 2024-09-21T06:06:45.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:45.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi031 2024-09-21T06:06:45.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-21T06:06:45.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:06:45.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:06:45.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:06:45.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:45.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi031 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:45.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi031 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi031 to 5697M 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:45.585 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:46.264 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:46 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:06:46.264 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:46 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:06:46.264 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:46 smithi107 ceph-mon[34380]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:46.264 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:46 smithi107 ceph-mon[34380]: osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] boot 2024-09-21T06:06:46.264 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:46 smithi107 ceph-mon[34380]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:06:46.264 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:46.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:46 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:06:46.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:46 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:06:46.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:46 smithi110 ceph-mon[34550]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:46.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:46 smithi110 ceph-mon[34550]: osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] boot 2024-09-21T06:06:46.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:46 smithi110 ceph-mon[34550]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:06:46.449 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:46.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:46 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:06:46.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:46 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:06:46.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:46 smithi031 ceph-mon[31237]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:06:46.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:46 smithi031 ceph-mon[31237]: osd.6 [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] boot 2024-09-21T06:06:46.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:46 smithi031 ceph-mon[31237]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:06:46.603 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:06:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:06:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/3563906144' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"} : dispatch 2024-09-21T06:06:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"} : dispatch 2024-09-21T06:06:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"}]': finished 2024-09-21T06:06:47.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:06:47.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/3563906144' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"} : dispatch 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"} : dispatch 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"}]': finished 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:06:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:06:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:06:47.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:06:47.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/3563906144' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"} : dispatch 2024-09-21T06:06:47.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"} : dispatch 2024-09-21T06:06:47.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26b1487e-7c8b-4705-8418-c1687458e181"}]': finished 2024-09-21T06:06:47.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:06:47.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:06:48.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:48 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/3668849869' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:06:48.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:48 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/3668849869' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:06:48.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:48 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/3668849869' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:06:49.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:49 smithi110 ceph-mon[34550]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:49.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:49 smithi107 ceph-mon[34380]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:49 smithi031 ceph-mon[31237]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:51 smithi110 ceph-mon[34550]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:51 smithi107 ceph-mon[34380]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:51 smithi031 ceph-mon[31237]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:53.509 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:53 smithi107 ceph-mon[34380]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:53.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:53 smithi110 ceph-mon[34550]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:53 smithi031 ceph-mon[31237]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:55.469 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:55 smithi107 ceph-mon[34380]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:55.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:55 smithi110 ceph-mon[34550]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:55 smithi031 ceph-mon[31237]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:56.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:06:56.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:56.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:06:56.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:06:56.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:57.406 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:57 smithi107 ceph-mon[34380]: Deploying daemon osd.7 on smithi107 2024-09-21T06:06:57.406 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:57 smithi107 ceph-mon[34380]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:57 smithi110 ceph-mon[34550]: Deploying daemon osd.7 on smithi107 2024-09-21T06:06:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:57 smithi110 ceph-mon[34550]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:57 smithi031 ceph-mon[31237]: Deploying daemon osd.7 on smithi107 2024-09-21T06:06:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:57 smithi031 ceph-mon[31237]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:59.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:06:59 smithi110 ceph-mon[34550]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:59.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:06:59 smithi107 ceph-mon[34380]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:06:59.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:06:59 smithi031 ceph-mon[31237]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:01 smithi110 ceph-mon[34550]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:01 smithi107 ceph-mon[34380]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:01 smithi031 ceph-mon[31237]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:02 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:02 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:02 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:02 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:02 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:02 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:02.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:02 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:02.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:02 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:02.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:02 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:03.446 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:03 smithi107 ceph-mon[34380]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:03.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:03 smithi110 ceph-mon[34550]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:03 smithi031 ceph-mon[31237]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.431+0000 7f1f5bfff640 1 -- 172.21.15.107:0/3036127985 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1f7410bc10 con 0x7f1f5c0b6590 2024-09-21T06:07:04.434 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 7 on host 'smithi107' 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f5c0b6590 msgr2=0x7f1f5c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f5c0b6590 0x7f1f5c0b8aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f64031440 tx=0x7f1f64032ed0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 msgr2=0x7f1f7419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 0x7f1f7419fcc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00eb10 tx=0x7f1f6c00ef10 comp rx=0 tx=0).stop 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 shutdown_connections 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f74109e10 0x7f1f7419fcc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f741094c0 0x7f1f7419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f5c0b6590 0x7f1f5c0b8aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 --2- 172.21.15.107:0/3036127985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f74103b80 0x7f1f7419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 >> 172.21.15.107:0/3036127985 conn(0x7f1f740ff5d0 msgr2=0x7f1f741019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 shutdown_connections 2024-09-21T06:07:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:04.434+0000 7f1f7926a640 1 -- 172.21.15.107:0/3036127985 wait complete. 2024-09-21T06:07:05.031 DEBUG:teuthology.orchestra.run.smithi107:osd.7> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T06:07:05.033 INFO:tasks.cephadm:Deploying osd.8 on smithi110 with /dev/vg_nvme/lv_2... 2024-09-21T06:07:05.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:07:05.290 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:07:05.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:05 smithi110 ceph-mon[34550]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:05.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.412 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:05 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:05 smithi107 ceph-mon[34380]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:05 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:05 smithi031 ceph-mon[31237]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:05.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:05 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:07.446 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:07 smithi110 ceph-mon[34550]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:07.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:07 smithi107 ceph-mon[34380]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:07.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:07 smithi031 ceph-mon[31237]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:07:07.789 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:07:07 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:07:07.751+0000 7fd6b2f98640 -1 osd.7 0 log_to_monitors true 2024-09-21T06:07:07.802 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-09-21T06:07:08.053 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:08.435 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.436 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:08.436 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.436 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:08.436 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 -- 172.21.15.110:0/2229929229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 msgr2=0x7fc26c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 --2- 172.21.15.110:0/2229929229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c10f520 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc264009ca0 tx=0x7fc26402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 -- 172.21.15.110:0/2229929229 shutdown_connections 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 --2- 172.21.15.110:0/2229929229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c10f520 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 --2- 172.21.15.110:0/2229929229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c1083a0 0x7fc26c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 --2- 172.21.15.110:0/2229929229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c102a60 0x7fc26c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 -- 172.21.15.110:0/2229929229 >> 172.21.15.110:0/2229929229 conn(0x7fc26c079730 msgr2=0x7fc26c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 -- 172.21.15.110:0/2229929229 shutdown_connections 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.525+0000 7fc271d10640 1 -- 172.21.15.110:0/2229929229 wait complete. 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 Processor -- start 2024-09-21T06:07:08.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 -- start start 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c102a60 0x7fc26c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c1083a0 0x7fc26c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1129c0 con 0x7fc26c1083a0 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c112840 con 0x7fc26c102a60 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc271d10640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c112b40 con 0x7fc26c108cf0 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc26bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53682/0 (socket says 172.21.15.110:53682) 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc26b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c102a60 0x7fc26c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc26bfff640 1 -- 172.21.15.110:0/2757049170 learned_addr learned my addr 172.21.15.110:0/2757049170 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.526+0000 7fc262ffd640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c1083a0 0x7fc26c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26bfff640 1 -- 172.21.15.110:0/2757049170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c102a60 msgr2=0x7fc26c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26bfff640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c102a60 0x7fc26c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26bfff640 1 -- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c1083a0 msgr2=0x7fc26c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26bfff640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c1083a0 0x7fc26c19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26bfff640 1 -- 172.21.15.110:0/2757049170 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c1a6580 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc262ffd640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c1083a0 0x7fc26c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26b7fe640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c102a60 0x7fc26c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc26bfff640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c19bdd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc264005cc0 tx=0x7fc26402f470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264003cf0 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2640044f0 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c1a6810 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26402fc30 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.527+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c1a6d50 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.528+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c102e70 con 0x7fc26c108cf0 2024-09-21T06:07:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.530+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc2640046a0 con 0x7fc26c108cf0 2024-09-21T06:07:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.531+0000 7fc2697fa640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc2580b65e0 0x7fc2580b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.531+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5316+0+0 (secure 0 0 0) 0x7fc264107eb0 con 0x7fc26c108cf0 2024-09-21T06:07:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.531+0000 7fc26b7fe640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc2580b65e0 0x7fc2580b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.531+0000 7fc26b7fe640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc2580b65e0 0x7fc2580b8af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc26c06b9e0 tx=0x7fc25c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.534+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2640cf250 con 0x7fc26c108cf0 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:08.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:08.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:08.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:08.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:08.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:08.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:08.719+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc26c10ab40 con 0x7fc2580b65e0 2024-09-21T06:07:09.362 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi107 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:09.363 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi107 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:07:09.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:07:09.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:09.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:09.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi107 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi107 to 5695M 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:10.376 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:10 smithi110 ceph-mon[34550]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:10.376 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:10 smithi110 ceph-mon[34550]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:07:10.376 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:10 smithi110 ceph-mon[34550]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:07:10.376 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:10.376 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:10.376 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:10 smithi110 ceph-mon[34550]: from='osd.7 ' entity='osd.7' 2024-09-21T06:07:10.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:07:10.150+0000 7fd6aef1a640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T06:07:10.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:07:10.237+0000 7fd6a6d10640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:07:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-mon[34380]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-mon[34380]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:07:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-mon[34380]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:07:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:10 smithi107 ceph-mon[34380]: from='osd.7 ' entity='osd.7' 2024-09-21T06:07:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:10 smithi031 ceph-mon[31237]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:10 smithi031 ceph-mon[31237]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:07:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:10 smithi031 ceph-mon[31237]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:07:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:10 smithi031 ceph-mon[31237]: from='osd.7 ' entity='osd.7' 2024-09-21T06:07:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:07:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:07:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] boot 2024-09-21T06:07:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:07:11.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:11.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/1669483920' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"} : dispatch 2024-09-21T06:07:11.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"} : dispatch 2024-09-21T06:07:11.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"}]': finished 2024-09-21T06:07:11.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T06:07:11.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:11 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] boot 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/1669483920' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"} : dispatch 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"} : dispatch 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"}]': finished 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T06:07:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:11 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:11.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:07:11.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: osd.7 [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] boot 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/1669483920' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"} : dispatch 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"} : dispatch 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee40c435-fe2e-4026-bfbc-2dfed559f5d9"}]': finished 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T06:07:11.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:11 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:12.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:12 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/2810911052' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:12.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:12 smithi110 ceph-mon[34550]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T06:07:12.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:12.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:12 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/2810911052' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:12.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:12 smithi107 ceph-mon[34380]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T06:07:12.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:12.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:12 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/2810911052' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:12.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:12 smithi031 ceph-mon[31237]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T06:07:12.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:13.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:13 smithi110 ceph-mon[34550]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:13 smithi107 ceph-mon[34380]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:13.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:13 smithi031 ceph-mon[31237]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:15.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:15 smithi110 ceph-mon[34550]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:15.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:15 smithi107 ceph-mon[34380]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:15.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:15 smithi031 ceph-mon[31237]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:17.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:17 smithi110 ceph-mon[34550]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:07:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:17 smithi107 ceph-mon[34380]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:07:17.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:17 smithi031 ceph-mon[31237]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:07:19.383 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:19 smithi110 ceph-mon[34550]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-09-21T06:07:19.383 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:07:19.383 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:19 smithi107 ceph-mon[34380]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-09-21T06:07:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:07:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:19 smithi031 ceph-mon[31237]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-09-21T06:07:19.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:07:19.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:20.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:20 smithi110 ceph-mon[34550]: Deploying daemon osd.8 on smithi110 2024-09-21T06:07:20.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:20 smithi107 ceph-mon[34380]: Deploying daemon osd.8 on smithi110 2024-09-21T06:07:20.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:20 smithi031 ceph-mon[31237]: Deploying daemon osd.8 on smithi110 2024-09-21T06:07:21.375 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:21 smithi110 ceph-mon[34550]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-21T06:07:21.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:21 smithi107 ceph-mon[34380]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-21T06:07:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:21 smithi031 ceph-mon[31237]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-09-21T06:07:23.462 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:23 smithi110 ceph-mon[34550]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:07:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:23 smithi031 ceph-mon[31237]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:07:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:23 smithi107 ceph-mon[34380]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:07:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:25 smithi031 ceph-mon[31237]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T06:07:25.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:25.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:25.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:25.737 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:25 smithi110 ceph-mon[34550]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T06:07:25.737 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:25.737 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:25.737 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:25 smithi107 ceph-mon[34380]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-21T06:07:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:27.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:27 smithi031 ceph-mon[31237]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:07:27.664 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:27 smithi110 ceph-mon[34550]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:07:27.705 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 8 on host 'smithi110' 2024-09-21T06:07:27.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.703+0000 7fc2697fa640 1 -- 172.21.15.110:0/2757049170 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc26c10ab40 con 0x7fc2580b65e0 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc2580b65e0 msgr2=0x7fc2580b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc2580b65e0 0x7fc2580b8af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc26c06b9e0 tx=0x7fc25c009110 comp rx=0 tx=0).stop 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 msgr2=0x7fc26c19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c19bdd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc264005cc0 tx=0x7fc26402f470 comp rx=0 tx=0).stop 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 shutdown_connections 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc26c108cf0 0x7fc26c19bdd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc2580b65e0 0x7fc2580b8af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.707 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc26c1083a0 0x7fc26c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 --2- 172.21.15.110:0/2757049170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc26c102a60 0x7fc26c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 >> 172.21.15.110:0/2757049170 conn(0x7fc26c079730 msgr2=0x7fc26c1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 shutdown_connections 2024-09-21T06:07:27.708 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:07:27.705+0000 7fc271d10640 1 -- 172.21.15.110:0/2757049170 wait complete. 2024-09-21T06:07:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:27 smithi107 ceph-mon[34380]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:07:28.378 DEBUG:teuthology.orchestra.run.smithi110:osd.8> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T06:07:28.380 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2024-09-21T06:07:28.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:07:28.656 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:28.976 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:28 smithi110 ceph-mon[34550]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:28.977 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:28.977 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:28.977 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:28.977 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:28 smithi031 ceph-mon[31237]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:29.011 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.012 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:28 smithi107 ceph-mon[34380]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:30.721 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:30 smithi031 ceph-mon[31237]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:30.974 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:30 smithi110 ceph-mon[34550]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:30 smithi107 ceph-mon[34380]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:31.278 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:07:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:07:31.007+0000 7f1b37675640 -1 osd.8 0 log_to_monitors true 2024-09-21T06:07:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:07:31.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2024-09-21T06:07:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:31.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:31 smithi031 ceph-mon[31237]: from='osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:07:31.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:31 smithi031 ceph-mon[31237]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:07:32.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:31 smithi110 ceph-mon[34550]: from='osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:07:32.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:31 smithi110 ceph-mon[34550]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:07:32.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:31 smithi107 ceph-mon[34380]: from='osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:07:32.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:31 smithi107 ceph-mon[34380]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:07:32.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2815202025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4102a60 msgr2=0x7fc0e4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2815202025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4102a60 0x7fc0e4102e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009900 tx=0x7fc0cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:07:32.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2815202025 shutdown_connections 2024-09-21T06:07:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2815202025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4108cf0 0x7fc0e410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2815202025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e41083a0 0x7fc0e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2815202025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4102a60 0x7fc0e4102e70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2815202025 >> 172.21.15.31:0/2815202025 conn(0x7fc0e4079730 msgr2=0x7fc0e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2815202025 shutdown_connections 2024-09-21T06:07:32.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.439+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2815202025 wait complete. 2024-09-21T06:07:32.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 Processor -- start 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 -- start start 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4102a60 0x7fc0e419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e41083a0 0x7fc0e419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 0x7fc0e419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4112960 con 0x7fc0e4108cf0 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41127e0 con 0x7fc0e41083a0 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0eacd4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4112ae0 con 0x7fc0e4102a60 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0e9271640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 0x7fc0e419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0e9271640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 0x7fc0e419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47788/0 (socket says 172.21.15.31:47788) 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.440+0000 7fc0e9271640 1 -- 172.21.15.31:0/2360927222 learned_addr learned my addr 172.21.15.31:0/2360927222 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e3fff640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e41083a0 0x7fc0e419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e8a70640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4102a60 0x7fc0e419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e9271640 1 -- 172.21.15.31:0/2360927222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4102a60 msgr2=0x7fc0e419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e9271640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4102a60 0x7fc0e419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e9271640 1 -- 172.21.15.31:0/2360927222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e41083a0 msgr2=0x7fc0e419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e9271640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e41083a0 0x7fc0e419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e9271640 1 -- 172.21.15.31:0/2360927222 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e41a0490 con 0x7fc0e4108cf0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e8a70640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4102a60 0x7fc0e419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e9271640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 0x7fc0e419fd60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00c970 tx=0x7fc0dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc007bb0 con 0x7fc0e4108cf0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e41a0780 con 0x7fc0e4108cf0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc007d50 con 0x7fc0e4108cf0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.441+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc004e60 con 0x7fc0e4108cf0 2024-09-21T06:07:32.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.442+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e41a6ee0 con 0x7fc0e4108cf0 2024-09-21T06:07:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.442+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e4102e70 con 0x7fc0e4108cf0 2024-09-21T06:07:32.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.444+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc0dc005000 con 0x7fc0e4108cf0 2024-09-21T06:07:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.445+0000 7fc0e1ffb640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc0d00b65e0 0x7fc0d00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.445+0000 7fc0e8a70640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc0d00b65e0 0x7fc0d00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.445+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5748+0+0 (secure 0 0 0) 0x7fc0dc0e69c0 con 0x7fc0e4108cf0 2024-09-21T06:07:32.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.446+0000 7fc0e8a70640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc0d00b65e0 0x7fc0d00b8af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc031440 tx=0x7fc0cc005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:32.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.449+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0dc014030 con 0x7fc0e4108cf0 2024-09-21T06:07:32.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:32.646+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc0e406aeb0 con 0x7fc0d00b65e0 2024-09-21T06:07:33.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:33.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:07:33.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi110 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:33.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:33.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:33.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:07:33.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:07:33.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi110 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:33.041 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.043 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:33.043 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:07:33.043 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:07:33.043 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi110 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi110 to 5696M 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:33.044 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:33.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:33.717 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:07:33.364+0000 7f1b335f7640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T06:07:33.717 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:07:33.375+0000 7f1b2b3ed640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:33 smithi110 ceph-mon[34550]: from='osd.8 ' entity='osd.8' 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:33 smithi107 ceph-mon[34380]: from='osd.8 ' entity='osd.8' 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:07:34.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:33 smithi031 ceph-mon[31237]: from='osd.8 ' entity='osd.8' 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] boot 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:07:34.909 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:35.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:07:35.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:07:35.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:35.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:35.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] boot 2024-09-21T06:07:35.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:07:35.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:07:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:07:35.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:07:35.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:35.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: osd.8 [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] boot 2024-09-21T06:07:35.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:07:35.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:07:36.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:36 smithi110 ceph-mon[34550]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:07:36.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3352400231' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d5a01fe-27f8-4e91-86b1-031461e41633"} : dispatch 2024-09-21T06:07:36.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3352400231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d5a01fe-27f8-4e91-86b1-031461e41633"}]': finished 2024-09-21T06:07:36.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:36 smithi110 ceph-mon[34550]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T06:07:36.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:36 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:36.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3801750044' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:36.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:36 smithi107 ceph-mon[34380]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:07:36.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3352400231' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d5a01fe-27f8-4e91-86b1-031461e41633"} : dispatch 2024-09-21T06:07:36.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3352400231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d5a01fe-27f8-4e91-86b1-031461e41633"}]': finished 2024-09-21T06:07:36.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:36 smithi107 ceph-mon[34380]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T06:07:36.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:36 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:36.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3801750044' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:36.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:36 smithi031 ceph-mon[31237]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:07:36.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3352400231' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d5a01fe-27f8-4e91-86b1-031461e41633"} : dispatch 2024-09-21T06:07:36.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3352400231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d5a01fe-27f8-4e91-86b1-031461e41633"}]': finished 2024-09-21T06:07:36.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:36 smithi031 ceph-mon[31237]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T06:07:36.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:36 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:36.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3801750044' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:07:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:37 smithi031 ceph-mon[31237]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:37 smithi110 ceph-mon[34550]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:37 smithi107 ceph-mon[34380]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:39 smithi031 ceph-mon[31237]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:39 smithi110 ceph-mon[34550]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:39 smithi107 ceph-mon[34380]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:41.574 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:41 smithi031 ceph-mon[31237]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:41 smithi110 ceph-mon[34550]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:41 smithi107 ceph-mon[34380]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:43.383 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:43 smithi031 ceph-mon[31237]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T06:07:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:43 smithi110 ceph-mon[34550]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T06:07:43.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:43 smithi107 ceph-mon[34380]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T06:07:45.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:45 smithi031 ceph-mon[31237]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T06:07:45.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:07:45.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:45.580 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:45 smithi110 ceph-mon[34550]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T06:07:45.580 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:07:45.581 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:45.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:45 smithi107 ceph-mon[34380]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T06:07:45.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:07:45.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:46.596 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:46 smithi110 ceph-mon[34550]: Deploying daemon osd.9 on smithi031 2024-09-21T06:07:46.627 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:46 smithi031 ceph-mon[31237]: Deploying daemon osd.9 on smithi031 2024-09-21T06:07:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:46 smithi107 ceph-mon[34380]: Deploying daemon osd.9 on smithi031 2024-09-21T06:07:47.458 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:47 smithi031 ceph-mon[31237]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:07:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:47 smithi110 ceph-mon[34550]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:07:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:47 smithi107 ceph-mon[34380]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:07:49.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:49 smithi031 ceph-mon[31237]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:49 smithi110 ceph-mon[34550]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:49 smithi107 ceph-mon[34380]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:51 smithi031 ceph-mon[31237]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:51.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:51.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:51.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:51 smithi110 ceph-mon[34550]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:51 smithi107 ceph-mon[34380]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:53 smithi031 ceph-mon[31237]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:53 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:53 smithi110 ceph-mon[34550]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:53.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:53 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:53 smithi107 ceph-mon[34380]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:07:53.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:53.791 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:53 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.023+0000 7fc0e1ffb640 1 -- 172.21.15.31:0/2360927222 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc0e406aeb0 con 0x7fc0d00b65e0 2024-09-21T06:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc0d00b65e0 msgr2=0x7fc0d00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc0d00b65e0 0x7fc0d00b8af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc031440 tx=0x7fc0cc005d50 comp rx=0 tx=0).stop 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 msgr2=0x7fc0e419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 0x7fc0e419fd60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00c970 tx=0x7fc0dc00ce30 comp rx=0 tx=0).stop 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 shutdown_connections 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc0d00b65e0 0x7fc0d00b8af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0e4108cf0 0x7fc0e419fd60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e41083a0 0x7fc0e419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 --2- 172.21.15.31:0/2360927222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0e4102a60 0x7fc0e419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 >> 172.21.15.31:0/2360927222 conn(0x7fc0e4079730 msgr2=0x7fc0e4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 shutdown_connections 2024-09-21T06:07:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:07:54.026+0000 7fc0eacd4640 1 -- 172.21.15.31:0/2360927222 wait complete. 2024-09-21T06:07:54.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.665 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T06:07:54.667 INFO:tasks.cephadm:Deploying osd.10 on smithi107 with /dev/vg_nvme/lv_1... 2024-09-21T06:07:54.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:07:54.694 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:54.695 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:54.695 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.695 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:54.695 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.695 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:54.922 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:07:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:55 smithi031 ceph-mon[31237]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:55.692 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:55 smithi107 ceph-mon[34380]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:55 smithi110 ceph-mon[34550]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:56.734 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:07:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:07:56.474+0000 7f5e07f27640 -1 osd.9 0 log_to_monitors true 2024-09-21T06:07:57.497 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-21T06:07:57.524 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-09-21T06:07:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:57 smithi031 ceph-mon[31237]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:57 smithi031 ceph-mon[31237]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:07:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:57 smithi110 ceph-mon[34550]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:57 smithi110 ceph-mon[34550]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:07:57.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:57 smithi107 ceph-mon[34380]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:57.786 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:57 smithi107 ceph-mon[34380]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:07:57.824 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:07:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 -- 172.21.15.107:0/751692591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 msgr2=0x7fb284110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 --2- 172.21.15.107:0/751692591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb284110300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009ca0 tx=0x7fb27c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 -- 172.21.15.107:0/751692591 shutdown_connections 2024-09-21T06:07:58.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 --2- 172.21.15.107:0/751692591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb284110300 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 --2- 172.21.15.107:0/751692591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 0x7fb28410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 --2- 172.21.15.107:0/751692591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb284104840 0x7fb284104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 -- 172.21.15.107:0/751692591 >> 172.21.15.107:0/751692591 conn(0x7fb284079730 msgr2=0x7fb284079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.298+0000 7fb289eaf640 1 -- 172.21.15.107:0/751692591 shutdown_connections 2024-09-21T06:07:58.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.299+0000 7fb289eaf640 1 -- 172.21.15.107:0/751692591 wait complete. 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.299+0000 7fb289eaf640 1 Processor -- start 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 -- start start 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb284104840 0x7fb284102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 0x7fb284102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb2841031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb284113a90 con 0x7fb284105270 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb284113910 con 0x7fb28410ddf0 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb289eaf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb284113c10 con 0x7fb284104840 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb2841031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:58.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb2841031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45870/0 (socket says 172.21.15.107:45870) 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 -- 172.21.15.107:0/1939099760 learned_addr learned my addr 172.21.15.107:0/1939099760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb2837fe640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb284104840 0x7fb284102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb282ffd640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 0x7fb284102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 -- 172.21.15.107:0/1939099760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb284104840 msgr2=0x7fb284102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb284104840 0x7fb284102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 -- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 msgr2=0x7fb284102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 0x7fb284102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb283fff640 1 -- 172.21.15.107:0/1939099760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2841a8720 con 0x7fb28410ddf0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.300+0000 7fb282ffd640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 0x7fb284102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.301+0000 7fb283fff640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb2841031d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0098d0 tx=0x7fb27c0317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.301+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c003b70 con 0x7fb28410ddf0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.301+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27c003d10 con 0x7fb28410ddf0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.301+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2841a89b0 con 0x7fb28410ddf0 2024-09-21T06:07:58.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.301+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2841a8ef0 con 0x7fb28410ddf0 2024-09-21T06:07:58.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.301+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c02fa20 con 0x7fb28410ddf0 2024-09-21T06:07:58.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.302+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb284104c50 con 0x7fb28410ddf0 2024-09-21T06:07:58.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.304+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb27c02fbc0 con 0x7fb28410ddf0 2024-09-21T06:07:58.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.305+0000 7fb280ff9640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb26c0b6560 0x7fb26c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:58.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.305+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6180+0+0 (secure 0 0 0) 0x7fb27c107fc0 con 0x7fb28410ddf0 2024-09-21T06:07:58.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.306+0000 7fb2837fe640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb26c0b6560 0x7fb26c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:58.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.306+0000 7fb2837fe640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb26c0b6560 0x7fb26c0b8a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2841041a0 tx=0x7fb278005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:58.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.308+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb27c0cf0a0 con 0x7fb28410ddf0 2024-09-21T06:07:58.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:07:58.513+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb284100c70 con 0x7fb26c0b6560 2024-09-21T06:07:58.635 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:07:58.357+0000 7f5e03ea9640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T06:07:58.636 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:07:58.372+0000 7f5dffca7640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:07:58.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:07:58.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:07:58.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:07:58.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:58.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:58.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.638 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:58 smithi031 ceph-mon[31237]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:07:58.774 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:07:58.774 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:07:58.774 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:07:58.774 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.775 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:58 smithi107 ceph-mon[34380]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:07:58.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:58 smithi110 ceph-mon[34550]: from='osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi031 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:07:59 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi031 2024-09-21T06:07:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-21T06:07:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:07:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:59.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:59.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:59.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:59.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:07:59 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi031 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi031 to 4273M 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:07:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:07:59 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:00.378 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:00 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:08:00.378 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:00 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:08:00.378 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:00 smithi107 ceph-mon[34380]: from='client.24536 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:00.378 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:00 smithi107 ceph-mon[34380]: osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] boot 2024-09-21T06:08:00.378 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:00 smithi107 ceph-mon[34380]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T06:08:00.378 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:08:00.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:00 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:08:00.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:00 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:08:00.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:00 smithi031 ceph-mon[31237]: from='client.24536 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:00.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:00 smithi031 ceph-mon[31237]: osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] boot 2024-09-21T06:08:00.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:00 smithi031 ceph-mon[31237]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T06:08:00.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:08:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:00 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:08:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:00 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:08:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:00 smithi110 ceph-mon[34550]: from='client.24536 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:00 smithi110 ceph-mon[34550]: osd.9 [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] boot 2024-09-21T06:08:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:00 smithi110 ceph-mon[34550]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T06:08:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:08:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:08:01.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:08:01.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/3707095678' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"} : dispatch 2024-09-21T06:08:01.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"} : dispatch 2024-09-21T06:08:01.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"}]': finished 2024-09-21T06:08:01.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T06:08:01.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:01 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/3707095678' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"} : dispatch 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"} : dispatch 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"}]': finished 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T06:08:01.637 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:01 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/3707095678' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"} : dispatch 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"} : dispatch 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c49045f-5d86-4a85-9c2b-22e12fb42351"}]': finished 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T06:08:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:01 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:02.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:02 smithi110 ceph-mon[34550]: from='client.? 172.21.15.107:0/2059954664' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:02.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:02 smithi107 ceph-mon[34380]: from='client.? 172.21.15.107:0/2059954664' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:02.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:02 smithi031 ceph-mon[31237]: from='client.? 172.21.15.107:0/2059954664' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:03 smithi110 ceph-mon[34550]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:03 smithi107 ceph-mon[34380]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:03 smithi031 ceph-mon[31237]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:05.731 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:05 smithi107 ceph-mon[34380]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:05 smithi110 ceph-mon[34550]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:05 smithi031 ceph-mon[31237]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:07.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:07 smithi107 ceph-mon[34380]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:07 smithi110 ceph-mon[34550]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:07 smithi031 ceph-mon[31237]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:09.703 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:09 smithi107 ceph-mon[34380]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:09.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:08:09.704 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:09 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:09 smithi110 ceph-mon[34550]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:08:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:09 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:09.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:09 smithi031 ceph-mon[31237]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:09.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:08:09.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:09 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:10.710 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:10 smithi107 ceph-mon[34380]: Deploying daemon osd.10 on smithi107 2024-09-21T06:08:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:10 smithi110 ceph-mon[34550]: Deploying daemon osd.10 on smithi107 2024-09-21T06:08:10.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:10 smithi031 ceph-mon[31237]: Deploying daemon osd.10 on smithi107 2024-09-21T06:08:11.670 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:11 smithi107 ceph-mon[34380]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:11 smithi110 ceph-mon[34550]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:11 smithi031 ceph-mon[31237]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:13.620 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:13 smithi107 ceph-mon[34380]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:13 smithi110 ceph-mon[34550]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:13 smithi031 ceph-mon[31237]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:15.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:15 smithi031 ceph-mon[31237]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:15.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:15.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:15.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:15.690 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:15 smithi107 ceph-mon[34380]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:15.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:15.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:15.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:15 smithi110 ceph-mon[34550]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:15.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:17 smithi110 ceph-mon[34550]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:17 smithi107 ceph-mon[34380]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:17.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:17 smithi031 ceph-mon[31237]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:18.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.073+0000 7fb280ff9640 1 -- 172.21.15.107:0/1939099760 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb284100c70 con 0x7fb26c0b6560 2024-09-21T06:08:18.075 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 10 on host 'smithi107' 2024-09-21T06:08:18.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb26c0b6560 msgr2=0x7fb26c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb26c0b6560 0x7fb26c0b8a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2841041a0 tx=0x7fb278005f90 comp rx=0 tx=0).stop 2024-09-21T06:08:18.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 msgr2=0x7fb2841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:18.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb2841031d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0098d0 tx=0x7fb27c0317b0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 shutdown_connections 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb28410ddf0 0x7fb2841031d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb26c0b6560 0x7fb26c0b8a70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb284105270 0x7fb284102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 --2- 172.21.15.107:0/1939099760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb284104840 0x7fb284102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.075+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 >> 172.21.15.107:0/1939099760 conn(0x7fb284079730 msgr2=0x7fb2840ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.076+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 shutdown_connections 2024-09-21T06:08:18.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-21T06:08:18.076+0000 7fb289eaf640 1 -- 172.21.15.107:0/1939099760 wait complete. 2024-09-21T06:08:18.624 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.624 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.624 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:18.624 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:18.624 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.625 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:18.625 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:18.625 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.625 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:18 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.657 DEBUG:teuthology.orchestra.run.smithi107:osd.10> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T06:08:18.660 INFO:tasks.cephadm:Deploying osd.11 on smithi110 with /dev/vg_nvme/lv_1... 2024-09-21T06:08:18.660 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:08:18.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:18.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:18.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:18 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:18.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:18.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:18.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:18.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:18 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:18.915 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:08:20.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:20 smithi107 ceph-mon[34380]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:20.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:20 smithi107 ceph-mon[34380]: from='osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:08:20.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:20 smithi107 ceph-mon[34380]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:08:20.790 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:08:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:08:20.530+0000 7fe71a4b2640 -1 osd.10 0 log_to_monitors true 2024-09-21T06:08:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:20 smithi110 ceph-mon[34550]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:20 smithi110 ceph-mon[34550]: from='osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:08:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:20 smithi110 ceph-mon[34550]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:08:21.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:20 smithi031 ceph-mon[31237]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:21.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:20 smithi031 ceph-mon[31237]: from='osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:08:21.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:20 smithi031 ceph-mon[31237]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:08:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-21T06:08:21.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-09-21T06:08:21.678 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi107 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:21.915 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:08:21.916 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:21 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:08:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:08:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:08:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:08:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi107 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:08:22.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:21 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:22.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.124+0000 7f22f8f9a640 1 -- 172.21.15.110:0/1428899326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f41021f0 msgr2=0x7f22f4102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.124+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/1428899326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f41021f0 0x7f22f4102600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009900 tx=0x7f22e402a410 comp rx=0 tx=0).stop 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.124+0000 7f22f8f9a640 1 -- 172.21.15.110:0/1428899326 shutdown_connections 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.124+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/1428899326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 0x7f22f410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.124+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/1428899326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f4107b30 0x7f22f4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.125+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/1428899326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f41021f0 0x7f22f4102600 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.125+0000 7f22f8f9a640 1 -- 172.21.15.110:0/1428899326 >> 172.21.15.110:0/1428899326 conn(0x7f22f4077c60 msgr2=0x7f22f40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.125+0000 7f22f8f9a640 1 -- 172.21.15.110:0/1428899326 shutdown_connections 2024-09-21T06:08:22.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.125+0000 7f22f8f9a640 1 -- 172.21.15.110:0/1428899326 wait complete. 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 Processor -- start 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 -- start start 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f41021f0 0x7f22f419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f4107b30 0x7f22f419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 0x7f22f419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41118c0 con 0x7f22f41021f0 2024-09-21T06:08:22.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4111740 con 0x7f22f4108480 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f8f9a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22f4111a40 con 0x7f22f4107b30 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f4107b30 0x7f22f419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 0x7f22f419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22ebfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f4107b30 0x7f22f419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35820/0 (socket says 172.21.15.110:35820) 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22ebfff640 1 -- 172.21.15.110:0/2303957656 learned_addr learned my addr 172.21.15.110:0/2303957656 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f259c640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f41021f0 0x7f22f419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f2d9d640 1 -- 172.21.15.110:0/2303957656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f4107b30 msgr2=0x7f22f419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.126+0000 7f22f2d9d640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f4107b30 0x7f22f419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22f2d9d640 1 -- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f41021f0 msgr2=0x7f22f419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22f2d9d640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f41021f0 0x7f22f419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22f2d9d640 1 -- 172.21.15.110:0/2303957656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f41a0390 con 0x7f22f4108480 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22f2d9d640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 0x7f22f419fd20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005fd0 tx=0x7f22ec00c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec004d20 con 0x7f22f4108480 2024-09-21T06:08:22.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f41a0680 con 0x7f22f4108480 2024-09-21T06:08:22.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22ec00ba20 con 0x7f22f4108480 2024-09-21T06:08:22.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec005660 con 0x7f22f4108480 2024-09-21T06:08:22.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.127+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f4197fc0 con 0x7f22f4108480 2024-09-21T06:08:22.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.128+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f4103470 con 0x7f22f4108480 2024-09-21T06:08:22.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.130+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f22ec005800 con 0x7f22f4108480 2024-09-21T06:08:22.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.130+0000 7f22eb7fe640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f22e00b6590 0x7f22e00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.130+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6629+0+0 (secure 0 0 0) 0x7f22ec0e7600 con 0x7f22f4108480 2024-09-21T06:08:22.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.131+0000 7f22f259c640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f22e00b6590 0x7f22e00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.131+0000 7f22f259c640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f22e00b6590 0x7f22e00b8aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22e4031440 tx=0x7f22e4009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:08:22.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi107 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:22.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:22.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:22.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-21T06:08:22.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:08:22.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:21 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:22.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.137+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f22ec0ae480 con 0x7f22f4108480 2024-09-21T06:08:22.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:22.329+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f22f4063800 con 0x7f22e00b6590 2024-09-21T06:08:22.983 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-21T06:08:22.983 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:22.983 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:22.983 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:22.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:22.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:22.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:22.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='osd.10 ' entity='osd.10' 2024-09-21T06:08:22.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:08:22.984 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:23.039 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:08:22.584+0000 7fe716434640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T06:08:23.039 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:08:22.737+0000 7fe70e22a640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:08:23.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='osd.10 ' entity='osd.10' 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:08:23.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:23.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi107 to 4271M 2024-09-21T06:08:23.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='osd.10 ' entity='osd.10' 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:08:23.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:23.729 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:23 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:08:23.729 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:23 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:08:23.729 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:23 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:23 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:08:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:23 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:08:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:23 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:24.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:23 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:08:24.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:23 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:08:24.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:23 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:24.999 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:24 smithi110 ceph-mon[34550]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:24.999 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:24.999 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:24 smithi110 ceph-mon[34550]: osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] boot 2024-09-21T06:08:24.999 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:24 smithi110 ceph-mon[34550]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:08:24.999 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:24 smithi107 ceph-mon[34380]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:24 smithi107 ceph-mon[34380]: osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] boot 2024-09-21T06:08:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:24 smithi107 ceph-mon[34380]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:08:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:25.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:24 smithi031 ceph-mon[31237]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:08:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:24 smithi031 ceph-mon[31237]: osd.10 [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] boot 2024-09-21T06:08:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:24 smithi031 ceph-mon[31237]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:08:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:08:25.954 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:08:25.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/3070957340' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"} : dispatch 2024-09-21T06:08:25.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"} : dispatch 2024-09-21T06:08:25.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"}]': finished 2024-09-21T06:08:25.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T06:08:25.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:25.955 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:25 smithi107 ceph-mon[34380]: from='client.? 172.21.15.110:0/1285032464' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/3070957340' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"} : dispatch 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"} : dispatch 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"}]': finished 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:26.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:25 smithi110 ceph-mon[34550]: from='client.? 172.21.15.110:0/1285032464' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/3070957340' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"} : dispatch 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"} : dispatch 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3922d4ae-a015-4a0e-92e8-2eb5daa7afe4"}]': finished 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:25 smithi031 ceph-mon[31237]: from='client.? 172.21.15.110:0/1285032464' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:08:26.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:26 smithi107 ceph-mon[34380]: pgmap v214: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:26.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:26 smithi107 ceph-mon[34380]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T06:08:26.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:26 smithi031 ceph-mon[31237]: pgmap v214: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:26.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:26 smithi031 ceph-mon[31237]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T06:08:27.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:26 smithi110 ceph-mon[34550]: pgmap v214: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:27.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:26 smithi110 ceph-mon[34550]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-09-21T06:08:29.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:28 smithi110 ceph-mon[34550]: pgmap v215: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:28 smithi107 ceph-mon[34380]: pgmap v215: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:29.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:28 smithi031 ceph-mon[31237]: pgmap v215: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:30.965 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:30 smithi110 ceph-mon[34550]: pgmap v216: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:30 smithi107 ceph-mon[34380]: pgmap v216: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:31.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:30 smithi031 ceph-mon[31237]: pgmap v216: 1 pgs: 1 activating+degraded; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-09-21T06:08:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:31 smithi110 ceph-mon[34550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T06:08:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:31 smithi110 ceph-mon[34550]: Cluster is now healthy 2024-09-21T06:08:32.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:31 smithi107 ceph-mon[34380]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T06:08:32.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:31 smithi107 ceph-mon[34380]: Cluster is now healthy 2024-09-21T06:08:32.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:31 smithi031 ceph-mon[31237]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-09-21T06:08:32.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:31 smithi031 ceph-mon[31237]: Cluster is now healthy 2024-09-21T06:08:33.014 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:32 smithi110 ceph-mon[34550]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T06:08:33.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:32 smithi107 ceph-mon[34380]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T06:08:33.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:32 smithi031 ceph-mon[31237]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T06:08:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:08:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:33 smithi110 ceph-mon[34550]: Deploying daemon osd.11 on smithi110 2024-09-21T06:08:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:08:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:33 smithi107 ceph-mon[34380]: Deploying daemon osd.11 on smithi110 2024-09-21T06:08:34.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:08:34.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:34.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:33 smithi031 ceph-mon[31237]: Deploying daemon osd.11 on smithi110 2024-09-21T06:08:34.957 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:34 smithi110 ceph-mon[34550]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T06:08:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:34 smithi107 ceph-mon[34380]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T06:08:35.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:34 smithi031 ceph-mon[31237]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T06:08:37.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:36 smithi110 ceph-mon[34550]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:08:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:36 smithi107 ceph-mon[34380]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:08:37.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:36 smithi031 ceph-mon[31237]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T06:08:38.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:38 smithi110 ceph-mon[34550]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:38.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:38 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:39.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:38 smithi107 ceph-mon[34380]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:39.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:38 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:39.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:38 smithi031 ceph-mon[31237]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:39.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:38 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:40.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:39 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:40.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:39 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:40.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:39 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:40.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:39 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:40.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:39 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:40.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:39 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:41.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:40 smithi110 ceph-mon[34550]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:41.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:40 smithi107 ceph-mon[34380]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:41.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:40 smithi031 ceph-mon[31237]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:42.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.194+0000 7f22eb7fe640 1 -- 172.21.15.110:0/2303957656 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f22f4063800 con 0x7f22e00b6590 2024-09-21T06:08:42.196 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 11 on host 'smithi110' 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f22e00b6590 msgr2=0x7f22e00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f22e00b6590 0x7f22e00b8aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f22e4031440 tx=0x7f22e4009560 comp rx=0 tx=0).stop 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 msgr2=0x7f22f419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 0x7f22f419fd20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005fd0 tx=0x7f22ec00c7a0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 shutdown_connections 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4108480 0x7f22f419fd20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22f4107b30 0x7f22f419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f22e00b6590 0x7f22e00b8aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 --2- 172.21.15.110:0/2303957656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f41021f0 0x7f22f419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 >> 172.21.15.110:0/2303957656 conn(0x7f22f4077c60 msgr2=0x7f22f4105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:42.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 shutdown_connections 2024-09-21T06:08:42.200 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:42.197+0000 7f22f8f9a640 1 -- 172.21.15.110:0/2303957656 wait complete. 2024-09-21T06:08:42.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:42.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:42.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:42.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:42.891 DEBUG:teuthology.orchestra.run.smithi110:osd.11> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T06:08:42.893 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T06:08:42.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 -- 172.21.15.31:0/1164013329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 msgr2=0x7f15b4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/1164013329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b4110670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009ca0 tx=0x7f15ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 -- 172.21.15.31:0/1164013329 shutdown_connections 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/1164013329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b4110670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/1164013329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 0x7f15b4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/1164013329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b4103bb0 0x7f15b4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 -- 172.21.15.31:0/1164013329 >> 172.21.15.31:0/1164013329 conn(0x7f15b40ff600 msgr2=0x7f15b4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 -- 172.21.15.31:0/1164013329 shutdown_connections 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.626+0000 7f15bc3b2640 1 -- 172.21.15.31:0/1164013329 wait complete. 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.627+0000 7f15bc3b2640 1 Processor -- start 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.627+0000 7f15bc3b2640 1 -- start start 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.627+0000 7f15bc3b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b4103bb0 0x7f15b419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15bc3b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 0x7f15b419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15bc3b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15bc3b2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4113b00 con 0x7f15b4109e40 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15bc3b2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4113980 con 0x7f15b41094f0 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15bc3b2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4113c80 con 0x7f15b4103bb0 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49642/0 (socket says 172.21.15.31:49642) 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 -- 172.21.15.31:0/2071733923 learned_addr learned my addr 172.21.15.31:0/2071733923 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba14e640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b4103bb0 0x7f15b419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15b994d640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 0x7f15b419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 -- 172.21.15.31:0/2071733923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b4103bb0 msgr2=0x7f15b419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b4103bb0 0x7f15b419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 -- 172.21.15.31:0/2071733923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 msgr2=0x7f15b419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 0x7f15b419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 -- 172.21.15.31:0/2071733923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b41a03a0 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15b994d640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 0x7f15b419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ba94f640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b419fc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005cc0 tx=0x7f15ac02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.628+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003bf0 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.629+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ac003d90 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.629+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b41a0630 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.629+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac02fbc0 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.629+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41a6df0 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.629+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b4103fc0 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.631+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f15ac02fd60 con 0x7f15b4109e40 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.632+0000 7f15ab7fe640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15a00b6560 0x7f15a00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.632+0000 7f15ba14e640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15a00b6560 0x7f15a00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.632+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7024+0+0 (secure 0 0 0) 0x7f15ac1085a0 con 0x7f15b4109e40 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.633+0000 7f15ba14e640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15a00b6560 0x7f15a00b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005d00 tx=0x7f15b000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.635+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15ac0cf2c0 con 0x7f15b4109e40 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.823+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f15b419b560 con 0x7f15b4109e40 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.823+0000 7f15ab7fe640 1 -- 172.21.15.31:0/2071733923 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f15b419b560 con 0x7f15b4109e40 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15a00b6560 msgr2=0x7f15a00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15a00b6560 0x7f15a00b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005d00 tx=0x7f15b000b410 comp rx=0 tx=0).stop 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 msgr2=0x7f15b419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b419fc70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005cc0 tx=0x7f15ac02f380 comp rx=0 tx=0).stop 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 shutdown_connections 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15a00b6560 0x7f15a00b8a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15b4109e40 0x7f15b419fc70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b41094f0 0x7f15b419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 --2- 172.21.15.31:0/2071733923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15b4103bb0 0x7f15b419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.825+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 >> 172.21.15.31:0/2071733923 conn(0x7f15b40ff600 msgr2=0x7f15b4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.826+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 shutdown_connections 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:43.826+0000 7f15bc3b2640 1 -- 172.21.15.31:0/2071733923 wait complete. 2024-09-21T06:08:44.667 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726898903,"num_in_osds":12,"osd_in_since":1726898904,"num_remapped_pgs":0} 2024-09-21T06:08:44.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:44 smithi031 ceph-mon[31237]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:44.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:44 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2071733923' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:45.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:44 smithi110 ceph-mon[34550]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:45.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:44 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2071733923' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:45.029 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:08:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:08:44.799+0000 7f377ce05640 -1 osd.11 0 log_to_monitors true 2024-09-21T06:08:45.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:44 smithi107 ceph-mon[34380]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:45.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:44 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2071733923' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:45.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:08:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:46.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:45 smithi110 ceph-mon[34550]: from='osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:08:46.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:45 smithi110 ceph-mon[34550]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:08:46.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:45 smithi107 ceph-mon[34380]: from='osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:08:46.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:45 smithi107 ceph-mon[34380]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:08:46.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:45 smithi031 ceph-mon[31237]: from='osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:08:46.066 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:45 smithi031 ceph-mon[31237]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 -- 172.21.15.31:0/2018204669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 msgr2=0x7f2abc1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 --2- 172.21.15.31:0/2018204669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc1098e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009900 tx=0x7f2ab002a410 comp rx=0 tx=0).stop 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 -- 172.21.15.31:0/2018204669 shutdown_connections 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 --2- 172.21.15.31:0/2018204669 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc109e20 0x7f2abc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 --2- 172.21.15.31:0/2018204669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc1098e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 --2- 172.21.15.31:0/2018204669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc103b90 0x7f2abc103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 -- 172.21.15.31:0/2018204669 >> 172.21.15.31:0/2018204669 conn(0x7f2abc0ff600 msgr2=0x7f2abc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 -- 172.21.15.31:0/2018204669 shutdown_connections 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 -- 172.21.15.31:0/2018204669 wait complete. 2024-09-21T06:08:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.451+0000 7f2ac217b640 1 Processor -- start 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 -- start start 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 0x7f2abc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc109e20 0x7f2abc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc113ae0 con 0x7f2abc1094d0 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc113960 con 0x7f2abc109e20 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2ac217b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc113c60 con 0x7f2abc103b90 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 0x7f2abc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49662/0 (socket says 172.21.15.31:49662) 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 0x7f2abc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:57764/0 (socket says 172.21.15.31:57764) 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abaffd640 1 -- 172.21.15.31:0/1902730935 learned_addr learned my addr 172.21.15.31:0/1902730935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abbfff640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc109e20 0x7f2abc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abb7fe640 1 -- 172.21.15.31:0/1902730935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc109e20 msgr2=0x7f2abc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.452+0000 7f2abb7fe640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc109e20 0x7f2abc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2abb7fe640 1 -- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 msgr2=0x7f2abc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2abb7fe640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2abb7fe640 1 -- 172.21.15.31:0/1902730935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc1a0420 con 0x7f2abc103b90 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2abaffd640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2abb7fe640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 0x7f2abc19a4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00d6d0 tx=0x7f2aac00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac017070 con 0x7f2abc103b90 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc1a0710 con 0x7f2abc103b90 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aac009820 con 0x7f2abc103b90 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac004c40 con 0x7f2abc103b90 2024-09-21T06:08:46.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.453+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc1a6e70 con 0x7f2abc103b90 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.454+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc103fa0 con 0x7f2abc103b90 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.457+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2aac005220 con 0x7f2abc103b90 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.458+0000 7f2ab8ff9640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2aa40b6560 0x7f2aa40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.458+0000 7f2abaffd640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2aa40b6560 0x7f2aa40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.458+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7046+0+0 (secure 0 0 0) 0x7f2aac0e6cc0 con 0x7f2abc103b90 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.458+0000 7f2abaffd640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2aa40b6560 0x7f2aa40b8a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2abc108ec0 tx=0x7f2ab0032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:46.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.459+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2aac0b1e20 con 0x7f2abc103b90 2024-09-21T06:08:46.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.653+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2abc02e640 con 0x7f2abc103b90 2024-09-21T06:08:46.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.662+0000 7f2ab8ff9640 1 -- 172.21.15.31:0/1902730935 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f2aac0b6d60 con 0x7f2abc103b90 2024-09-21T06:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2aa40b6560 msgr2=0x7f2aa40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2aa40b6560 0x7f2aa40b8a70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2abc108ec0 tx=0x7f2ab0032ed0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 msgr2=0x7f2abc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 0x7f2abc19a4b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00d6d0 tx=0x7f2aac00db90 comp rx=0 tx=0).stop 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 shutdown_connections 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2abc109e20 0x7f2abc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2aa40b6560 0x7f2aa40b8a70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc1094d0 0x7f2abc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 --2- 172.21.15.31:0/1902730935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2abc103b90 0x7f2abc19a4b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:46.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 >> 172.21.15.31:0/1902730935 conn(0x7f2abc0ff600 msgr2=0x7f2abc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 shutdown_connections 2024-09-21T06:08:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:46.665+0000 7f2ac217b640 1 -- 172.21.15.31:0/1902730935 wait complete. 2024-09-21T06:08:47.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:47.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:08:47.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: Detected new or changed devices on smithi110 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:47.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:47.030 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:08:47.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: Detected new or changed devices on smithi110 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:47.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]} : dispatch 2024-09-21T06:08:47.110 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: Detected new or changed devices on smithi110 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi110 to 4272M 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:47.111 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726898903,"num_in_osds":12,"osd_in_since":1726898904,"num_remapped_pgs":0} 2024-09-21T06:08:47.659 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:08:47.346+0000 7f3778d87640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T06:08:47.660 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:08:47.364+0000 7f3770b7d640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:08:47.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:08:47.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:08:47.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:47.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:47.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1902730935' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:47.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: from='osd.11 ' entity='osd.11' 2024-09-21T06:08:47.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:08:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:08:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1902730935' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: from='osd.11 ' entity='osd.11' 2024-09-21T06:08:48.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1902730935' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: from='osd.11 ' entity='osd.11' 2024-09-21T06:08:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:48.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:08:48.844 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:48.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:48 smithi031 ceph-mon[31237]: purged_snaps scrub starts 2024-09-21T06:08:48.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:48 smithi031 ceph-mon[31237]: purged_snaps scrub ok 2024-09-21T06:08:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:48 smithi031 ceph-mon[31237]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:48 smithi031 ceph-mon[31237]: osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] boot 2024-09-21T06:08:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:48 smithi031 ceph-mon[31237]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T06:08:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:48 smithi110 ceph-mon[34550]: purged_snaps scrub starts 2024-09-21T06:08:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:48 smithi110 ceph-mon[34550]: purged_snaps scrub ok 2024-09-21T06:08:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:48 smithi110 ceph-mon[34550]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:48 smithi110 ceph-mon[34550]: osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] boot 2024-09-21T06:08:49.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:48 smithi110 ceph-mon[34550]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T06:08:49.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:48 smithi107 ceph-mon[34380]: purged_snaps scrub starts 2024-09-21T06:08:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:48 smithi107 ceph-mon[34380]: purged_snaps scrub ok 2024-09-21T06:08:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:48 smithi107 ceph-mon[34380]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:08:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:48 smithi107 ceph-mon[34380]: osd.11 [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] boot 2024-09-21T06:08:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:48 smithi107 ceph-mon[34380]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T06:08:49.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:08:49.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 -- 172.21.15.31:0/2307214189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c38108d00 msgr2=0x7f0c3810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/2307214189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c38108d00 0x7f0c3810f530 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009ca0 tx=0x7f0c3002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 -- 172.21.15.31:0/2307214189 shutdown_connections 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/2307214189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c38108d00 0x7f0c3810f530 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/2307214189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c381083b0 0x7f0c381087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/2307214189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 0x7f0c38102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 -- 172.21.15.31:0/2307214189 >> 172.21.15.31:0/2307214189 conn(0x7f0c38079710 msgr2=0x7f0c38079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 -- 172.21.15.31:0/2307214189 shutdown_connections 2024-09-21T06:08:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 -- 172.21.15.31:0/2307214189 wait complete. 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 Processor -- start 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.298+0000 7f0c3f13e640 1 -- start start 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3f13e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 0x7f0c38199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3f13e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 0x7f0c381998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3f13e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c38108d00 0x7f0c3819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3f13e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c381129c0 con 0x7f0c381083b0 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3f13e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112840 con 0x7f0c38108d00 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3f13e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112b40 con 0x7f0c38102a70 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 0x7f0c381998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 0x7f0c381998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37032/0 (socket says 172.21.15.31:37032) 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 -- 172.21.15.31:0/3999297039 learned_addr learned my addr 172.21.15.31:0/3999297039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3d6db640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c38108d00 0x7f0c3819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3ceda640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 0x7f0c38199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 -- 172.21.15.31:0/3999297039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 msgr2=0x7f0c38199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 0x7f0c38199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 -- 172.21.15.31:0/3999297039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c38108d00 msgr2=0x7f0c3819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c38108d00 0x7f0c3819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 -- 172.21.15.31:0/3999297039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3819f200 con 0x7f0c381083b0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c3ceda640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 0x7f0c38199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.299+0000 7f0c37fff640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 0x7f0c381998f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000e9f0 tx=0x7f0c2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.300+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000ccc0 con 0x7f0c381083b0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.300+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c200108e0 con 0x7f0c381083b0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.300+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3819f4f0 con 0x7f0c381083b0 2024-09-21T06:08:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.300+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20018860 con 0x7f0c381083b0 2024-09-21T06:08:49.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.300+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a5d10 con 0x7f0c381083b0 2024-09-21T06:08:49.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.301+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38103cf0 con 0x7f0c381083b0 2024-09-21T06:08:49.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.303+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f0c20010430 con 0x7f0c381083b0 2024-09-21T06:08:49.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.303+0000 7f0c35ffb640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0c240b6590 0x7f0c240b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:49.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.304+0000 7f0c3ceda640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0c240b6590 0x7f0c240b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:49.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.304+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f0c200e7840 con 0x7f0c381083b0 2024-09-21T06:08:49.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.304+0000 7f0c3ceda640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0c240b6590 0x7f0c240b8aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800baa0 tx=0x7f0c28005d90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:49.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.307+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0c200ae4a0 con 0x7f0c381083b0 2024-09-21T06:08:49.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.493+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0c3810ab20 con 0x7f0c381083b0 2024-09-21T06:08:49.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.494+0000 7f0c35ffb640 1 -- 172.21.15.31:0/3999297039 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7f0c200b2880 con 0x7f0c381083b0 2024-09-21T06:08:49.496 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:08:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0c240b6590 msgr2=0x7f0c240b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0c240b6590 0x7f0c240b8aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800baa0 tx=0x7f0c28005d90 comp rx=0 tx=0).stop 2024-09-21T06:08:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 msgr2=0x7f0c381998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 0x7f0c381998f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000e9f0 tx=0x7f0c2000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 shutdown_connections 2024-09-21T06:08:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c38108d00 0x7f0c3819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0c240b6590 0x7f0c240b8aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c381083b0 0x7f0c381998f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 --2- 172.21.15.31:0/3999297039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c38102a70 0x7f0c38199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 >> 172.21.15.31:0/3999297039 conn(0x7f0c38079710 msgr2=0x7f0c38106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 shutdown_connections 2024-09-21T06:08:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:49.496+0000 7f0c3f13e640 1 -- 172.21.15.31:0/3999297039 wait complete. 2024-09-21T06:08:49.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:49 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3999297039' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:50.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:49 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3999297039' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:50.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:49 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3999297039' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:08:50.370 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726898928,"num_in_osds":12,"osd_in_since":1726898904,"num_remapped_pgs":0} 2024-09-21T06:08:50.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:08:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:50.919 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:50 smithi031 ceph-mon[31237]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:50.920 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:50 smithi031 ceph-mon[31237]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:08:51.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:50 smithi110 ceph-mon[34550]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:51.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:50 smithi110 ceph-mon[34550]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:08:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:50 smithi107 ceph-mon[34380]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:51.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:50 smithi107 ceph-mon[34380]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/364876956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581020c0 msgr2=0x7f8a581024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/364876956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581020c0 0x7f8a581024d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009900 tx=0x7f8a4802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/364876956 shutdown_connections 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/364876956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a58106c00 0x7f8a58110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/364876956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a581062b0 0x7f8a581066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/364876956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581020c0 0x7f8a581024d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.111+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/364876956 >> 172.21.15.31:0/364876956 conn(0x7f8a58079e40 msgr2=0x7f8a5807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.112+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/364876956 shutdown_connections 2024-09-21T06:08:51.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.112+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/364876956 wait complete. 2024-09-21T06:08:51.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.112+0000 7f8a5e5a5640 1 Processor -- start 2024-09-21T06:08:51.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.112+0000 7f8a5e5a5640 1 -- start start 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5e5a5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a581020c0 0x7f8a5819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5e5a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 0x7f8a5819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5e5a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a58106c00 0x7f8a5819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5e5a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113b00 con 0x7f8a581062b0 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5e5a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113980 con 0x7f8a58106c00 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5e5a5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113c80 con 0x7f8a581020c0 2024-09-21T06:08:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 0x7f8a5819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 0x7f8a5819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37054/0 (socket says 172.21.15.31:37054) 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 -- 172.21.15.31:0/165471720 learned_addr learned my addr 172.21.15.31:0/165471720 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a57fff640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a581020c0 0x7f8a5819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a5cb42640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a58106c00 0x7f8a5819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 -- 172.21.15.31:0/165471720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a581020c0 msgr2=0x7f8a5819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a581020c0 0x7f8a5819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 -- 172.21.15.31:0/165471720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a58106c00 msgr2=0x7f8a5819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a58106c00 0x7f8a5819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 -- 172.21.15.31:0/165471720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581a03a0 con 0x7f8a581062b0 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.113+0000 7f8a577fe640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 0x7f8a5819aa90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00c8c0 tx=0x7f8a4c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.114+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c007bb0 con 0x7f8a581062b0 2024-09-21T06:08:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.114+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581a0690 con 0x7f8a581062b0 2024-09-21T06:08:51.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.114+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4c007d50 con 0x7f8a581062b0 2024-09-21T06:08:51.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.114+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0187c0 con 0x7f8a581062b0 2024-09-21T06:08:51.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.114+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581a6eb0 con 0x7f8a581062b0 2024-09-21T06:08:51.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.115+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a581024d0 con 0x7f8a581062b0 2024-09-21T06:08:51.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.116+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f8a4c00fa20 con 0x7f8a581062b0 2024-09-21T06:08:51.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.117+0000 7f8a557fa640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a400b6590 0x7f8a400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:51.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.117+0000 7f8a57fff640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a400b6590 0x7f8a400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:51.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.117+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f8a4c0e72a0 con 0x7f8a581062b0 2024-09-21T06:08:51.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.118+0000 7f8a57fff640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a400b6590 0x7f8a400b8aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a48031440 tx=0x7f8a48005e80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.120+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a4c014030 con 0x7f8a581062b0 2024-09-21T06:08:51.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.308+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a5819b620 con 0x7f8a581062b0 2024-09-21T06:08:51.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.309+0000 7f8a557fa640 1 -- 172.21.15.31:0/165471720 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18799 (secure 0 0 0) 0x7f8a5819b620 con 0x7f8a581062b0 2024-09-21T06:08:51.311 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:08:51.311 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":70,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:08:49.660033+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.311+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a400b6590 msgr2=0x7f8a400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:51.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.311+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a400b6590 0x7f8a400b8aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a48031440 tx=0x7f8a48005e80 comp rx=0 tx=0).stop 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.311+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 msgr2=0x7f8a5819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.311+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 0x7f8a5819aa90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00c8c0 tx=0x7f8a4c00cd80 comp rx=0 tx=0).stop 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 shutdown_connections 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a58106c00 0x7f8a5819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a400b6590 0x7f8a400b8aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a581062b0 0x7f8a5819aa90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 --2- 172.21.15.31:0/165471720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a581020c0 0x7f8a5819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 >> 172.21.15.31:0/165471720 conn(0x7f8a58079e40 msgr2=0x7f8a581080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 shutdown_connections 2024-09-21T06:08:51.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:51.312+0000 7f8a5e5a5640 1 -- 172.21.15.31:0/165471720 wait complete. 2024-09-21T06:08:51.914 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:05:05.978676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:08:51.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T06:08:51.938 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:51 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/165471720' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:08:52.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:51 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/165471720' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:08:52.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:51 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/165471720' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:08:52.170 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:52.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 -- 172.21.15.31:0/1160763837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 msgr2=0x7fcafc10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:52.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 --2- 172.21.15.31:0/1160763837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc10d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009900 tx=0x7fcaf002a410 comp rx=0 tx=0).stop 2024-09-21T06:08:52.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 -- 172.21.15.31:0/1160763837 shutdown_connections 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 --2- 172.21.15.31:0/1160763837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc10ded0 0x7fcafc110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 --2- 172.21.15.31:0/1160763837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc10d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 --2- 172.21.15.31:0/1160763837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcafc104920 0x7fcafc104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 -- 172.21.15.31:0/1160763837 >> 172.21.15.31:0/1160763837 conn(0x7fcafc0797b0 msgr2=0x7fcafc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 -- 172.21.15.31:0/1160763837 shutdown_connections 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.676+0000 7fcb01f64640 1 -- 172.21.15.31:0/1160763837 wait complete. 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 Processor -- start 2024-09-21T06:08:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 -- start start 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc104920 0x7fcafc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcafc10ded0 0x7fcafc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc113af0 con 0x7fcafc105270 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc113970 con 0x7fcafc104920 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcb01f64640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc113c70 con 0x7fcafc10ded0 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcafaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcafaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37070/0 (socket says 172.21.15.31:37070) 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcafaffd640 1 -- 172.21.15.31:0/2776959874 learned_addr learned my addr 172.21.15.31:0/2776959874 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcafb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc104920 0x7fcafc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:52.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.677+0000 7fcafbfff640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcafc10ded0 0x7fcafc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafaffd640 1 -- 172.21.15.31:0/2776959874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcafc10ded0 msgr2=0x7fcafc19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafaffd640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcafc10ded0 0x7fcafc19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafaffd640 1 -- 172.21.15.31:0/2776959874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc104920 msgr2=0x7fcafc19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafaffd640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc104920 0x7fcafc19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafaffd640 1 -- 172.21.15.31:0/2776959874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc1a04f0 con 0x7fcafc105270 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafb7fe640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc104920 0x7fcafc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcafaffd640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc19ab20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005ab0 tx=0x7fcaf0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf003f070 con 0x7fcafc105270 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf0031d60 con 0x7fcafc105270 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0035030 con 0x7fcafc105270 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc1a0780 con 0x7fcafc105270 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.678+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc102710 con 0x7fcafc105270 2024-09-21T06:08:52.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.679+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc06be10 con 0x7fcafc105270 2024-09-21T06:08:52.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.681+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fcaf0042070 con 0x7fcafc105270 2024-09-21T06:08:52.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.682+0000 7fcaf8ff9640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcae40b65e0 0x7fcae40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:52.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.682+0000 7fcafb7fe640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcae40b65e0 0x7fcae40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:52.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.682+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fcaf0107e30 con 0x7fcafc105270 2024-09-21T06:08:52.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.683+0000 7fcafb7fe640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcae40b65e0 0x7fcae40b8af0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:52.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.685+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcaf00d2dd0 con 0x7fcafc105270 2024-09-21T06:08:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.872+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcafc108fa0 con 0x7fcafc105270 2024-09-21T06:08:52.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:52 smithi031 ceph-mon[31237]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.872+0000 7fcaf8ff9640 1 -- 172.21.15.31:0/2776959874 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fcaf00d7d10 con 0x7fcafc105270 2024-09-21T06:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-21T06:08:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcae40b65e0 msgr2=0x7fcae40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcae40b65e0 0x7fcae40b8af0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009fd0 tx=0x7fcaec009680 comp rx=0 tx=0).stop 2024-09-21T06:08:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 msgr2=0x7fcafc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc19ab20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005ab0 tx=0x7fcaf0003cd0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 shutdown_connections 2024-09-21T06:08:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcafc10ded0 0x7fcafc19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcae40b65e0 0x7fcae40b8af0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcafc105270 0x7fcafc19ab20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 --2- 172.21.15.31:0/2776959874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcafc104920 0x7fcafc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 >> 172.21.15.31:0/2776959874 conn(0x7fcafc0797b0 msgr2=0x7fcafc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 shutdown_connections 2024-09-21T06:08:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:52.875+0000 7fcb01f64640 1 -- 172.21.15.31:0/2776959874 wait complete. 2024-09-21T06:08:53.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:52 smithi110 ceph-mon[34550]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:53.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:52 smithi107 ceph-mon[34380]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:53.479 INFO:tasks.cephadm:Adding mds.a on smithi031 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.d on smithi031 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.g on smithi031 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.j on smithi031 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.b on smithi107 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.e on smithi107 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.h on smithi107 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.k on smithi107 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.c on smithi110 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.f on smithi110 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.i on smithi110 2024-09-21T06:08:53.480 INFO:tasks.cephadm:Adding mds.l on smithi110 2024-09-21T06:08:53.480 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l' 2024-09-21T06:08:53.744 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:08:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:53 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2776959874' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:08:53.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:53 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2776959874' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:08:54.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:53 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2776959874' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 -- 172.21.15.110:0/2867288979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4107b70 msgr2=0x7f19b4107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 --2- 172.21.15.110:0/2867288979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4107b70 0x7f19b4107f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f199c00aa10 tx=0x7f199c02b720 comp rx=0 tx=0).stop 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 -- 172.21.15.110:0/2867288979 shutdown_connections 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 --2- 172.21.15.110:0/2867288979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b41084c0 0x7f19b410ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 --2- 172.21.15.110:0/2867288979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4107b70 0x7f19b4107f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 --2- 172.21.15.110:0/2867288979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b4102230 0x7f19b4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 -- 172.21.15.110:0/2867288979 >> 172.21.15.110:0/2867288979 conn(0x7f19b4079e40 msgr2=0x7f19b407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 -- 172.21.15.110:0/2867288979 shutdown_connections 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.214+0000 7f19b9e29640 1 -- 172.21.15.110:0/2867288979 wait complete. 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 Processor -- start 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 -- start start 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 0x7f19b40fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b4107b70 0x7f19b40ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b41084c0 0x7f19b40ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4111900 con 0x7f19b41084c0 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4111780 con 0x7f19b4107b70 2024-09-21T06:08:54.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b9e29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19b4111a80 con 0x7f19b4102230 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 0x7f19b40fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 0x7f19b40fee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:33778/0 (socket says 172.21.15.110:33778) 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 -- 172.21.15.110:0/1152655181 learned_addr learned my addr 172.21.15.110:0/1152655181 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b3fff640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b41084c0 0x7f19b40ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19aaffd640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b4107b70 0x7f19b40ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 -- 172.21.15.110:0/1152655181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b4107b70 msgr2=0x7f19b40ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b4107b70 0x7f19b40ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 -- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b41084c0 msgr2=0x7f19b40ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.215+0000 7f19b37fe640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b41084c0 0x7f19b40ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b37fe640 1 -- 172.21.15.110:0/1152655181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b41a87b0 con 0x7f19b4102230 2024-09-21T06:08:54.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b37fe640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 0x7f19b40fee70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f19a400c890 tx=0x7f19a400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:54.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4004220 con 0x7f19b4102230 2024-09-21T06:08:54.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19a40043c0 con 0x7f19b4102230 2024-09-21T06:08:54.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b41a8aa0 con 0x7f19b4102230 2024-09-21T06:08:54.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a4004a00 con 0x7f19b4102230 2024-09-21T06:08:54.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.216+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b41a8fb0 con 0x7f19b4102230 2024-09-21T06:08:54.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.217+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b4102640 con 0x7f19b4102230 2024-09-21T06:08:54.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.219+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f19a40079b0 con 0x7f19b4102230 2024-09-21T06:08:54.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.220+0000 7f19b17fa640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6560 0x7f19a00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:54.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.220+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f19a40e7110 con 0x7f19b4102230 2024-09-21T06:08:54.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.220+0000 7f19aaffd640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6560 0x7f19a00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:54.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.221+0000 7f19aaffd640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6560 0x7f19a00b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f199c00a340 tx=0x7f199c030510 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:54.228 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.223+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19a4014030 con 0x7f19b4102230 2024-09-21T06:08:54.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.411+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}) -- 0x7f19b4063800 con 0x7f19a00b6560 2024-09-21T06:08:54.421 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.419+0000 7f19b17fa640 1 -- 172.21.15.110:0/1152655181 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f19b4063800 con 0x7f19a00b6560 2024-09-21T06:08:54.421 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mds.all update... 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6560 msgr2=0x7f19a00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6560 0x7f19a00b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f199c00a340 tx=0x7f199c030510 comp rx=0 tx=0).stop 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 msgr2=0x7f19b40fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 0x7f19b40fee70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f19a400c890 tx=0x7f19a400cd50 comp rx=0 tx=0).stop 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 shutdown_connections 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6560 0x7f19a00b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.424 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b41084c0 0x7f19b40ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b4107b70 0x7f19b40ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 --2- 172.21.15.110:0/1152655181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b4102230 0x7f19b40fee70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 >> 172.21.15.110:0/1152655181 conn(0x7f19b4079e40 msgr2=0x7f19b4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 shutdown_connections 2024-09-21T06:08:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-21T06:08:54.422+0000 7f19b9e29640 1 -- 172.21.15.110:0/1152655181 wait complete. 2024-09-21T06:08:55.112 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T06:08:55.115 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T06:08:55.159 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T06:08:55.202 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T06:08:55.245 DEBUG:teuthology.orchestra.run.smithi107:mds.b> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T06:08:55.248 DEBUG:teuthology.orchestra.run.smithi107:mds.e> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T06:08:55.291 DEBUG:teuthology.orchestra.run.smithi107:mds.h> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T06:08:55.334 DEBUG:teuthology.orchestra.run.smithi107:mds.k> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T06:08:55.376 DEBUG:teuthology.orchestra.run.smithi110:mds.c> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T06:08:55.379 DEBUG:teuthology.orchestra.run.smithi110:mds.f> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T06:08:55.423 DEBUG:teuthology.orchestra.run.smithi110:mds.i> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T06:08:55.465 DEBUG:teuthology.orchestra.run.smithi110:mds.l> sudo journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T06:08:55.510 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T06:08:55.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:08:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-09-21T06:08:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:08:55.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:55.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:08:55.623 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:08:55.623 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:55.884 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='client.24604 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-09-21T06:08:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:55.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:55.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:08:55.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:08:55.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.193+0000 7f70eed7f640 1 -- 172.21.15.31:0/1841389964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8103b60 msgr2=0x7f70e8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.193+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1841389964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8103b60 0x7f70e8103f70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009900 tx=0x7f70d002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 -- 172.21.15.31:0/1841389964 shutdown_connections 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1841389964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70e8109df0 0x7f70e8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1841389964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70e81094a0 0x7f70e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1841389964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8103b60 0x7f70e8103f70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 -- 172.21.15.31:0/1841389964 >> 172.21.15.31:0/1841389964 conn(0x7f70e807b850 msgr2=0x7f70e807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:56.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 -- 172.21.15.31:0/1841389964 shutdown_connections 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 -- 172.21.15.31:0/1841389964 wait complete. 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.194+0000 7f70eed7f640 1 Processor -- start 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 -- start start 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70e8103b60 0x7f70e819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70e81094a0 0x7f70e819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 0x7f70e819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:56.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8113ab0 con 0x7f70e8109df0 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8113930 con 0x7f70e81094a0 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70eed7f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8113c30 con 0x7f70e8103b60 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 0x7f70e819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 0x7f70e819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37082/0 (socket says 172.21.15.31:37082) 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ecb1b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70e8103b60 0x7f70e819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 -- 172.21.15.31:0/1980345674 learned_addr learned my addr 172.21.15.31:0/1980345674 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70e7fff640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70e81094a0 0x7f70e819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 -- 172.21.15.31:0/1980345674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70e8103b60 msgr2=0x7f70e819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:56.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70e8103b60 0x7f70e819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 -- 172.21.15.31:0/1980345674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70e81094a0 msgr2=0x7f70e819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:56.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70e81094a0 0x7f70e819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.195+0000 7f70ed31c640 1 -- 172.21.15.31:0/1980345674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e81a03b0 con 0x7f70e8109df0 2024-09-21T06:08:56.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.196+0000 7f70ed31c640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 0x7f70e819fd40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70e000cce0 tx=0x7f70e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:56.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.196+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e000f450 con 0x7f70e8109df0 2024-09-21T06:08:56.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.196+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70e000faa0 con 0x7f70e8109df0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.196+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e81a06a0 con 0x7f70e8109df0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.196+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0018b70 con 0x7f70e8109df0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.196+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a0b90 con 0x7f70e8109df0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.197+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8103f70 con 0x7f70e8109df0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.198+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f70e0018d10 con 0x7f70e8109df0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.199+0000 7f70e5ffb640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70d40b6590 0x7f70d40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.199+0000 7f70ecb1b640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70d40b6590 0x7f70d40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:56.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.199+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f70e00e72f0 con 0x7f70e8109df0 2024-09-21T06:08:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.200+0000 7f70ecb1b640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70d40b6590 0x7f70d40b8aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70d0031440 tx=0x7f70d0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:56.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.204+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70e81a0b90 con 0x7f70e8109df0 2024-09-21T06:08:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.431+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70e810bbf0 con 0x7f70e8109df0 2024-09-21T06:08:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.432+0000 7f70e5ffb640 1 -- 172.21.15.31:0/1980345674 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f70e0014030 con 0x7f70e8109df0 2024-09-21T06:08:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:08:56.434 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"btime":"2024-09-21T06:01:12:820344+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T06:08:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2024-09-21T06:08:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70d40b6590 msgr2=0x7f70d40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70d40b6590 0x7f70d40b8aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f70d0031440 tx=0x7f70d0009560 comp rx=0 tx=0).stop 2024-09-21T06:08:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 msgr2=0x7f70e819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 0x7f70e819fd40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f70e000cce0 tx=0x7f70e000a620 comp rx=0 tx=0).stop 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 shutdown_connections 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70d40b6590 0x7f70d40b8aa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70e8109df0 0x7f70e819fd40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70e81094a0 0x7f70e819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 --2- 172.21.15.31:0/1980345674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f70e8103b60 0x7f70e819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 >> 172.21.15.31:0/1980345674 conn(0x7f70e807b850 msgr2=0x7f70e81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 shutdown_connections 2024-09-21T06:08:56.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:56.434+0000 7f70eed7f640 1 -- 172.21.15.31:0/1980345674 wait complete. 2024-09-21T06:08:56.446 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:56 smithi031 ceph-mon[31237]: Deploying daemon mds.l on smithi110 2024-09-21T06:08:56.575 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:56 smithi110 ceph-mon[34550]: Deploying daemon mds.l on smithi110 2024-09-21T06:08:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:56 smithi107 ceph-mon[34380]: Deploying daemon mds.l on smithi110 2024-09-21T06:08:56.968 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:56 smithi110 systemd[1]: Starting Ceph mds.l for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:08:57.239 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:56 smithi110 podman[63357]: 2024-09-21 06:08:56.869793847 +0000 UTC m=+0.015702113 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:08:57.239 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:56 smithi110 podman[63357]: 2024-09-21 06:08:56.977147463 +0000 UTC m=+0.123055769 container create 9ed8dca21fd2bb50753f79490ac4f38e54e818f27c48e9b54aa39c05ce1018ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:08:57.239 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:57 smithi110 podman[63357]: 2024-09-21 06:08:57.237878093 +0000 UTC m=+0.383786356 container init 9ed8dca21fd2bb50753f79490ac4f38e54e818f27c48e9b54aa39c05ce1018ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T06:08:57.528 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:57 smithi110 podman[63357]: 2024-09-21 06:08:57.240971019 +0000 UTC m=+0.386879291 container start 9ed8dca21fd2bb50753f79490ac4f38e54e818f27c48e9b54aa39c05ce1018ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:08:57.528 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mds[63376]: -- 172.21.15.110:0/3351064434 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f33f1f1860 con 0x55f343399600 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l[63372]: starting mds.l at 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:57 smithi110 bash[63357]: 9ed8dca21fd2bb50753f79490ac4f38e54e818f27c48e9b54aa39c05ce1018ea 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:08:57 smithi110 systemd[1]: Started Ceph mds.l for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1980345674' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.110:6832/128096132,v1:172.21.15.110:6833/128096132] up:boot 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: fsmap 1 up:standby 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:08:57.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:08:57.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:08:57.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:57.530 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:57 smithi110 ceph-mon[34550]: Deploying daemon mds.k on smithi107 2024-09-21T06:08:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1980345674' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:08:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.110:6832/128096132,v1:172.21.15.110:6833/128096132] up:boot 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: fsmap 1 up:standby 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:57.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:57 smithi107 ceph-mon[34380]: Deploying daemon mds.k on smithi107 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1980345674' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.110:6832/128096132,v1:172.21.15.110:6833/128096132] up:boot 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: fsmap 1 up:standby 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:08:57.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:08:57.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:57.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:57 smithi031 ceph-mon[31237]: Deploying daemon mds.k on smithi107 2024-09-21T06:08:58.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:08:58 smithi107 ceph-mon[34380]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:08:58 smithi110 ceph-mon[34550]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:58.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:08:58 smithi031 ceph-mon[31237]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:08:59.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:08:59.216 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 systemd[1]: Starting Ceph mds.k for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:08:59.424 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:59.516 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 podman[62956]: 2024-09-21 06:08:59.32822891 +0000 UTC m=+0.015558269 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:08:59.516 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 podman[62956]: 2024-09-21 06:08:59.426974092 +0000 UTC m=+0.114303444 container create f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:08:59.789 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 podman[62956]: 2024-09-21 06:08:59.671471232 +0000 UTC m=+0.358800581 container init f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T06:08:59.789 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 podman[62956]: 2024-09-21 06:08:59.674357496 +0000 UTC m=+0.361686855 container start f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T06:08:59.789 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 ceph-mds[62975]: -- 172.21.15.107:0/3839384681 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb10bcd860 con 0x55cb14d75180 2024-09-21T06:08:59.789 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k[62971]: starting mds.k at 2024-09-21T06:08:59.789 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 bash[62956]: f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 2024-09-21T06:08:59.789 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:08:59 smithi107 systemd[1]: Started Ceph mds.k for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:08:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 -- 172.21.15.31:0/2457334560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 msgr2=0x7fb9481087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/2457334560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb9481087b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb930009900 tx=0x7fb93002a410 comp rx=0 tx=0).stop 2024-09-21T06:08:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 -- 172.21.15.31:0/2457334560 shutdown_connections 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/2457334560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb948108cf0 0x7fb94810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/2457334560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb9481087b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/2457334560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb948102a60 0x7fb948102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 -- 172.21.15.31:0/2457334560 >> 172.21.15.31:0/2457334560 conn(0x7fb948079730 msgr2=0x7fb948079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 -- 172.21.15.31:0/2457334560 shutdown_connections 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 -- 172.21.15.31:0/2457334560 wait complete. 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 Processor -- start 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.906+0000 7fb94d5b5640 1 -- start start 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb94d5b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb948102a60 0x7fb94819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb94d5b5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb94819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb94d5b5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb948108cf0 0x7fb94819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb94d5b5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb948112960 con 0x7fb9481083a0 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb94d5b5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9481127e0 con 0x7fb948108cf0 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb94d5b5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb948112ae0 con 0x7fb948102a60 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb94819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb94819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51050/0 (socket says 172.21.15.31:51050) 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 -- 172.21.15.31:0/1991509594 learned_addr learned my addr 172.21.15.31:0/1991509594 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb946ffd640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb948102a60 0x7fb94819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb9477fe640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb948108cf0 0x7fb94819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 -- 172.21.15.31:0/1991509594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb948102a60 msgr2=0x7fb94819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb948102a60 0x7fb94819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 -- 172.21.15.31:0/1991509594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb948108cf0 msgr2=0x7fb94819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb948108cf0 0x7fb94819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 -- 172.21.15.31:0/1991509594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481a03d0 con 0x7fb9481083a0 2024-09-21T06:08:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.907+0000 7fb93ffff640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb94819aa00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb930006010 tx=0x7fb930004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.908+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93003f030 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.908+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb930003aa0 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.908+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9481a0660 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.908+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930035070 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.908+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9481a6e20 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.909+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948103ce0 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.912+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb930042030 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.913+0000 7fb944ff9640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb9340b6560 0x7fb9340b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.913+0000 7fb946ffd640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb9340b6560 0x7fb9340b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.913+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fb9301078a0 con 0x7fb9481083a0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.913+0000 7fb946ffd640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb9340b6560 0x7fb9340b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb938005d00 tx=0x7fb93800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:08:59.915+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9300d2840 con 0x7fb9481083a0 2024-09-21T06:09:00.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.158+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb94810baf0 con 0x7fb9481083a0 2024-09-21T06:09:00.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.159+0000 7fb944ff9640 1 -- 172.21.15.31:0/1991509594 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fb9300d7780 con 0x7fb9481083a0 2024-09-21T06:09:00.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:00.161 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"btime":"2024-09-21T06:08:57:279913+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T06:09:00.161 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb9340b6560 msgr2=0x7fb9340b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb9340b6560 0x7fb9340b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb938005d00 tx=0x7fb93800b410 comp rx=0 tx=0).stop 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 msgr2=0x7fb94819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb94819aa00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb930006010 tx=0x7fb930004060 comp rx=0 tx=0).stop 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 shutdown_connections 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb948108cf0 0x7fb94819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb9340b6560 0x7fb9340b8a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:00.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9481083a0 0x7fb94819aa00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 --2- 172.21.15.31:0/1991509594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb948102a60 0x7fb94819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 >> 172.21.15.31:0/1991509594 conn(0x7fb948079730 msgr2=0x7fb94806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 shutdown_connections 2024-09-21T06:09:00.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:00.161+0000 7fb94d5b5640 1 -- 172.21.15.31:0/1991509594 wait complete. 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:01.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:01.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: Deploying daemon mds.j on smithi031 2024-09-21T06:09:01.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1991509594' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:01.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6832/4010348472,v1:172.21.15.107:6833/4010348472] up:boot 2024-09-21T06:09:01.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: fsmap 2 up:standby 2024-09-21T06:09:01.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:00 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: Deploying daemon mds.j on smithi031 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1991509594' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:01.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6832/4010348472,v1:172.21.15.107:6833/4010348472] up:boot 2024-09-21T06:09:01.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: fsmap 2 up:standby 2024-09-21T06:09:01.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:00 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:09:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: Deploying daemon mds.j on smithi031 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1991509594' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6832/4010348472,v1:172.21.15.107:6833/4010348472] up:boot 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: fsmap 2 up:standby 2024-09-21T06:09:01.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:00 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:09:02.629 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 systemd[1]: Starting Ceph mds.j for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:02.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:02 smithi031 ceph-mon[31237]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:02.881 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 podman[73049]: 2024-09-21 06:09:02.530236169 +0000 UTC m=+0.016507030 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:02.881 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 podman[73049]: 2024-09-21 06:09:02.630680996 +0000 UTC m=+0.116951841 container create 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:09:02.882 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 podman[73049]: 2024-09-21 06:09:02.874831319 +0000 UTC m=+0.361102165 container init 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T06:09:02.882 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 podman[73049]: 2024-09-21 06:09:02.877979156 +0000 UTC m=+0.364250003 container start 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T06:09:03.140 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 ceph-mds[73068]: -- 172.21.15.31:0/2436600700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562ad86db860 con 0x562adc883a80 2024-09-21T06:09:03.140 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j[73064]: starting mds.j at 2024-09-21T06:09:03.140 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 bash[73049]: 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 2024-09-21T06:09:03.140 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:09:02 smithi031 systemd[1]: Started Ceph mds.j for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:03.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:02 smithi110 ceph-mon[34550]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:03.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:02 smithi107 ceph-mon[34380]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:03.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:04.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: Deploying daemon mds.d on smithi031 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.31:6834/3018651170,v1:172.21.15.31:6835/3018651170] up:boot 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: fsmap 3 up:standby 2024-09-21T06:09:04.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:04 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: Deploying daemon mds.d on smithi031 2024-09-21T06:09:04.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.31:6834/3018651170,v1:172.21.15.31:6835/3018651170] up:boot 2024-09-21T06:09:04.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: fsmap 3 up:standby 2024-09-21T06:09:04.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:04 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:09:04.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: Deploying daemon mds.d on smithi031 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.31:6834/3018651170,v1:172.21.15.31:6835/3018651170] up:boot 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: fsmap 3 up:standby 2024-09-21T06:09:04.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:04 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:09:04.686 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:05.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:05 smithi031 ceph-mon[31237]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 -- 172.21.15.31:0/1255133216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3201083c0 msgr2=0x7fb3201087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 --2- 172.21.15.31:0/1255133216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3201083c0 0x7fb3201087d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb310009900 tx=0x7fb31002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 -- 172.21.15.31:0/1255133216 shutdown_connections 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 --2- 172.21.15.31:0/1255133216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb320108d10 0x7fb32010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 --2- 172.21.15.31:0/1255133216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3201083c0 0x7fb3201087d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 --2- 172.21.15.31:0/1255133216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb320102a80 0x7fb320102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 -- 172.21.15.31:0/1255133216 >> 172.21.15.31:0/1255133216 conn(0x7fb320079740 msgr2=0x7fb320079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 -- 172.21.15.31:0/1255133216 shutdown_connections 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 -- 172.21.15.31:0/1255133216 wait complete. 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.264+0000 7fb328594640 1 Processor -- start 2024-09-21T06:09:05.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 -- start start 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb320102a80 0x7fb32019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3201083c0 0x7fb32019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 0x7fb3201a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb320112980 con 0x7fb320108d10 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb320112800 con 0x7fb320102a80 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb328594640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb320112b00 con 0x7fb3201083c0 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 0x7fb3201a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 0x7fb3201a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51090/0 (socket says 172.21.15.31:51090) 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 -- 172.21.15.31:0/1159074884 learned_addr learned my addr 172.21.15.31:0/1159074884 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb325b2f640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3201083c0 0x7fb32019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:05.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326330640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb320102a80 0x7fb32019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:05.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 -- 172.21.15.31:0/1159074884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3201083c0 msgr2=0x7fb32019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:05.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3201083c0 0x7fb32019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 -- 172.21.15.31:0/1159074884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb320102a80 msgr2=0x7fb32019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:05.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb320102a80 0x7fb32019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.265+0000 7fb326b31640 1 -- 172.21.15.31:0/1159074884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3201a25e0 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.266+0000 7fb326b31640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 0x7fb3201a1eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb31800eaa0 tx=0x7fb31800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.266+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800cdb0 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.266+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb318005020 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.266+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3201a28d0 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.266+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3180116d0 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.266+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3201a9030 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.267+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb320103d00 con 0x7fb320108d10 2024-09-21T06:09:05.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.268+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb3180051d0 con 0x7fb320108d10 2024-09-21T06:09:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.269+0000 7fb3177fe640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb30c0b6560 0x7fb30c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.269+0000 7fb326330640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb30c0b6560 0x7fb30c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.269+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fb3180e70f0 con 0x7fb320108d10 2024-09-21T06:09:05.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.270+0000 7fb326330640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb30c0b6560 0x7fb30c0b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0058c0 tx=0x7fb31c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:05.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.274+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb318016030 con 0x7fb320108d10 2024-09-21T06:09:05.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.511+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb32019d7a0 con 0x7fb320108d10 2024-09-21T06:09:05.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.511+0000 7fb3177fe640 1 -- 172.21.15.31:0/1159074884 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fb3180b2130 con 0x7fb320108d10 2024-09-21T06:09:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:05.514 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"btime":"2024-09-21T06:09:03:789524+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:09:05.514 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2024-09-21T06:09:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb30c0b6560 msgr2=0x7fb30c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb30c0b6560 0x7fb30c0b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb31c0058c0 tx=0x7fb31c00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 msgr2=0x7fb3201a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 0x7fb3201a1eb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb31800eaa0 tx=0x7fb31800ef60 comp rx=0 tx=0).stop 2024-09-21T06:09:05.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 shutdown_connections 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb30c0b6560 0x7fb30c0b8a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb320108d10 0x7fb3201a1eb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3201083c0 0x7fb32019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 --2- 172.21.15.31:0/1159074884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb320102a80 0x7fb32019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 >> 172.21.15.31:0/1159074884 conn(0x7fb320079740 msgr2=0x7fb32006bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 shutdown_connections 2024-09-21T06:09:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:05.514+0000 7fb328594640 1 -- 172.21.15.31:0/1159074884 wait complete. 2024-09-21T06:09:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:05 smithi110 ceph-mon[34550]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:05 smithi107 ceph-mon[34380]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:06.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:06 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1159074884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:06.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:06 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1159074884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:06.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:06 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1159074884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:06.636 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:09:06 smithi031 systemd[1]: Started Ceph mds.d for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:07.173 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:07.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:07.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:07.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:07.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:07.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:08.162 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:08 smithi107 ceph-mon[34380]: Deploying daemon mds.e on smithi107 2024-09-21T06:09:08.162 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:08 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.31:6836/594088756,v1:172.21.15.31:6837/594088756] up:boot 2024-09-21T06:09:08.162 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:08 smithi107 ceph-mon[34380]: fsmap 4 up:standby 2024-09-21T06:09:08.162 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:08 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:09:08.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:08.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:08 smithi031 ceph-mon[31237]: Deploying daemon mds.e on smithi107 2024-09-21T06:09:08.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:08 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.31:6836/594088756,v1:172.21.15.31:6837/594088756] up:boot 2024-09-21T06:09:08.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:08 smithi031 ceph-mon[31237]: fsmap 4 up:standby 2024-09-21T06:09:08.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:08 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:09:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:08 smithi110 ceph-mon[34550]: Deploying daemon mds.e on smithi107 2024-09-21T06:09:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:08 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.31:6836/594088756,v1:172.21.15.31:6837/594088756] up:boot 2024-09-21T06:09:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:08 smithi110 ceph-mon[34550]: fsmap 4 up:standby 2024-09-21T06:09:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:08 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:09:08.789 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:08 smithi107 systemd[1]: Starting Ceph mds.e for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.931+0000 7f45e8be4640 1 -- 172.21.15.31:0/4251556675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e41083d0 msgr2=0x7f45e41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.931+0000 7f45e8be4640 1 --2- 172.21.15.31:0/4251556675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e41083d0 0x7f45e41087e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:09:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 -- 172.21.15.31:0/4251556675 shutdown_connections 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 --2- 172.21.15.31:0/4251556675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4108d20 0x7f45e410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 --2- 172.21.15.31:0/4251556675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e41083d0 0x7f45e41087e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 --2- 172.21.15.31:0/4251556675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e4102a90 0x7f45e4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 -- 172.21.15.31:0/4251556675 >> 172.21.15.31:0/4251556675 conn(0x7f45e4079750 msgr2=0x7f45e4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 -- 172.21.15.31:0/4251556675 shutdown_connections 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.932+0000 7f45e8be4640 1 -- 172.21.15.31:0/4251556675 wait complete. 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 Processor -- start 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 -- start start 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4102a90 0x7f45e419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e41083d0 0x7f45e419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 0x7f45e419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4112990 con 0x7f45e4108d20 2024-09-21T06:09:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4112810 con 0x7f45e4102a90 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e8be4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45e4112b10 con 0x7f45e41083d0 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 0x7f45e419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 0x7f45e419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51132/0 (socket says 172.21.15.31:51132) 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e2d9d640 1 -- 172.21.15.31:0/1246782128 learned_addr learned my addr 172.21.15.31:0/1246782128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.933+0000 7f45e259c640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4102a90 0x7f45e419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e1d9b640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e41083d0 0x7f45e419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e2d9d640 1 -- 172.21.15.31:0/1246782128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e41083d0 msgr2=0x7f45e419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e2d9d640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e41083d0 0x7f45e419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e2d9d640 1 -- 172.21.15.31:0/1246782128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4102a90 msgr2=0x7f45e419b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e2d9d640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4102a90 0x7f45e419b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e2d9d640 1 -- 172.21.15.31:0/1246782128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e41a65c0 con 0x7f45e4108d20 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e259c640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4102a90 0x7f45e419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e1d9b640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e41083d0 0x7f45e419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e2d9d640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 0x7f45e419bd80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e970 tx=0x7f45dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00ccc0 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45dc00ce60 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e41a68b0 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc010430 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.934+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e41a6d70 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.935+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e4103d10 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.937+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f45dc004ff0 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.938+0000 7f45db7fe640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45d00b6560 0x7f45d00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.938+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f45dc0e7400 con 0x7f45e4108d20 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.938+0000 7f45e259c640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45d00b6560 0x7f45d00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.938+0000 7f45e259c640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45d00b6560 0x7f45d00b8a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45d4005c40 tx=0x7f45d400a590 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:08.942+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45dc0ae060 con 0x7f45e4108d20 2024-09-21T06:09:09.099 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:09 smithi107 ceph-mon[34380]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:08 smithi107 podman[63242]: 2024-09-21 06:09:08.837906392 +0000 UTC m=+0.109102912 container create d00d1338772c0ec2a6c65ff021c0e09658847a677bde4556c2545a7e8692363f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:08 smithi107 podman[63242]: 2024-09-21 06:09:08.744642524 +0000 UTC m=+0.015839008 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:09 smithi107 podman[63242]: 2024-09-21 06:09:09.023416418 +0000 UTC m=+0.294612897 container init d00d1338772c0ec2a6c65ff021c0e09658847a677bde4556c2545a7e8692363f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:09 smithi107 podman[63242]: 2024-09-21 06:09:09.026097383 +0000 UTC m=+0.297293869 container start d00d1338772c0ec2a6c65ff021c0e09658847a677bde4556c2545a7e8692363f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:09 smithi107 ceph-mds[63262]: -- 172.21.15.107:0/3823664267 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a594ffd860 con 0x55a5991a5180 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:09 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e[63258]: starting mds.e at 2024-09-21T06:09:09.100 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:09 smithi107 bash[63242]: d00d1338772c0ec2a6c65ff021c0e09658847a677bde4556c2545a7e8692363f 2024-09-21T06:09:09.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:09 smithi031 ceph-mon[31237]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.172+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45e402e640 con 0x7f45e4108d20 2024-09-21T06:09:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.172+0000 7f45db7fe640 1 -- 172.21.15.31:0/1246782128 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f45dc0b2440 con 0x7f45e4108d20 2024-09-21T06:09:09.174 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:09.175 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"btime":"2024-09-21T06:09:09:065342+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T06:09:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45d00b6560 msgr2=0x7f45d00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45d00b6560 0x7f45d00b8a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45d4005c40 tx=0x7f45d400a590 comp rx=0 tx=0).stop 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 msgr2=0x7f45e419bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 0x7f45e419bd80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00e970 tx=0x7f45dc00ed70 comp rx=0 tx=0).stop 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 shutdown_connections 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45d00b6560 0x7f45d00b8a70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45e4108d20 0x7f45e419bd80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45e41083d0 0x7f45e419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 --2- 172.21.15.31:0/1246782128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e4102a90 0x7f45e419b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 >> 172.21.15.31:0/1246782128 conn(0x7f45e4079750 msgr2=0x7f45e4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 shutdown_connections 2024-09-21T06:09:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:09.175+0000 7f45e8be4640 1 -- 172.21.15.31:0/1246782128 wait complete. 2024-09-21T06:09:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:09 smithi110 ceph-mon[34550]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:09.539 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:09:09 smithi107 systemd[1]: Started Ceph mds.e for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6834/306946935,v1:172.21.15.107:6835/306946935] up:boot 2024-09-21T06:09:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: fsmap 5 up:standby 2024-09-21T06:09:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:09:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: Deploying daemon mds.f on smithi110 2024-09-21T06:09:10.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:10 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1246782128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:10.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6834/306946935,v1:172.21.15.107:6835/306946935] up:boot 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: fsmap 5 up:standby 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: Deploying daemon mds.f on smithi110 2024-09-21T06:09:10.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:10 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1246782128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6834/306946935,v1:172.21.15.107:6835/306946935] up:boot 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: fsmap 5 up:standby 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: Deploying daemon mds.f on smithi110 2024-09-21T06:09:10.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:10 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1246782128' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:11.177 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:10 smithi110 systemd[1]: Starting Ceph mds.f for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:11.433 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:11 smithi110 ceph-mon[34550]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:11.433 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 podman[63627]: 2024-09-21 06:09:11.112722745 +0000 UTC m=+0.015748326 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:11.433 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 podman[63627]: 2024-09-21 06:09:11.221749347 +0000 UTC m=+0.124774929 container create 4c4fe3b56612acdd2fd7e02e0af2e9fac7e595c3ab12f1e5c5ab66de5659bd15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:09:11.434 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 podman[63627]: 2024-09-21 06:09:11.432284159 +0000 UTC m=+0.335309740 container init 4c4fe3b56612acdd2fd7e02e0af2e9fac7e595c3ab12f1e5c5ab66de5659bd15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T06:09:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:11 smithi107 ceph-mon[34380]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:11.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:11 smithi031 ceph-mon[31237]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:11.778 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 podman[63627]: 2024-09-21 06:09:11.43528502 +0000 UTC m=+0.338310604 container start 4c4fe3b56612acdd2fd7e02e0af2e9fac7e595c3ab12f1e5c5ab66de5659bd15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-21T06:09:11.778 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 ceph-mds[63646]: -- 172.21.15.110:0/1723342101 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad4c267860 con 0x55ad5040f180 2024-09-21T06:09:11.779 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f[63642]: starting mds.f at 2024-09-21T06:09:11.779 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 bash[63627]: 4c4fe3b56612acdd2fd7e02e0af2e9fac7e595c3ab12f1e5c5ab66de5659bd15 2024-09-21T06:09:11.779 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:09:11 smithi110 systemd[1]: Started Ceph mds.f for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:12.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:12.722 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: Deploying daemon mds.c on smithi110 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.110:6834/1733260219,v1:172.21.15.110:6835/1733260219] up:boot 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: fsmap 6 up:standby 2024-09-21T06:09:12.723 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:12 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:12.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: Deploying daemon mds.c on smithi110 2024-09-21T06:09:12.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.110:6834/1733260219,v1:172.21.15.110:6835/1733260219] up:boot 2024-09-21T06:09:12.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: fsmap 6 up:standby 2024-09-21T06:09:12.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:12 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 -- 172.21.15.31:0/1157739978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b8107b70 msgr2=0x7f24b8107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1157739978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b8107b70 0x7f24b8107f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009900 tx=0x7f24a002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 -- 172.21.15.31:0/1157739978 shutdown_connections 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1157739978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81084c0 0x7f24b810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1157739978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b8107b70 0x7f24b8107f80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1157739978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8102230 0x7f24b8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 -- 172.21.15.31:0/1157739978 >> 172.21.15.31:0/1157739978 conn(0x7f24b8079e40 msgr2=0x7f24b807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:12.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 -- 172.21.15.31:0/1157739978 shutdown_connections 2024-09-21T06:09:12.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.812+0000 7f24bcb97640 1 -- 172.21.15.31:0/1157739978 wait complete. 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 Processor -- start 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 -- start start 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 0x7f24b81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8107b70 0x7f24b8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 0x7f24b819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8112a00 con 0x7f24b81084c0 2024-09-21T06:09:12.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8112880 con 0x7f24b8102230 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24bcb97640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8112b80 con 0x7f24b8107b70 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 0x7f24b819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24b659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 0x7f24b81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24b5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8107b70 0x7f24b8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24b659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 0x7f24b81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33134/0 (socket says 172.21.15.31:33134) 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24b659c640 1 -- 172.21.15.31:0/1253164162 learned_addr learned my addr 172.21.15.31:0/1253164162 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.813+0000 7f24b6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 0x7f24b819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43172/0 (socket says 172.21.15.31:43172) 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b659c640 1 -- 172.21.15.31:0/1253164162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8107b70 msgr2=0x7f24b8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b659c640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8107b70 0x7f24b8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b659c640 1 -- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 msgr2=0x7f24b819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b659c640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 0x7f24b819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b659c640 1 -- 172.21.15.31:0/1253164162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b819f360 con 0x7f24b8102230 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b5d9b640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8107b70 0x7f24b8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b6d9d640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 0x7f24b819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24b659c640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 0x7f24b81993f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24a800ea80 tx=0x7f24a800ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a800ce40 con 0x7f24b8102230 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b819f650 con 0x7f24b8102230 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a8005020 con 0x7f24b8102230 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80116d0 con 0x7f24b8102230 2024-09-21T06:09:12.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.814+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b810f6b0 con 0x7f24b8102230 2024-09-21T06:09:12.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.815+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b8102640 con 0x7f24b8102230 2024-09-21T06:09:12.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.817+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f24a80054d0 con 0x7f24b8102230 2024-09-21T06:09:12.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.818+0000 7f24af7fe640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24a40b6560 0x7f24a40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:12.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.818+0000 7f24b5d9b640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24a40b6560 0x7f24a40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:12.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.818+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f24a80e7630 con 0x7f24b8102230 2024-09-21T06:09:12.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.819+0000 7f24b5d9b640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24a40b6560 0x7f24a40b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009230 tx=0x7f24a0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:12.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:12.821+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24a8016030 con 0x7f24b8102230 2024-09-21T06:09:12.924 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: Deploying daemon mds.c on smithi110 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.110:6834/1733260219,v1:172.21.15.110:6835/1733260219] up:boot 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: fsmap 6 up:standby 2024-09-21T06:09:12.925 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:12 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:09:13.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.052+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24b819a520 con 0x7f24b8102230 2024-09-21T06:09:13.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.052+0000 7f24af7fe640 1 -- 172.21.15.31:0/1253164162 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7f24b819a520 con 0x7f24b8102230 2024-09-21T06:09:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:13.055 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":7,"btime":"2024-09-21T06:09:12:175892+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T06:09:13.056 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 7 2024-09-21T06:09:13.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24a40b6560 msgr2=0x7f24a40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24a40b6560 0x7f24a40b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009230 tx=0x7f24a0031400 comp rx=0 tx=0).stop 2024-09-21T06:09:13.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 msgr2=0x7f24b81993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 0x7f24b81993f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24a800ea80 tx=0x7f24a800ef40 comp rx=0 tx=0).stop 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 shutdown_connections 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24a40b6560 0x7f24a40b8a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24b81084c0 0x7f24b819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24b8107b70 0x7f24b8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 --2- 172.21.15.31:0/1253164162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b8102230 0x7f24b81993f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 >> 172.21.15.31:0/1253164162 conn(0x7f24b8079e40 msgr2=0x7f24b8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 shutdown_connections 2024-09-21T06:09:13.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:13.056+0000 7f24bcb97640 1 -- 172.21.15.31:0/1253164162 wait complete. 2024-09-21T06:09:14.028 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:09:13 smithi110 systemd[1]: Started Ceph mds.c for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1253164162' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:14.327 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:14 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:14.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1253164162' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:14.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:14.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:14.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:14 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:14.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1253164162' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:14.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:14.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:14.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:14.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:14 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:15.222 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:15 smithi107 ceph-mon[34380]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:15.222 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:15 smithi107 ceph-mon[34380]: Deploying daemon mds.h on smithi107 2024-09-21T06:09:15.222 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:15 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.110:6836/969332587,v1:172.21.15.110:6837/969332587] up:boot 2024-09-21T06:09:15.223 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:15 smithi107 ceph-mon[34380]: fsmap 7 up:standby 2024-09-21T06:09:15.223 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:15 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:09:15.301 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:15 smithi031 ceph-mon[31237]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:15.301 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:15 smithi031 ceph-mon[31237]: Deploying daemon mds.h on smithi107 2024-09-21T06:09:15.301 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:15 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.110:6836/969332587,v1:172.21.15.110:6837/969332587] up:boot 2024-09-21T06:09:15.302 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:15 smithi031 ceph-mon[31237]: fsmap 7 up:standby 2024-09-21T06:09:15.302 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:15 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:09:15.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:15 smithi110 ceph-mon[34550]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:15.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:15 smithi110 ceph-mon[34550]: Deploying daemon mds.h on smithi107 2024-09-21T06:09:15.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:15 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.110:6836/969332587,v1:172.21.15.110:6837/969332587] up:boot 2024-09-21T06:09:15.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:15 smithi110 ceph-mon[34550]: fsmap 7 up:standby 2024-09-21T06:09:15.430 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:15 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:09:15.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:15.790 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:15 smithi107 systemd[1]: Starting Ceph mds.h for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:16.053 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:16.160 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:15 smithi107 podman[63512]: 2024-09-21 06:09:15.881974467 +0000 UTC m=+0.110210594 container create c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T06:09:16.161 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:15 smithi107 podman[63512]: 2024-09-21 06:09:15.7874863 +0000 UTC m=+0.015722434 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:16.161 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:16 smithi107 podman[63512]: 2024-09-21 06:09:16.159381098 +0000 UTC m=+0.387617226 container init c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T06:09:16.506 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:16 smithi107 podman[63512]: 2024-09-21 06:09:16.163118634 +0000 UTC m=+0.391354762 container start c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T06:09:16.506 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:16 smithi107 ceph-mds[63532]: -- 172.21.15.107:0/1536303427 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560f3333c20 con 0x5560f74db600 2024-09-21T06:09:16.506 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h[63527]: starting mds.h at 2024-09-21T06:09:16.506 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:16 smithi107 bash[63512]: c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 2024-09-21T06:09:16.506 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:09:16 smithi107 systemd[1]: Started Ceph mds.h for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 -- 172.21.15.31:0/3073209776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30102890 msgr2=0x7f2f30112520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 --2- 172.21.15.31:0/3073209776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30102890 0x7f2f30112520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009ca0 tx=0x7f2f2802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 -- 172.21.15.31:0/3073209776 shutdown_connections 2024-09-21T06:09:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 --2- 172.21.15.31:0/3073209776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30102890 0x7f2f30112520 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 --2- 172.21.15.31:0/3073209776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f30101d70 0x7f2f30102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 --2- 172.21.15.31:0/3073209776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f30104660 0x7f2f30104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 -- 172.21.15.31:0/3073209776 >> 172.21.15.31:0/3073209776 conn(0x7f2f30077c60 msgr2=0x7f2f300780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:16.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 -- 172.21.15.31:0/3073209776 shutdown_connections 2024-09-21T06:09:16.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 -- 172.21.15.31:0/3073209776 wait complete. 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.572+0000 7f2f37e88640 1 Processor -- start 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 -- start start 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f30101d70 0x7f2f3019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f30102890 0x7f2f3019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 0x7f2f301a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30115c90 con 0x7f2f30104660 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30115b10 con 0x7f2f30101d70 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f37e88640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f30115e10 con 0x7f2f30102890 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 0x7f2f301a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 0x7f2f301a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43180/0 (socket says 172.21.15.31:43180) 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 -- 172.21.15.31:0/2753196275 learned_addr learned my addr 172.21.15.31:0/2753196275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f35c24640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f30101d70 0x7f2f3019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f35423640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f30102890 0x7f2f3019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 -- 172.21.15.31:0/2753196275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f30102890 msgr2=0x7f2f3019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f30102890 0x7f2f3019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 -- 172.21.15.31:0/2753196275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f30101d70 msgr2=0x7f2f3019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f30101d70 0x7f2f3019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.573+0000 7f2f36425640 1 -- 172.21.15.31:0/2753196275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f301a2610 con 0x7f2f30104660 2024-09-21T06:09:16.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.574+0000 7f2f36425640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 0x7f2f301a1ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f280098d0 tx=0x7f2f280317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.574+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28003b70 con 0x7f2f30104660 2024-09-21T06:09:16.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.574+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f301a28a0 con 0x7f2f30104660 2024-09-21T06:09:16.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.574+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f28003d10 con 0x7f2f30104660 2024-09-21T06:09:16.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.574+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f301a9060 con 0x7f2f30104660 2024-09-21T06:09:16.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.574+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2802fa20 con 0x7f2f30104660 2024-09-21T06:09:16.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.575+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f30102180 con 0x7f2f30104660 2024-09-21T06:09:16.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.577+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2f2802fcc0 con 0x7f2f30104660 2024-09-21T06:09:16.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.577+0000 7f2f26ffd640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f1c0b6560 0x7f2f1c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.578+0000 7f2f35c24640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f1c0b6560 0x7f2f1c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.578+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f2f281081d0 con 0x7f2f30104660 2024-09-21T06:09:16.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.578+0000 7f2f35c24640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f1c0b6560 0x7f2f1c0b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bb60 tx=0x7f2f2c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.582+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f280cee30 con 0x7f2f30104660 2024-09-21T06:09:16.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.812+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f3019d7d0 con 0x7f2f30104660 2024-09-21T06:09:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.813+0000 7f2f26ffd640 1 -- 172.21.15.31:0/2753196275 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f2f280d3210 con 0x7f2f30104660 2024-09-21T06:09:16.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:16.815 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"btime":"2024-09-21T06:09:14:034310+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T06:09:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2024-09-21T06:09:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.815+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f1c0b6560 msgr2=0x7f2f1c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.815+0000 7f2f37e88640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f1c0b6560 0x7f2f1c0b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bb60 tx=0x7f2f2c00b390 comp rx=0 tx=0).stop 2024-09-21T06:09:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.815+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 msgr2=0x7f2f301a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.815+0000 7f2f37e88640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 0x7f2f301a1ee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f280098d0 tx=0x7f2f280317b0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 shutdown_connections 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f1c0b6560 0x7f2f1c0b8a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f30104660 0x7f2f301a1ee0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f30102890 0x7f2f3019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 --2- 172.21.15.31:0/2753196275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f30101d70 0x7f2f3019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 >> 172.21.15.31:0/2753196275 conn(0x7f2f30077c60 msgr2=0x7f2f3010c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 shutdown_connections 2024-09-21T06:09:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:16.816+0000 7f2f37e88640 1 -- 172.21.15.31:0/2753196275 wait complete. 2024-09-21T06:09:17.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:17.508 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: Deploying daemon mds.i on smithi110 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2753196275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6836/2795950298,v1:172.21.15.107:6837/2795950298] up:boot 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: fsmap 8 up:standby 2024-09-21T06:09:17.509 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:17 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: Deploying daemon mds.i on smithi110 2024-09-21T06:09:17.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2753196275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:17.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6836/2795950298,v1:172.21.15.107:6837/2795950298] up:boot 2024-09-21T06:09:17.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: fsmap 8 up:standby 2024-09-21T06:09:17.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:17 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:09:17.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:17.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: Deploying daemon mds.i on smithi110 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2753196275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6836/2795950298,v1:172.21.15.107:6837/2795950298] up:boot 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: fsmap 8 up:standby 2024-09-21T06:09:17.712 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:17 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:09:18.283 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 systemd[1]: Starting Ceph mds.i for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:18.659 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 podman[64184]: 2024-09-21 06:09:18.288418901 +0000 UTC m=+0.015945943 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:18.660 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 podman[64184]: 2024-09-21 06:09:18.40407363 +0000 UTC m=+0.131600706 container create 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T06:09:18.660 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 podman[64184]: 2024-09-21 06:09:18.623128222 +0000 UTC m=+0.350655257 container init 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:18.660 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 podman[64184]: 2024-09-21 06:09:18.625900314 +0000 UTC m=+0.353427360 container start 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:09:19.028 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 ceph-mds[64203]: -- 172.21.15.110:0/3919010874 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d16eab860 con 0x564d1b053a80 2024-09-21T06:09:19.028 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i[64199]: starting mds.i at 2024-09-21T06:09:19.028 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 bash[64184]: 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 2024-09-21T06:09:19.028 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:09:18 smithi110 systemd[1]: Started Ceph mds.i for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:19.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:19.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.494 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:19.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:19.495 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:19 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:19.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:19.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:19.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:19 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:19.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:19 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:20.366 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:20 smithi031 ceph-mon[31237]: Deploying daemon mds.g on smithi031 2024-09-21T06:09:20.366 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:20 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.110:6838/1863570857,v1:172.21.15.110:6839/1863570857] up:boot 2024-09-21T06:09:20.367 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:20 smithi031 ceph-mon[31237]: fsmap 9 up:standby 2024-09-21T06:09:20.367 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:20 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:09:20.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:20 smithi110 ceph-mon[34550]: Deploying daemon mds.g on smithi031 2024-09-21T06:09:20.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:20 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.110:6838/1863570857,v1:172.21.15.110:6839/1863570857] up:boot 2024-09-21T06:09:20.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:20 smithi110 ceph-mon[34550]: fsmap 9 up:standby 2024-09-21T06:09:20.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:20 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:09:20.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:20 smithi107 ceph-mon[34380]: Deploying daemon mds.g on smithi031 2024-09-21T06:09:20.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:20 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.110:6838/1863570857,v1:172.21.15.110:6839/1863570857] up:boot 2024-09-21T06:09:20.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:20 smithi107 ceph-mon[34380]: fsmap 9 up:standby 2024-09-21T06:09:20.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:20 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 -- 172.21.15.31:0/4203072280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0102650 msgr2=0x7f88c0102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 --2- 172.21.15.31:0/4203072280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0102650 0x7f88c0102a60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009900 tx=0x7f88bc02a480 comp rx=0 tx=0).stop 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 -- 172.21.15.31:0/4203072280 shutdown_connections 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 --2- 172.21.15.31:0/4203072280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88c0110200 0x7f88c0112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 --2- 172.21.15.31:0/4203072280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 0x7f88c010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 --2- 172.21.15.31:0/4203072280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0102650 0x7f88c0102a60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 -- 172.21.15.31:0/4203072280 >> 172.21.15.31:0/4203072280 conn(0x7f88c0079770 msgr2=0x7f88c0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 -- 172.21.15.31:0/4203072280 shutdown_connections 2024-09-21T06:09:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 -- 172.21.15.31:0/4203072280 wait complete. 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.847+0000 7f88c7c71640 1 Processor -- start 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 -- start start 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88c0102650 0x7f88c019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 0x7f88c019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 0x7f88c01a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0115cf0 con 0x7f88c0110200 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0115b70 con 0x7f88c0102fa0 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c7c71640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0115e70 con 0x7f88c0102650 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 0x7f88c01a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 0x7f88c01a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60714/0 (socket says 172.21.15.31:60714) 2024-09-21T06:09:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 -- 172.21.15.31:0/2777921152 learned_addr learned my addr 172.21.15.31:0/2777921152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c5a0d640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88c0102650 0x7f88c019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c520c640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 0x7f88c019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 -- 172.21.15.31:0/2777921152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88c0102650 msgr2=0x7f88c019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88c0102650 0x7f88c019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 -- 172.21.15.31:0/2777921152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 msgr2=0x7f88c019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 0x7f88c019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c620e640 1 -- 172.21.15.31:0/2777921152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c01a2610 con 0x7f88c0110200 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.848+0000 7f88c520c640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 0x7f88c019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.849+0000 7f88c620e640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 0x7f88c01a1ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88b800eaa0 tx=0x7f88b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.849+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b800cdb0 con 0x7f88c0110200 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.849+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b8005020 con 0x7f88c0110200 2024-09-21T06:09:20.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.849+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c01a2900 con 0x7f88c0110200 2024-09-21T06:09:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.849+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80116d0 con 0x7f88c0110200 2024-09-21T06:09:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.849+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c01a9060 con 0x7f88c0110200 2024-09-21T06:09:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.850+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c006c260 con 0x7f88c0110200 2024-09-21T06:09:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.851+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f88b80051d0 con 0x7f88c0110200 2024-09-21T06:09:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.852+0000 7f88b6ffd640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88ac0b65e0 0x7f88ac0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.852+0000 7f88c5a0d640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88ac0b65e0 0x7f88ac0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.852+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f88b80e70e0 con 0x7f88c0110200 2024-09-21T06:09:20.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.853+0000 7f88c5a0d640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88ac0b65e0 0x7f88ac0b8af0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88bc031440 tx=0x7f88bc009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:20.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:20.857+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88b8016030 con 0x7f88c0110200 2024-09-21T06:09:21.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.089+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88c019d7d0 con 0x7f88c0110200 2024-09-21T06:09:21.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.090+0000 7f88b6ffd640 1 -- 172.21.15.31:0/2777921152 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8046 (secure 0 0 0) 0x7f88b80add40 con 0x7f88c0110200 2024-09-21T06:09:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:21.092 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"btime":"2024-09-21T06:09:19:307094+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24686,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T06:09:21.092 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 10 2024-09-21T06:09:21.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88ac0b65e0 msgr2=0x7f88ac0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:21.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88ac0b65e0 0x7f88ac0b8af0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88bc031440 tx=0x7f88bc009560 comp rx=0 tx=0).stop 2024-09-21T06:09:21.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 msgr2=0x7f88c01a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:21.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 0x7f88c01a1ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88b800eaa0 tx=0x7f88b800ef60 comp rx=0 tx=0).stop 2024-09-21T06:09:21.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 shutdown_connections 2024-09-21T06:09:21.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88ac0b65e0 0x7f88ac0b8af0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0110200 0x7f88c01a1ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88c0102fa0 0x7f88c019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 --2- 172.21.15.31:0/2777921152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88c0102650 0x7f88c019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 >> 172.21.15.31:0/2777921152 conn(0x7f88c0079770 msgr2=0x7f88c0108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 shutdown_connections 2024-09-21T06:09:21.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:21.092+0000 7f88c7c71640 1 -- 172.21.15.31:0/2777921152 wait complete. 2024-09-21T06:09:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:21 smithi031 ceph-mon[31237]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:21 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2777921152' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:21 smithi110 ceph-mon[34550]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:21 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2777921152' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:21 smithi107 ceph-mon[34380]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:21 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2777921152' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:22.135 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:09:21 smithi031 systemd[1]: Started Ceph mds.g for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: Deploying daemon mds.a on smithi031 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.31:6838/3970721911,v1:172.21.15.31:6839/3970721911] up:boot 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: fsmap 10 up:standby 2024-09-21T06:09:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:22 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:09:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: Deploying daemon mds.a on smithi031 2024-09-21T06:09:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.31:6838/3970721911,v1:172.21.15.31:6839/3970721911] up:boot 2024-09-21T06:09:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: fsmap 10 up:standby 2024-09-21T06:09:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:22 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:09:23.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:23.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: Deploying daemon mds.a on smithi031 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.31:6838/3970721911,v1:172.21.15.31:6839/3970721911] up:boot 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: fsmap 10 up:standby 2024-09-21T06:09:23.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:22 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:09:23.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:24.085 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:23 smithi031 systemd[1]: Starting Ceph mds.a for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:24.336 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 podman[74451]: 2024-09-21 06:09:23.986882734 +0000 UTC m=+0.018170972 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:24.336 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 podman[74451]: 2024-09-21 06:09:24.092788622 +0000 UTC m=+0.124076845 container create 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T06:09:24.336 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 podman[74451]: 2024-09-21 06:09:24.29525757 +0000 UTC m=+0.326545802 container init 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:09:24.336 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 podman[74451]: 2024-09-21 06:09:24.298557777 +0000 UTC m=+0.329845997 container start 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T06:09:24.336 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mds[74494]: -- 172.21.15.31:0/1320952974 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637ec4e3860 con 0x5637f068ba80 2024-09-21T06:09:24.336 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a[74478]: starting mds.a at 2024-09-21T06:09:24.459 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:24.597 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 bash[74451]: 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 2024-09-21T06:09:24.597 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:09:24 smithi031 systemd[1]: Started Ceph mds.a for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 -- 172.21.15.31:0/1448692340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8104a30 msgr2=0x7f93a8104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 --2- 172.21.15.31:0/1448692340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8104a30 0x7f93a8104e40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9398009900 tx=0x7f939802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 -- 172.21.15.31:0/1448692340 shutdown_connections 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 --2- 172.21.15.31:0/1448692340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93a810de90 0x7f93a8110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 --2- 172.21.15.31:0/1448692340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93a8105380 0x7f93a810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 --2- 172.21.15.31:0/1448692340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a8104a30 0x7f93a8104e40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 -- 172.21.15.31:0/1448692340 >> 172.21.15.31:0/1448692340 conn(0x7f93a8079770 msgr2=0x7f93a8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:24.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 -- 172.21.15.31:0/1448692340 shutdown_connections 2024-09-21T06:09:24.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 -- 172.21.15.31:0/1448692340 wait complete. 2024-09-21T06:09:24.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.949+0000 7f93ad361640 1 Processor -- start 2024-09-21T06:09:24.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 -- start start 2024-09-21T06:09:24.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93a8104a30 0x7f93a8102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93a8105380 0x7f93a8102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 0x7f93a81031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8113af0 con 0x7f93a810de90 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8113970 con 0x7f93a8105380 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93ad361640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93a8113c70 con 0x7f93a8104a30 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 0x7f93a81031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 0x7f93a81031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60790/0 (socket says 172.21.15.31:60790) 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 -- 172.21.15.31:0/4280142765 learned_addr learned my addr 172.21.15.31:0/4280142765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a67fc640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93a8105380 0x7f93a8102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a6ffd640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93a8104a30 0x7f93a8102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 -- 172.21.15.31:0/4280142765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93a8104a30 msgr2=0x7f93a8102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93a8104a30 0x7f93a8102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 -- 172.21.15.31:0/4280142765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93a8105380 msgr2=0x7f93a8102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93a8105380 0x7f93a8102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.950+0000 7f93a77fe640 1 -- 172.21.15.31:0/4280142765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a81a87b0 con 0x7f93a810de90 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.951+0000 7f93a77fe640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 0x7f93a81031f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93a000cce0 tx=0x7f93a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.951+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a000f450 con 0x7f93a810de90 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.951+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a81a8aa0 con 0x7f93a810de90 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.951+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a000faa0 con 0x7f93a810de90 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.951+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0018b70 con 0x7f93a810de90 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.951+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a81a8fe0 con 0x7f93a810de90 2024-09-21T06:09:24.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.952+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a806c260 con 0x7f93a810de90 2024-09-21T06:09:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.953+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f93a0018d10 con 0x7f93a810de90 2024-09-21T06:09:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.954+0000 7f938ffff640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93900b6560 0x7f93900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.954+0000 7f93a6ffd640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93900b6560 0x7f93900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.954+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f93a00e7420 con 0x7f93a810de90 2024-09-21T06:09:24.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.955+0000 7f93a6ffd640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93900b6560 0x7f93900b8a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f9398031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:24.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:24.959+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93a0014030 con 0x7f93a810de90 2024-09-21T06:09:25.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:25.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:25.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:24 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.193+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93a8108a00 con 0x7f93a810de90 2024-09-21T06:09:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.194+0000 7f938ffff640 1 -- 172.21.15.31:0/4280142765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9698 (secure 0 0 0) 0x7f93a00adfe0 con 0x7f93a810de90 2024-09-21T06:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"btime":"2024-09-21T06:09:24:979044+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24686,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T06:09:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2024-09-21T06:09:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93900b6560 msgr2=0x7f93900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93900b6560 0x7f93900b8a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f9398031400 comp rx=0 tx=0).stop 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 msgr2=0x7f93a81031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 0x7f93a81031f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93a000cce0 tx=0x7f93a000a620 comp rx=0 tx=0).stop 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 shutdown_connections 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93900b6560 0x7f93900b8a70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93a810de90 0x7f93a81031f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93a8105380 0x7f93a8102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.196+0000 7f93ad361640 1 --2- 172.21.15.31:0/4280142765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93a8104a30 0x7f93a8102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.197+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 >> 172.21.15.31:0/4280142765 conn(0x7f93a8079770 msgr2=0x7f93a81007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.197+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 shutdown_connections 2024-09-21T06:09:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:25.197+0000 7f93ad361640 1 -- 172.21.15.31:0/4280142765 wait complete. 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:25.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:24 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:25.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:09:25.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:09:25.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:24 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:26.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:25 smithi031 ceph-mon[31237]: Deploying daemon mds.b on smithi107 2024-09-21T06:09:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:25 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.31:6840/4177463092,v1:172.21.15.31:6841/4177463092] up:boot 2024-09-21T06:09:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:25 smithi031 ceph-mon[31237]: fsmap 11 up:standby 2024-09-21T06:09:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:25 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:09:26.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:25 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4280142765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:26.246 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:25 smithi107 ceph-mon[34380]: Deploying daemon mds.b on smithi107 2024-09-21T06:09:26.246 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:25 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.31:6840/4177463092,v1:172.21.15.31:6841/4177463092] up:boot 2024-09-21T06:09:26.246 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:25 smithi107 ceph-mon[34380]: fsmap 11 up:standby 2024-09-21T06:09:26.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:25 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:09:26.247 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:25 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4280142765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:26.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:25 smithi110 ceph-mon[34550]: Deploying daemon mds.b on smithi107 2024-09-21T06:09:26.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:25 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.31:6840/4177463092,v1:172.21.15.31:6841/4177463092] up:boot 2024-09-21T06:09:26.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:25 smithi110 ceph-mon[34550]: fsmap 11 up:standby 2024-09-21T06:09:26.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:25 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:09:26.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:25 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4280142765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:26.539 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:26 smithi107 systemd[1]: Starting Ceph mds.b for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:09:26.888 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:26 smithi107 podman[63791]: 2024-09-21 06:09:26.538686423 +0000 UTC m=+0.015749727 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:26.888 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:26 smithi107 podman[63791]: 2024-09-21 06:09:26.64332517 +0000 UTC m=+0.120388457 container create 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T06:09:27.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:26 smithi110 ceph-mon[34550]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:27.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:26 smithi107 ceph-mon[34380]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:27.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:26 smithi107 podman[63791]: 2024-09-21 06:09:26.987211815 +0000 UTC m=+0.464275102 container init 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:09:27.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:26 smithi107 podman[63791]: 2024-09-21 06:09:26.989877521 +0000 UTC m=+0.466940809 container start 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:09:27.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:27 smithi107 ceph-mds[63810]: -- 172.21.15.107:0/2000921222 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620bbb9b860 con 0x5620bfd43180 2024-09-21T06:09:27.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: starting mds.b at 2024-09-21T06:09:27.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:27 smithi107 bash[63791]: 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 2024-09-21T06:09:27.290 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:27 smithi107 systemd[1]: Started Ceph mds.b for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:09:27.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:26 smithi031 ceph-mon[31237]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:27.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:28.351 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:28.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] up:boot 2024-09-21T06:09:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: fsmap 12 up:standby 2024-09-21T06:09:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:09:28.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] up:boot 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: fsmap 12 up:standby 2024-09-21T06:09:28.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:09:28.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:28.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:28.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] up:boot 2024-09-21T06:09:28.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: fsmap 12 up:standby 2024-09-21T06:09:28.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:09:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 -- 172.21.15.31:0/3308346816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed01094a0 msgr2=0x7fbed01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 --2- 172.21.15.31:0/3308346816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed01094a0 0x7fbed01098b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009900 tx=0x7fbec402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 -- 172.21.15.31:0/3308346816 shutdown_connections 2024-09-21T06:09:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 --2- 172.21.15.31:0/3308346816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0109df0 0x7fbed0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 --2- 172.21.15.31:0/3308346816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed01094a0 0x7fbed01098b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 --2- 172.21.15.31:0/3308346816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed0103b60 0x7fbed0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 -- 172.21.15.31:0/3308346816 >> 172.21.15.31:0/3308346816 conn(0x7fbed007b850 msgr2=0x7fbed007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 -- 172.21.15.31:0/3308346816 shutdown_connections 2024-09-21T06:09:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 -- 172.21.15.31:0/3308346816 wait complete. 2024-09-21T06:09:28.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.851+0000 7fbed607f640 1 Processor -- start 2024-09-21T06:09:28.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 -- start start 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 0x7fbed019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed01094a0 0x7fbed019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0109df0 0x7fbed019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113ab0 con 0x7fbed0103b60 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113930 con 0x7fbed01094a0 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbed607f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113c30 con 0x7fbed0109df0 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 0x7fbed019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 0x7fbed019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60796/0 (socket says 172.21.15.31:60796) 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 -- 172.21.15.31:0/643231967 learned_addr learned my addr 172.21.15.31:0/643231967 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecffff640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0109df0 0x7fbed019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbeceffd640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed01094a0 0x7fbed019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 -- 172.21.15.31:0/643231967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0109df0 msgr2=0x7fbed019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0109df0 0x7fbed019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 -- 172.21.15.31:0/643231967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed01094a0 msgr2=0x7fbed019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed01094a0 0x7fbed019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbecf7fe640 1 -- 172.21.15.31:0/643231967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed01a03f0 con 0x7fbed0103b60 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.852+0000 7fbeceffd640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed01094a0 0x7fbed019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.853+0000 7fbecf7fe640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 0x7fbed019a480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbec000e9f0 tx=0x7fbec000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.853+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec000cd80 con 0x7fbed0103b60 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.853+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed01a06e0 con 0x7fbed0103b60 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.853+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec00108e0 con 0x7fbed0103b60 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.853+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0018980 con 0x7fbed0103b60 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.853+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a6e40 con 0x7fbed0103b60 2024-09-21T06:09:28.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.854+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed0103f70 con 0x7fbed0103b60 2024-09-21T06:09:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.857+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbec0010430 con 0x7fbed0103b60 2024-09-21T06:09:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.857+0000 7fbeccff9640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbeb80b65e0 0x7fbeb80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.857+0000 7fbeceffd640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbeb80b65e0 0x7fbeb80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.857+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fbec00e7780 con 0x7fbed0103b60 2024-09-21T06:09:28.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.858+0000 7fbeceffd640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbeb80b65e0 0x7fbeb80b8af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009230 tx=0x7fbec4000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:28.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:28.862+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbec00ae3e0 con 0x7fbed0103b60 2024-09-21T06:09:29.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.112+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbed019b5b0 con 0x7fbed0103b60 2024-09-21T06:09:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.113+0000 7fbeccff9640 1 -- 172.21.15.31:0/643231967 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7fbec00b27c0 con 0x7fbed0103b60 2024-09-21T06:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"btime":"2024-09-21T06:09:27:996255+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24686,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T06:09:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2024-09-21T06:09:29.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbeb80b65e0 msgr2=0x7fbeb80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbeb80b65e0 0x7fbeb80b8af0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009230 tx=0x7fbec4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 msgr2=0x7fbed019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 0x7fbed019a480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbec000e9f0 tx=0x7fbec000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 shutdown_connections 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0109df0 0x7fbed019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbed01094a0 0x7fbed019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbeb80b65e0 0x7fbeb80b8af0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 --2- 172.21.15.31:0/643231967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed0103b60 0x7fbed019a480 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.115+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 >> 172.21.15.31:0/643231967 conn(0x7fbed007b850 msgr2=0x7fbed0105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.116+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 shutdown_connections 2024-09-21T06:09:29.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:29.116+0000 7fbed607f640 1 -- 172.21.15.31:0/643231967 wait complete. 2024-09-21T06:09:29.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:29.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:29 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/643231967' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:29 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/643231967' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:29 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/643231967' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:29.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T06:09:30.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:30 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:30.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:30 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:30.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:30 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:09:31.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 -- 172.21.15.31:0/2226882792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102650 msgr2=0x7f5f5c102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 --2- 172.21.15.31:0/2226882792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102650 0x7f5f5c102a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009900 tx=0x7f5f4c02a480 comp rx=0 tx=0).stop 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 -- 172.21.15.31:0/2226882792 shutdown_connections 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 --2- 172.21.15.31:0/2226882792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c110200 0x7f5f5c112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 --2- 172.21.15.31:0/2226882792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 --2- 172.21.15.31:0/2226882792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102650 0x7f5f5c102a60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 -- 172.21.15.31:0/2226882792 >> 172.21.15.31:0/2226882792 conn(0x7f5f5c079730 msgr2=0x7f5f5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 -- 172.21.15.31:0/2226882792 shutdown_connections 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.230+0000 7f5f611cb640 1 -- 172.21.15.31:0/2226882792 wait complete. 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 Processor -- start 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 -- start start 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c102650 0x7f5f5c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c110200 0x7f5f5c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c115cc0 con 0x7f5f5c102fa0 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c115b40 con 0x7f5f5c102650 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.231+0000 7f5f611cb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c115e40 con 0x7f5f5c110200 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53584/0 (socket says 172.21.15.31:53584) 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 -- 172.21.15.31:0/4259924538 learned_addr learned my addr 172.21.15.31:0/4259924538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5ad9d640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c102650 0x7f5f5c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5b59e640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c110200 0x7f5f5c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 -- 172.21.15.31:0/4259924538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c110200 msgr2=0x7f5f5c1a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c110200 0x7f5f5c1a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 -- 172.21.15.31:0/4259924538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c102650 msgr2=0x7f5f5c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c102650 0x7f5f5c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 -- 172.21.15.31:0/4259924538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c1a2520 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f5a59c640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c19cc10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e9f0 tx=0x7f5f4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400cd80 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f440054d0 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c1a2810 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44010690 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.232+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c1a9030 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.233+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c102a60 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.235+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f5f44004e30 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.236+0000 7f5f53fff640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f480b6590 0x7f5f480b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.236+0000 7f5f5ad9d640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f480b6590 0x7f5f480b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.236+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f5f440e75f0 con 0x7f5f5c102fa0 2024-09-21T06:09:31.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.236+0000 7f5f5ad9d640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f480b6590 0x7f5f480b8aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c005ab0 tx=0x7f5f4c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:31.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.239+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f440ae1b0 con 0x7f5f5c102fa0 2024-09-21T06:09:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:31 smithi031 ceph-mon[31237]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:31 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.437+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5f5c19d7a0 con 0x7f5f5c102fa0 2024-09-21T06:09:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.438+0000 7f5f53fff640 1 -- 172.21.15.31:0/4259924538 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7f5f5c19d7a0 con 0x7f5f5c102fa0 2024-09-21T06:09:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:31.440 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2024-09-21T06:09:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:09:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2024-09-21T06:09:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2024-09-21T06:09:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:09:31.440 INFO:teuthology.orchestra.run.smithi031.stdout:] 2024-09-21T06:09:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f480b6590 msgr2=0x7f5f480b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f480b6590 0x7f5f480b8aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c005ab0 tx=0x7f5f4c005a20 comp rx=0 tx=0).stop 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 msgr2=0x7f5f5c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c19cc10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e9f0 tx=0x7f5f4400edf0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 shutdown_connections 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c110200 0x7f5f5c1a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f480b6590 0x7f5f480b8aa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c102fa0 0x7f5f5c19cc10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 --2- 172.21.15.31:0/4259924538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c102650 0x7f5f5c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 >> 172.21.15.31:0/4259924538 conn(0x7f5f5c079730 msgr2=0x7f5f5c108cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 shutdown_connections 2024-09-21T06:09:31.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:31.440+0000 7f5f611cb640 1 -- 172.21.15.31:0/4259924538 wait complete. 2024-09-21T06:09:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:31 smithi110 ceph-mon[34550]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:31 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:31 smithi107 ceph-mon[34380]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:31 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:32.076 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:09:32.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:09:32.360 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:32.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:32 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4259924538' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:09:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:32 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4259924538' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:09:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:32 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4259924538' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:09:32.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 -- 172.21.15.31:0/2161810424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd140102a60 msgr2=0x7fd140102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 --2- 172.21.15.31:0/2161810424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd140102a60 0x7fd140102e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd130009900 tx=0x7fd13002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:32.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 -- 172.21.15.31:0/2161810424 shutdown_connections 2024-09-21T06:09:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 --2- 172.21.15.31:0/2161810424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd140108cf0 0x7fd14010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 --2- 172.21.15.31:0/2161810424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1401083a0 0x7fd1401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 --2- 172.21.15.31:0/2161810424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd140102a60 0x7fd140102e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 -- 172.21.15.31:0/2161810424 >> 172.21.15.31:0/2161810424 conn(0x7fd140079730 msgr2=0x7fd140079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 -- 172.21.15.31:0/2161810424 shutdown_connections 2024-09-21T06:09:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.821+0000 7fd144f38640 1 -- 172.21.15.31:0/2161810424 wait complete. 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.822+0000 7fd144f38640 1 Processor -- start 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.822+0000 7fd144f38640 1 -- start start 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd144f38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140102a60 0x7fd1401993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd144f38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 0x7fd140199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd144f38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd140108cf0 0x7fd14019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd144f38640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401129c0 con 0x7fd1401083a0 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd144f38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd140112840 con 0x7fd140102a60 2024-09-21T06:09:32.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd144f38640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd140112b40 con 0x7fd140108cf0 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 0x7fd140199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 0x7fd140199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53596/0 (socket says 172.21.15.31:53596) 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 -- 172.21.15.31:0/2947693782 learned_addr learned my addr 172.21.15.31:0/2947693782 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13e59c640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140102a60 0x7fd1401993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13ed9d640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd140108cf0 0x7fd14019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 -- 172.21.15.31:0/2947693782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd140108cf0 msgr2=0x7fd14019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd140108cf0 0x7fd14019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 -- 172.21.15.31:0/2947693782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140102a60 msgr2=0x7fd1401993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140102a60 0x7fd1401993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 -- 172.21.15.31:0/2947693782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14019f260 con 0x7fd1401083a0 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13e59c640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140102a60 0x7fd1401993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:32.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.823+0000 7fd13dd9b640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 0x7fd140199950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd13400c970 tx=0x7fd13400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.824+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134014070 con 0x7fd1401083a0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.824+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd134004440 con 0x7fd1401083a0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.824+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14019f550 con 0x7fd1401083a0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.824+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134003e50 con 0x7fd1401083a0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.824+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401a5d70 con 0x7fd1401083a0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.825+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd140102e70 con 0x7fd1401083a0 2024-09-21T06:09:32.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.826+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd134017070 con 0x7fd1401083a0 2024-09-21T06:09:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.827+0000 7fd12f7fe640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1280b6730 0x7fd1280b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.827+0000 7fd13e59c640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1280b6730 0x7fd1280b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:32.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.827+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fd1340e73c0 con 0x7fd1401083a0 2024-09-21T06:09:32.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.828+0000 7fd13e59c640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1280b6730 0x7fd1280b8c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd130031440 tx=0x7fd130005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:32.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:32.830+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd13401e030 con 0x7fd1401083a0 2024-09-21T06:09:33.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.018+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd14019a4e0 con 0x7fd1401083a0 2024-09-21T06:09:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.247+0000 7fd12f7fe640 1 -- 172.21.15.31:0/2947693782 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7fd14019a4e0 con 0x7fd1401083a0 2024-09-21T06:09:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1280b6730 msgr2=0x7fd1280b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1280b6730 0x7fd1280b8c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd130031440 tx=0x7fd130005d50 comp rx=0 tx=0).stop 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 msgr2=0x7fd140199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 0x7fd140199950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd13400c970 tx=0x7fd13400ce30 comp rx=0 tx=0).stop 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 shutdown_connections 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd140108cf0 0x7fd14019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1280b6730 0x7fd1280b8c40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1401083a0 0x7fd140199950 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 --2- 172.21.15.31:0/2947693782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140102a60 0x7fd1401993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 >> 172.21.15.31:0/2947693782 conn(0x7fd140079730 msgr2=0x7fd140101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 shutdown_connections 2024-09-21T06:09:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:33.249+0000 7fd144f38640 1 -- 172.21.15.31:0/2947693782 wait complete. 2024-09-21T06:09:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:33 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2947693782' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.542 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:33 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2947693782' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:09:33.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:33 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2947693782' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:09:33.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:09:34.132 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:34.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:34 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2947693782' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:09:34.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:34 smithi110 ceph-mon[34550]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:09:34.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:34 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2947693782' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:09:34.539 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:34 smithi031 ceph-mon[31237]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:09:34.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:34 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2947693782' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:09:34.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:34 smithi107 ceph-mon[34380]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:09:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 -- 172.21.15.31:0/3925065877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 msgr2=0x7f78b8103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 --2- 172.21.15.31:0/3925065877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b8103fc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009900 tx=0x7f78a802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 -- 172.21.15.31:0/3925065877 shutdown_connections 2024-09-21T06:09:34.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 --2- 172.21.15.31:0/3925065877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b8109e40 0x7f78b8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 --2- 172.21.15.31:0/3925065877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b81094f0 0x7f78b8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 --2- 172.21.15.31:0/3925065877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b8103fc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 -- 172.21.15.31:0/3925065877 >> 172.21.15.31:0/3925065877 conn(0x7f78b80ff600 msgr2=0x7f78b8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 -- 172.21.15.31:0/3925065877 shutdown_connections 2024-09-21T06:09:34.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.636+0000 7f78bdd81640 1 -- 172.21.15.31:0/3925065877 wait complete. 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.637+0000 7f78bdd81640 1 Processor -- start 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.637+0000 7f78bdd81640 1 -- start start 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78bdd81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78bdd81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b81094f0 0x7f78b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78bdd81640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b8109e40 0x7f78b819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78bdd81640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8113b60 con 0x7f78b8103bb0 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78bdd81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81139e0 con 0x7f78b81094f0 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78bdd81640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8113ce0 con 0x7f78b8109e40 2024-09-21T06:09:34.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53610/0 (socket says 172.21.15.31:53610) 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 -- 172.21.15.31:0/1154552726 learned_addr learned my addr 172.21.15.31:0/1154552726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b7fff640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b8109e40 0x7f78b819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b6ffd640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b81094f0 0x7f78b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 -- 172.21.15.31:0/1154552726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b8109e40 msgr2=0x7f78b819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b8109e40 0x7f78b819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 -- 172.21.15.31:0/1154552726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b81094f0 msgr2=0x7f78b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b81094f0 0x7f78b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 -- 172.21.15.31:0/1154552726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b81a04a0 con 0x7f78b8103bb0 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.638+0000 7f78b77fe640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b819a530 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005940 tx=0x7f78a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.639+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a803f030 con 0x7f78b8103bb0 2024-09-21T06:09:34.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.639+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a8003cb0 con 0x7f78b8103bb0 2024-09-21T06:09:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.639+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8035070 con 0x7f78b8103bb0 2024-09-21T06:09:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.639+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b81a0730 con 0x7f78b8103bb0 2024-09-21T06:09:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.639+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81a6ef0 con 0x7f78b8103bb0 2024-09-21T06:09:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.640+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b8103fc0 con 0x7f78b8103bb0 2024-09-21T06:09:34.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.641+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f78a8042030 con 0x7f78b8103bb0 2024-09-21T06:09:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.642+0000 7f78b4ff9640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78a00b65e0 0x7f78a00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.642+0000 7f78b6ffd640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78a00b65e0 0x7f78a00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.642+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7712+0+0 (secure 0 0 0) 0x7f78a8107910 con 0x7f78b8103bb0 2024-09-21T06:09:34.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.643+0000 7f78b6ffd640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78a00b65e0 0x7f78a00b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f78b8108eb0 tx=0x7f78ac00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.647+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78a80d2760 con 0x7f78b8103bb0 2024-09-21T06:09:34.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:34.837+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f78b810b680 con 0x7f78b8103bb0 2024-09-21T06:09:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.264+0000 7f78b4ff9640 1 -- 172.21.15.31:0/1154552726 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f78a80d76a0 con 0x7f78b8103bb0 2024-09-21T06:09:35.266 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78a00b65e0 msgr2=0x7f78a00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78a00b65e0 0x7f78a00b8af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f78b8108eb0 tx=0x7f78ac00b390 comp rx=0 tx=0).stop 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 msgr2=0x7f78b819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b819a530 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005940 tx=0x7f78a80042f0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 shutdown_connections 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78b8109e40 0x7f78b819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78b81094f0 0x7f78b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78a00b65e0 0x7f78a00b8af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 --2- 172.21.15.31:0/1154552726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78b8103bb0 0x7f78b819a530 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:35.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.267+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 >> 172.21.15.31:0/1154552726 conn(0x7f78b80ff600 msgr2=0x7f78b8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:35.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.268+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 shutdown_connections 2024-09-21T06:09:35.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:35.268+0000 7f78bdd81640 1 -- 172.21.15.31:0/1154552726 wait complete. 2024-09-21T06:09:35.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:35 smithi110 ceph-mon[34550]: pgmap v253: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:35.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:35 smithi110 ceph-mon[34550]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:09:35.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:35 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1154552726' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:09:35.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:35 smithi107 ceph-mon[34380]: pgmap v253: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:35.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:35 smithi107 ceph-mon[34380]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:09:35.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:35 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1154552726' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:09:35.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:35 smithi031 ceph-mon[31237]: pgmap v253: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:35.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:35 smithi031 ceph-mon[31237]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:09:35.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:35 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1154552726' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:09:35.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:09:36.178 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:36.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1154552726' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:09:36.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:36 smithi110 ceph-mon[34550]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:09:36.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1154552726' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:09:36.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:36 smithi107 ceph-mon[34380]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:09:36.572 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1154552726' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:09:36.572 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:36 smithi031 ceph-mon[31237]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:09:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 -- 172.21.15.31:0/1912181171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8630102ae0 msgr2=0x7f8630102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 --2- 172.21.15.31:0/1912181171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8630102ae0 0x7f8630102ef0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8624009900 tx=0x7f862402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:36.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 -- 172.21.15.31:0/1912181171 shutdown_connections 2024-09-21T06:09:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 --2- 172.21.15.31:0/1912181171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8630108d20 0x7f863010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 --2- 172.21.15.31:0/1912181171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86301083d0 0x7f86301087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 --2- 172.21.15.31:0/1912181171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8630102ae0 0x7f8630102ef0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 -- 172.21.15.31:0/1912181171 >> 172.21.15.31:0/1912181171 conn(0x7f8630079760 msgr2=0x7f8630079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 -- 172.21.15.31:0/1912181171 shutdown_connections 2024-09-21T06:09:36.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.666+0000 7f8637063640 1 -- 172.21.15.31:0/1912181171 wait complete. 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.667+0000 7f8637063640 1 Processor -- start 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.667+0000 7f8637063640 1 -- start start 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.667+0000 7f8637063640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8630102ae0 0x7f8630199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8637063640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 0x7f8630199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8637063640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8630108d20 0x7f863019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8637063640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8630112990 con 0x7f86301083d0 2024-09-21T06:09:36.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8637063640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8630112810 con 0x7f8630102ae0 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8637063640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8630112b10 con 0x7f8630108d20 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 0x7f8630199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8634dff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8630102ae0 0x7f8630199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 0x7f8630199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53632/0 (socket says 172.21.15.31:53632) 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 -- 172.21.15.31:0/2641955542 learned_addr learned my addr 172.21.15.31:0/2641955542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f8635600640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8630108d20 0x7f863019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 -- 172.21.15.31:0/2641955542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8630108d20 msgr2=0x7f863019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8630108d20 0x7f863019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 -- 172.21.15.31:0/2641955542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8630102ae0 msgr2=0x7f8630199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8630102ae0 0x7f8630199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 -- 172.21.15.31:0/2641955542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863019f2d0 con 0x7f86301083d0 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.668+0000 7f862ffff640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 0x7f8630199900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c970 tx=0x7f861c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:36.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.669+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c007bb0 con 0x7f86301083d0 2024-09-21T06:09:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.669+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f861c007d50 con 0x7f86301083d0 2024-09-21T06:09:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.669+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863019f5c0 con 0x7f86301083d0 2024-09-21T06:09:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.669+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c018730 con 0x7f86301083d0 2024-09-21T06:09:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.669+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a5d20 con 0x7f86301083d0 2024-09-21T06:09:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.670+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8630103d10 con 0x7f86301083d0 2024-09-21T06:09:36.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.671+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f861c00f990 con 0x7f86301083d0 2024-09-21T06:09:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.672+0000 7f862dffb640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f86200b6560 0x7f86200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.672+0000 7f8634dff640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f86200b6560 0x7f86200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.672+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8094+0+0 (secure 0 0 0) 0x7f861c0e7610 con 0x7f86301083d0 2024-09-21T06:09:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.673+0000 7f8634dff640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f86200b6560 0x7f86200b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8624031440 tx=0x7f8624005e80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.676+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f861c014030 con 0x7f86301083d0 2024-09-21T06:09:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:36.902+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8630102ef0 con 0x7f86301083d0 2024-09-21T06:09:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.280+0000 7f862dffb640 1 -- 172.21.15.31:0/2641955542 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8630102ef0 con 0x7f86301083d0 2024-09-21T06:09:37.282 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T06:09:37.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f86200b6560 msgr2=0x7f86200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f86200b6560 0x7f86200b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8624031440 tx=0x7f8624005e80 comp rx=0 tx=0).stop 2024-09-21T06:09:37.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 msgr2=0x7f8630199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:37.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 0x7f8630199900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c970 tx=0x7f861c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 shutdown_connections 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8630108d20 0x7f863019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f86200b6560 0x7f86200b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86301083d0 0x7f8630199900 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 --2- 172.21.15.31:0/2641955542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8630102ae0 0x7f8630199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.282+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 >> 172.21.15.31:0/2641955542 conn(0x7f8630079760 msgr2=0x7f8630101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.283+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 shutdown_connections 2024-09-21T06:09:37.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:37.283+0000 7f8637063640 1 -- 172.21.15.31:0/2641955542 wait complete. 2024-09-21T06:09:37.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:37 smithi110 ceph-mon[34550]: pgmap v256: 97 pgs: 12 creating+peering, 55 unknown, 30 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:37.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:37 smithi110 ceph-mon[34550]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:09:37.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:37 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2641955542' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:09:37.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:37 smithi110 ceph-mon[34550]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:09:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:37 smithi107 ceph-mon[34380]: pgmap v256: 97 pgs: 12 creating+peering, 55 unknown, 30 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:37 smithi107 ceph-mon[34380]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:09:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:37 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2641955542' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:09:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:37 smithi107 ceph-mon[34380]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:09:37.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:37 smithi031 ceph-mon[31237]: pgmap v256: 97 pgs: 12 creating+peering, 55 unknown, 30 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:37.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:37 smithi031 ceph-mon[31237]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:09:37.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:37 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2641955542' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:09:37.593 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:37 smithi031 ceph-mon[31237]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:09:37.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:09:38.194 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:38.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:38 smithi110 ceph-mon[34550]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:09:38.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:38 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2641955542' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:09:38.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:38 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:09:38.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:38 smithi110 ceph-mon[34550]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:09:38.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:38 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] up:active 2024-09-21T06:09:38.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:38 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:38.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:38 smithi107 ceph-mon[34380]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:09:38.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:38 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2641955542' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:09:38.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:38 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:09:38.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:38 smithi107 ceph-mon[34380]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:09:38.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:38 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] up:active 2024-09-21T06:09:38.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:38 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:38.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:38 smithi031 ceph-mon[31237]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:09:38.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:38 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2641955542' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:09:38.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:38 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:09:38.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:38 smithi031 ceph-mon[31237]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:09:38.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:38 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] up:active 2024-09-21T06:09:38.564 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:38 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:38.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.654+0000 7fa14f453640 1 -- 172.21.15.31:0/3188045452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14810de10 msgr2=0x7fa1481102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.654+0000 7fa14f453640 1 --2- 172.21.15.31:0/3188045452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14810de10 0x7fa1481102f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa140009ca0 tx=0x7fa14002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.654+0000 7fa14f453640 1 -- 172.21.15.31:0/3188045452 shutdown_connections 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.654+0000 7fa14f453640 1 --2- 172.21.15.31:0/3188045452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa14810de10 0x7fa1481102f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.654+0000 7fa14f453640 1 --2- 172.21.15.31:0/3188045452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa148105290 0x7fa14810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.655+0000 7fa14f453640 1 --2- 172.21.15.31:0/3188045452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa148104860 0x7fa148104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.655+0000 7fa14f453640 1 -- 172.21.15.31:0/3188045452 >> 172.21.15.31:0/3188045452 conn(0x7fa148079730 msgr2=0x7fa148079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.655+0000 7fa14f453640 1 -- 172.21.15.31:0/3188045452 shutdown_connections 2024-09-21T06:09:38.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.655+0000 7fa14f453640 1 -- 172.21.15.31:0/3188045452 wait complete. 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.655+0000 7fa14f453640 1 Processor -- start 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.655+0000 7fa14f453640 1 -- start start 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14f453640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 0x7fa14819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14f453640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa148105290 0x7fa14819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14f453640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14810de10 0x7fa14819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14f453640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa148113a80 con 0x7fa148104860 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14f453640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa148113900 con 0x7fa148105290 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14f453640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa148113c00 con 0x7fa14810de10 2024-09-21T06:09:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 0x7fa14819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 0x7fa14819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53646/0 (socket says 172.21.15.31:53646) 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 -- 172.21.15.31:0/1437930825 learned_addr learned my addr 172.21.15.31:0/1437930825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 -- 172.21.15.31:0/1437930825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14810de10 msgr2=0x7fa14819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14810de10 0x7fa14819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 -- 172.21.15.31:0/1437930825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa148105290 msgr2=0x7fa14819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14c9ee640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa148105290 0x7fa14819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa148105290 0x7fa14819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 -- 172.21.15.31:0/1437930825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481a03a0 con 0x7fa148104860 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa14d1ef640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 0x7fa14819a430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d950 tx=0x7fa14400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.656+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144017070 con 0x7fa148104860 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.657+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa144009c30 con 0x7fa148104860 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.657+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a0690 con 0x7fa148104860 2024-09-21T06:09:38.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.657+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f620 con 0x7fa148104860 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.657+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a6df0 con 0x7fa148104860 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.658+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148104c70 con 0x7fa148104860 2024-09-21T06:09:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.659+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa144005030 con 0x7fa148104860 2024-09-21T06:09:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.660+0000 7fa13e7fc640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa1340b6560 0x7fa1340b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.660+0000 7fa14c9ee640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa1340b6560 0x7fa1340b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.660+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fa1440e7060 con 0x7fa148104860 2024-09-21T06:09:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.661+0000 7fa14c9ee640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa1340b6560 0x7fa1340b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa138005fd0 tx=0x7fa138005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.664+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1440ad8b0 con 0x7fa148104860 2024-09-21T06:09:38.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.852+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1481a71a0 con 0x7fa148104860 2024-09-21T06:09:38.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.854+0000 7fa13e7fc640 1 -- 172.21.15.31:0/1437930825 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22610 (secure 0 0 0) 0x7fa144009de0 con 0x7fa148104860 2024-09-21T06:09:38.856 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:38.856 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:09:36.903462+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa1340b6560 msgr2=0x7fa1340b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa1340b6560 0x7fa1340b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa138005fd0 tx=0x7fa138005850 comp rx=0 tx=0).stop 2024-09-21T06:09:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 msgr2=0x7fa14819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 0x7fa14819a430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d950 tx=0x7fa14400de10 comp rx=0 tx=0).stop 2024-09-21T06:09:38.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 shutdown_connections 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa14810de10 0x7fa14819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa148105290 0x7fa14819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa1340b6560 0x7fa1340b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 --2- 172.21.15.31:0/1437930825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa148104860 0x7fa14819a430 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 >> 172.21.15.31:0/1437930825 conn(0x7fa148079730 msgr2=0x7fa148107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 shutdown_connections 2024-09-21T06:09:38.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:38.857+0000 7fa14f453640 1 -- 172.21.15.31:0/1437930825 wait complete. 2024-09-21T06:09:39.493 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:39 smithi031 ceph-mon[31237]: pgmap v259: 97 pgs: 26 creating+peering, 38 unknown, 33 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:39.493 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:39 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1437930825' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:39.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:09:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:39 smithi107 ceph-mon[34380]: pgmap v259: 97 pgs: 26 creating+peering, 38 unknown, 33 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:39 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1437930825' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:39 smithi110 ceph-mon[34550]: pgmap v259: 97 pgs: 26 creating+peering, 38 unknown, 33 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:39 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1437930825' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 -- 172.21.15.31:0/3177291826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 msgr2=0x7fc2201098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 --2- 172.21.15.31:0/3177291826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc2201098b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc208009900 tx=0x7fc20802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 -- 172.21.15.31:0/3177291826 shutdown_connections 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 --2- 172.21.15.31:0/3177291826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc220109df0 0x7fc220110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 --2- 172.21.15.31:0/3177291826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc2201098b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 --2- 172.21.15.31:0/3177291826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc220103b60 0x7fc220103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 -- 172.21.15.31:0/3177291826 >> 172.21.15.31:0/3177291826 conn(0x7fc22007b850 msgr2=0x7fc22007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 -- 172.21.15.31:0/3177291826 shutdown_connections 2024-09-21T06:09:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 -- 172.21.15.31:0/3177291826 wait complete. 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.253+0000 7fc227067640 1 Processor -- start 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 -- start start 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc220103b60 0x7fc22019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc22019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc220109df0 0x7fc22019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc220113b00 con 0x7fc2201094a0 2024-09-21T06:09:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc220113980 con 0x7fc220103b60 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc227067640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc220113c80 con 0x7fc220109df0 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc22019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc22019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48130/0 (socket says 172.21.15.31:48130) 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 -- 172.21.15.31:0/674097149 learned_addr learned my addr 172.21.15.31:0/674097149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc224e03640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc220103b60 0x7fc22019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc225604640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc220109df0 0x7fc22019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 -- 172.21.15.31:0/674097149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc220109df0 msgr2=0x7fc22019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc220109df0 0x7fc22019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 -- 172.21.15.31:0/674097149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc220103b60 msgr2=0x7fc22019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc220103b60 0x7fc22019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.254+0000 7fc21ffff640 1 -- 172.21.15.31:0/674097149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2201a0360 con 0x7fc2201094a0 2024-09-21T06:09:40.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.255+0000 7fc21ffff640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc22019aa50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc208005ab0 tx=0x7fc208003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.255+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20803f070 con 0x7fc2201094a0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.255+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc208031d60 con 0x7fc2201094a0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.255+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2201a05f0 con 0x7fc2201094a0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.255+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208035030 con 0x7fc2201094a0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.255+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2201a0b20 con 0x7fc2201094a0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.256+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc220103f70 con 0x7fc2201094a0 2024-09-21T06:09:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.258+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc208042070 con 0x7fc2201094a0 2024-09-21T06:09:40.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.258+0000 7fc21dffb640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc20c0b6610 0x7fc20c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:40.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.258+0000 7fc224e03640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc20c0b6610 0x7fc20c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:40.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.258+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fc208108290 con 0x7fc2201094a0 2024-09-21T06:09:40.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.259+0000 7fc224e03640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc20c0b6610 0x7fc20c0b8b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc21000abe0 tx=0x7fc21000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:40.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.262+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2201a0b20 con 0x7fc2201094a0 2024-09-21T06:09:40.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.453+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc22002e640 con 0x7fc2201094a0 2024-09-21T06:09:40.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.455+0000 7fc21dffb640 1 -- 172.21.15.31:0/674097149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22610 (secure 0 0 0) 0x7fc2080ceb40 con 0x7fc2201094a0 2024-09-21T06:09:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:40.457 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:09:36.903462+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc20c0b6610 msgr2=0x7fc20c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc20c0b6610 0x7fc20c0b8b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc21000abe0 tx=0x7fc21000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 msgr2=0x7fc22019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc22019aa50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc208005ab0 tx=0x7fc208003cd0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 shutdown_connections 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc220109df0 0x7fc22019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc20c0b6610 0x7fc20c0b8b20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2201094a0 0x7fc22019aa50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 --2- 172.21.15.31:0/674097149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc220103b60 0x7fc22019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:40.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 >> 172.21.15.31:0/674097149 conn(0x7fc22007b850 msgr2=0x7fc220107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:40.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 shutdown_connections 2024-09-21T06:09:40.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:40.457+0000 7fc227067640 1 -- 172.21.15.31:0/674097149 wait complete. 2024-09-21T06:09:41.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:09:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:41 smithi107 ceph-mon[34380]: pgmap v260: 97 pgs: 26 creating+peering, 26 unknown, 45 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-21T06:09:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:41 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/674097149' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:41.568 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:41 smithi031 ceph-mon[31237]: pgmap v260: 97 pgs: 26 creating+peering, 26 unknown, 45 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-21T06:09:41.568 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:41 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/674097149' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:41 smithi110 ceph-mon[34550]: pgmap v260: 97 pgs: 26 creating+peering, 26 unknown, 45 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-21T06:09:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:41 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/674097149' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 -- 172.21.15.31:0/598754478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9330102a60 msgr2=0x7f9330102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 --2- 172.21.15.31:0/598754478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9330102a60 0x7f9330102e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9328009900 tx=0x7f932802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 -- 172.21.15.31:0/598754478 shutdown_connections 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 --2- 172.21.15.31:0/598754478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 0x7f933010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 --2- 172.21.15.31:0/598754478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93301083a0 0x7f93301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 --2- 172.21.15.31:0/598754478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9330102a60 0x7f9330102e70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 -- 172.21.15.31:0/598754478 >> 172.21.15.31:0/598754478 conn(0x7f9330079730 msgr2=0x7f9330079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 -- 172.21.15.31:0/598754478 shutdown_connections 2024-09-21T06:09:41.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.872+0000 7f9337b6e640 1 -- 172.21.15.31:0/598754478 wait complete. 2024-09-21T06:09:41.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 Processor -- start 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 -- start start 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9330102a60 0x7f933019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 0x7f933019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 0x7f933019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112960 con 0x7f93301083a0 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93301127e0 con 0x7f9330108cf0 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9337b6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9330112ae0 con 0x7f9330102a60 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9335109640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 0x7f933019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9335109640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 0x7f933019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48140/0 (socket says 172.21.15.31:48140) 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f9335109640 1 -- 172.21.15.31:0/1474106705 learned_addr learned my addr 172.21.15.31:0/1474106705 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f933590a640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9330102a60 0x7f933019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.873+0000 7f933610b640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 0x7f933019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9335109640 1 -- 172.21.15.31:0/1474106705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9330102a60 msgr2=0x7f933019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9335109640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9330102a60 0x7f933019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9335109640 1 -- 172.21.15.31:0/1474106705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 msgr2=0x7f933019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9335109640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 0x7f933019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9335109640 1 -- 172.21.15.31:0/1474106705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93301a0390 con 0x7f93301083a0 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f933610b640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 0x7f933019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9335109640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 0x7f933019a9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f932400c970 tx=0x7f932400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324007bb0 con 0x7f93301083a0 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93301a0680 con 0x7f93301083a0 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9324007d50 con 0x7f93301083a0 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932400f450 con 0x7f93301083a0 2024-09-21T06:09:41.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.874+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93301a6de0 con 0x7f93301083a0 2024-09-21T06:09:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.875+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9330102e70 con 0x7f93301083a0 2024-09-21T06:09:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.877+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f9324004f20 con 0x7f93301083a0 2024-09-21T06:09:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.877+0000 7f932effd640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93180b6560 0x7f93180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.878+0000 7f933590a640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93180b6560 0x7f93180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:41.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.878+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f93240e73c0 con 0x7f93301083a0 2024-09-21T06:09:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.878+0000 7f933590a640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93180b6560 0x7f93180b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9328031440 tx=0x7f9328009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:41.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:41.882+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9324014030 con 0x7f93301083a0 2024-09-21T06:09:42.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.110+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f93301a70c0 con 0x7f93301083a0 2024-09-21T06:09:42.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.291+0000 7f932effd640 1 -- 172.21.15.31:0/1474106705 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f93240adc70 con 0x7f93301083a0 2024-09-21T06:09:42.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93180b6560 msgr2=0x7f93180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93180b6560 0x7f93180b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9328031440 tx=0x7f9328009560 comp rx=0 tx=0).stop 2024-09-21T06:09:42.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 msgr2=0x7f933019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:42.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 0x7f933019a9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f932400c970 tx=0x7f932400ce30 comp rx=0 tx=0).stop 2024-09-21T06:09:42.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 shutdown_connections 2024-09-21T06:09:42.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9330108cf0 0x7f933019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93180b6560 0x7f93180b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93301083a0 0x7f933019a9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 --2- 172.21.15.31:0/1474106705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9330102a60 0x7f933019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.293+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 >> 172.21.15.31:0/1474106705 conn(0x7f9330079730 msgr2=0x7f9330101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.294+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 shutdown_connections 2024-09-21T06:09:42.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:42.294+0000 7f9337b6e640 1 -- 172.21.15.31:0/1474106705 wait complete. 2024-09-21T06:09:42.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:42 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1474106705' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:09:42.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:42 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1474106705' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:09:42.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:42 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1474106705' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:09:42.965 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-21T06:09:42.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T06:09:43.225 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:43.289 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:09:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: 2024-09-21T06:09:43.005+0000 7fac137b6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:09:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:43 smithi031 ceph-mon[31237]: pgmap v261: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-21T06:09:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:43 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1474106705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:09:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:43 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:43.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 -- 172.21.15.31:0/727074805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc10de60 msgr2=0x7f56bc110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 --2- 172.21.15.31:0/727074805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc10de60 0x7f56bc110330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009ca0 tx=0x7f56b402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 -- 172.21.15.31:0/727074805 shutdown_connections 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 --2- 172.21.15.31:0/727074805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc10de60 0x7f56bc110330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 --2- 172.21.15.31:0/727074805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56bc105380 0x7f56bc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 --2- 172.21.15.31:0/727074805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc104a30 0x7f56bc104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 -- 172.21.15.31:0/727074805 >> 172.21.15.31:0/727074805 conn(0x7f56bc079760 msgr2=0x7f56bc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 -- 172.21.15.31:0/727074805 shutdown_connections 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.728+0000 7f56c3706640 1 -- 172.21.15.31:0/727074805 wait complete. 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 Processor -- start 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 -- start start 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56bc104a30 0x7f56bc102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 0x7f56bc102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc10de60 0x7f56bc1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc113aa0 con 0x7f56bc105380 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc113920 con 0x7f56bc104a30 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c3706640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc113c20 con 0x7f56bc10de60 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c0ca1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 0x7f56bc102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c0ca1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 0x7f56bc102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48156/0 (socket says 172.21.15.31:48156) 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.729+0000 7f56c0ca1640 1 -- 172.21.15.31:0/3775886200 learned_addr learned my addr 172.21.15.31:0/3775886200 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c1ca3640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc10de60 0x7f56bc1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c14a2640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56bc104a30 0x7f56bc102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c0ca1640 1 -- 172.21.15.31:0/3775886200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc10de60 msgr2=0x7f56bc1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c0ca1640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc10de60 0x7f56bc1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c0ca1640 1 -- 172.21.15.31:0/3775886200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56bc104a30 msgr2=0x7f56bc102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c0ca1640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56bc104a30 0x7f56bc102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c0ca1640 1 -- 172.21.15.31:0/3775886200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc1a8620 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c1ca3640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc10de60 0x7f56bc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c0ca1640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 0x7f56bc102cb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00c8c0 tx=0x7f56ac00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac007bb0 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56ac007d50 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0187c0 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc1a8910 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.730+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1a8e50 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.731+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc06c1f0 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.734+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f56ac00fa20 con 0x7f56bc105380 2024-09-21T06:09:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.735+0000 7f56b27fc640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56a80b6560 0x7f56a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.735+0000 7f56c14a2640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56a80b6560 0x7f56a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.735+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f56ac0e77e0 con 0x7f56bc105380 2024-09-21T06:09:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.735+0000 7f56c14a2640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56a80b6560 0x7f56a80b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005fd0 tx=0x7f56b8005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.738+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f56ac014030 con 0x7f56bc105380 2024-09-21T06:09:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:43 smithi110 ceph-mon[34550]: pgmap v261: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-21T06:09:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:43 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1474106705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:09:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:43 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:43 smithi107 ceph-mon[34380]: pgmap v261: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-21T06:09:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:43 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1474106705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:09:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:43 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:43.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:43.968+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f56bc103a70 con 0x7f56bc105380 2024-09-21T06:09:44.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.299+0000 7f56b27fc640 1 -- 172.21.15.31:0/3775886200 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f56ac0ae120 con 0x7f56bc105380 2024-09-21T06:09:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56a80b6560 msgr2=0x7f56a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56a80b6560 0x7f56a80b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005fd0 tx=0x7f56b8005850 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 msgr2=0x7f56bc102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 0x7f56bc102cb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00c8c0 tx=0x7f56ac00cd80 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 shutdown_connections 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f56bc10de60 0x7f56bc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56a80b6560 0x7f56a80b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56bc105380 0x7f56bc102cb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 --2- 172.21.15.31:0/3775886200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56bc104a30 0x7f56bc102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 >> 172.21.15.31:0/3775886200 conn(0x7f56bc079760 msgr2=0x7f56bc105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:44.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 shutdown_connections 2024-09-21T06:09:44.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:44.302+0000 7f56c3706640 1 -- 172.21.15.31:0/3775886200 wait complete. 2024-09-21T06:09:44.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:44 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3775886200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:09:44.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:44 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3775886200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:09:44.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:44 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3775886200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:09:45.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T06:09:45.308 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:45 smithi031 ceph-mon[31237]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T06:09:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:45 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3775886200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:09:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:45 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:45 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:09:45.679 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:45 smithi110 ceph-mon[34550]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T06:09:45.680 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:45 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3775886200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:09:45.680 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:45 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:45.680 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:45 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:09:45.690 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:45 smithi107 ceph-mon[34380]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T06:09:45.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:45 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3775886200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:09:45.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:45 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:09:45.691 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:45 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:09:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/9073217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc1083c0 msgr2=0x7fe8bc1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/9073217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc1087d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009900 tx=0x7fe8a402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/9073217 shutdown_connections 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/9073217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8bc108d10 0x7fe8bc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/9073217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc1087d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/9073217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/9073217 >> 172.21.15.31:0/9073217 conn(0x7fe8bc0796d0 msgr2=0x7fe8bc079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/9073217 shutdown_connections 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.729+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/9073217 wait complete. 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 Processor -- start 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 -- start start 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8bc108d10 0x7fe8bc19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1129d0 con 0x7fe8bc102ad0 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc112850 con 0x7fe8bc108d10 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8c0bf5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc112b50 con 0x7fe8bc1083c0 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8ba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8ba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48184/0 (socket says 172.21.15.31:48184) 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.730+0000 7fe8ba59c640 1 -- 172.21.15.31:0/2693336525 learned_addr learned my addr 172.21.15.31:0/2693336525 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8bad9d640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8bc108d10 0x7fe8bc19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8b9d9b640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8ba59c640 1 -- 172.21.15.31:0/2693336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc1083c0 msgr2=0x7fe8bc19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8ba59c640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8ba59c640 1 -- 172.21.15.31:0/2693336525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8bc108d10 msgr2=0x7fe8bc19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8ba59c640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8bc108d10 0x7fe8bc19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8ba59c640 1 -- 172.21.15.31:0/2693336525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc1a0560 con 0x7fe8bc102ad0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8b9d9b640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8ba59c640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc19a5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00db80 tx=0x7fe8ac00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac017070 con 0x7fe8bc102ad0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac003b70 con 0x7fe8bc102ad0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc1a0850 con 0x7fe8bc102ad0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0106c0 con 0x7fe8bc102ad0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.731+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc1a6fb0 con 0x7fe8bc102ad0 2024-09-21T06:09:45.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.732+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8bc103d00 con 0x7fe8bc102ad0 2024-09-21T06:09:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.734+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe8ac004020 con 0x7fe8bc102ad0 2024-09-21T06:09:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.735+0000 7fe8b37fe640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8a80b6560 0x7fe8a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.735+0000 7fe8b9d9b640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8a80b6560 0x7fe8a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.735+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fe8ac0e7720 con 0x7fe8bc102ad0 2024-09-21T06:09:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.735+0000 7fe8b9d9b640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8a80b6560 0x7fe8a80b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009230 tx=0x7fe8a4000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:45.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.738+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe8ac0adfd0 con 0x7fe8bc102ad0 2024-09-21T06:09:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:45.965+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe8bc102ee0 con 0x7fe8bc102ad0 2024-09-21T06:09:46.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.310+0000 7fe8b37fe640 1 -- 172.21.15.31:0/2693336525 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7fe8bc102ee0 con 0x7fe8bc102ad0 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8a80b6560 msgr2=0x7fe8a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8a80b6560 0x7fe8a80b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009230 tx=0x7fe8a4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 msgr2=0x7fe8bc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc19a5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00db80 tx=0x7fe8ac00b630 comp rx=0 tx=0).stop 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 shutdown_connections 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8bc108d10 0x7fe8bc19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8bc1083c0 0x7fe8bc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8a80b6560 0x7fe8a80b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 --2- 172.21.15.31:0/2693336525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8bc102ad0 0x7fe8bc19a5f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 >> 172.21.15.31:0/2693336525 conn(0x7fe8bc0796d0 msgr2=0x7fe8bc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 shutdown_connections 2024-09-21T06:09:46.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:46.313+0000 7fe8c0bf5640 1 -- 172.21.15.31:0/2693336525 wait complete. 2024-09-21T06:09:46.616 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:46 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2693336525' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:09:46.692 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:46 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2693336525' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:09:46.703 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:46 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2693336525' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:09:46.961 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T06:09:46.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T06:09:47.224 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:47.595 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-mon[31237]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-21T06:09:47.595 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2693336525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:09:47.595 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:09:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 -- 172.21.15.31:0/3321737850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1083b0 msgr2=0x7f342c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 --2- 172.21.15.31:0/3321737850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1083b0 0x7f342c1087c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f341c009ca0 tx=0x7f341c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 -- 172.21.15.31:0/3321737850 shutdown_connections 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 --2- 172.21.15.31:0/3321737850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c108d00 0x7f342c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 --2- 172.21.15.31:0/3321737850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c1083b0 0x7f342c1087c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 --2- 172.21.15.31:0/3321737850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c102a70 0x7f342c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.691+0000 7f3432c6c640 1 -- 172.21.15.31:0/3321737850 >> 172.21.15.31:0/3321737850 conn(0x7f342c079710 msgr2=0x7f342c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.692+0000 7f3432c6c640 1 -- 172.21.15.31:0/3321737850 shutdown_connections 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.692+0000 7f3432c6c640 1 -- 172.21.15.31:0/3321737850 wait complete. 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.692+0000 7f3432c6c640 1 Processor -- start 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.692+0000 7f3432c6c640 1 -- start start 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3432c6c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c102a70 0x7f342c110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3432c6c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c1083b0 0x7f342c110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3432c6c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 0x7f342c110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3432c6c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113ae0 con 0x7f342c108d00 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3432c6c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113960 con 0x7f342c1083b0 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3432c6c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113c60 con 0x7f342c102a70 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 0x7f342c110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 0x7f342c110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48198/0 (socket says 172.21.15.31:48198) 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 -- 172.21.15.31:0/2124011595 learned_addr learned my addr 172.21.15.31:0/2124011595 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3430a08640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c102a70 0x7f342c110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f342bfff640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c1083b0 0x7f342c110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 -- 172.21.15.31:0/2124011595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c102a70 msgr2=0x7f342c110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c102a70 0x7f342c110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 -- 172.21.15.31:0/2124011595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c1083b0 msgr2=0x7f342c110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c1083b0 0x7f342c110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 -- 172.21.15.31:0/2124011595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1a87d0 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3431209640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 0x7f342c110a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f342400e970 tx=0x7f342400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.693+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400ccc0 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.694+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342400ce60 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.694+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a8ac0 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.694+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424010430 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.694+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a9020 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.695+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c103cf0 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.697+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3424004f70 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.698+0000 7f3429ffb640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b65e0 0x7f34180b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.698+0000 7f3430a08640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b65e0 0x7f34180b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.698+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f34240e7820 con 0x7f342c108d00 2024-09-21T06:09:47.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.699+0000 7f3430a08640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b65e0 0x7f34180b8af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f342c111a60 tx=0x7f3414005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:47.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.701+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3424056710 con 0x7f342c108d00 2024-09-21T06:09:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:47 smithi110 ceph-mon[34550]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-21T06:09:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:47 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2693336525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:09:47.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:47 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:09:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:47 smithi107 ceph-mon[34380]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-21T06:09:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:47 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2693336525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:09:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:47 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:09:47.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.916+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f342c063800 con 0x7f34180b65e0 2024-09-21T06:09:47.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.934+0000 7f3429ffb640 1 -- 172.21.15.31:0/2124011595 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f342c063800 con 0x7f34180b65e0 2024-09-21T06:09:47.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b65e0 msgr2=0x7f34180b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b65e0 0x7f34180b8af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f342c111a60 tx=0x7f3414005f90 comp rx=0 tx=0).stop 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 msgr2=0x7f342c110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 0x7f342c110a90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f342400e970 tx=0x7f342400ed70 comp rx=0 tx=0).stop 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 shutdown_connections 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b65e0 0x7f34180b8af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c108d00 0x7f342c110a90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c1083b0 0x7f342c110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 --2- 172.21.15.31:0/2124011595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c102a70 0x7f342c110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 >> 172.21.15.31:0/2124011595 conn(0x7f342c079710 msgr2=0x7f342c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 shutdown_connections 2024-09-21T06:09:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:47.937+0000 7f3432c6c640 1 -- 172.21.15.31:0/2124011595 wait complete. 2024-09-21T06:09:48.249 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:09:47.922+0000 7f24d3ece640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:48.249 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:09:47.922+0000 7f24d3ece640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:48.249 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:09:47.922+0000 7f24d3ece640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:48.249 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:09:47.922+0000 7f24d3ece640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:48.249 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:09:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:09:47.922+0000 7f24d3ece640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:09:48.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T06:09:48.855 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 -- 172.21.15.31:0/620916692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 msgr2=0x7f3f681098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 --2- 172.21.15.31:0/620916692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f681098b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009900 tx=0x7f3f5002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 -- 172.21.15.31:0/620916692 shutdown_connections 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 --2- 172.21.15.31:0/620916692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68109df0 0x7f3f68110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 --2- 172.21.15.31:0/620916692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f681098b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 --2- 172.21.15.31:0/620916692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68103b60 0x7f3f68103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 -- 172.21.15.31:0/620916692 >> 172.21.15.31:0/620916692 conn(0x7f3f6807b850 msgr2=0x7f3f6807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:49.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 -- 172.21.15.31:0/620916692 shutdown_connections 2024-09-21T06:09:49.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.335+0000 7f3f6e075640 1 -- 172.21.15.31:0/620916692 wait complete. 2024-09-21T06:09:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 Processor -- start 2024-09-21T06:09:49.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 -- start start 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68103b60 0x7f3f6819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f6819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68109df0 0x7f3f6819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68113af0 con 0x7f3f681094a0 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68113970 con 0x7f3f68103b60 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f6e075640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68113c70 con 0x7f3f68109df0 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f66ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f6819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f66ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f6819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52442/0 (socket says 172.21.15.31:52442) 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.336+0000 7f3f66ffd640 1 -- 172.21.15.31:0/3299522090 learned_addr learned my addr 172.21.15.31:0/3299522090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f67fff640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68109df0 0x7f3f6819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f677fe640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68103b60 0x7f3f6819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f66ffd640 1 -- 172.21.15.31:0/3299522090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68109df0 msgr2=0x7f3f6819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f66ffd640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68109df0 0x7f3f6819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f66ffd640 1 -- 172.21.15.31:0/3299522090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68103b60 msgr2=0x7f3f6819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f66ffd640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68103b60 0x7f3f6819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f66ffd640 1 -- 172.21.15.31:0/3299522090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f681a0450 con 0x7f3f681094a0 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f677fe640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68103b60 0x7f3f6819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f67fff640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68109df0 0x7f3f6819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f66ffd640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f6819aa80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3f5002a930 tx=0x7f3f50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5003f030 con 0x7f3f681094a0 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f681a06e0 con 0x7f3f681094a0 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f50003aa0 con 0x7f3f681094a0 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50035070 con 0x7f3f681094a0 2024-09-21T06:09:49.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.337+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f681a6ea0 con 0x7f3f681094a0 2024-09-21T06:09:49.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.338+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68103f70 con 0x7f3f681094a0 2024-09-21T06:09:49.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.340+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3f50042030 con 0x7f3f681094a0 2024-09-21T06:09:49.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.341+0000 7f3f64ff9640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3f540b6560 0x7f3f540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:49.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.341+0000 7f3f677fe640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3f540b6560 0x7f3f540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:49.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.341+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f3f50107f30 con 0x7f3f681094a0 2024-09-21T06:09:49.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.341+0000 7f3f677fe640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3f540b6560 0x7f3f540b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009b40 tx=0x7f3f58005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:49.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.344+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3f50139710 con 0x7f3f681094a0 2024-09-21T06:09:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.560+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3f68063800 con 0x7f3f540b6560 2024-09-21T06:09:49.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:49 smithi031 ceph-mon[31237]: pgmap v264: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-21T06:09:49.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:49 smithi031 ceph-mon[31237]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:49.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.588+0000 7f3f64ff9640 1 -- 172.21.15.31:0/3299522090 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f68063800 con 0x7f3f540b6560 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3f540b6560 msgr2=0x7f3f540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3f540b6560 0x7f3f540b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009b40 tx=0x7f3f58005e50 comp rx=0 tx=0).stop 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 msgr2=0x7f3f6819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f6819aa80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3f5002a930 tx=0x7f3f50004270 comp rx=0 tx=0).stop 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 shutdown_connections 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3f68109df0 0x7f3f6819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3f540b6560 0x7f3f540b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f681094a0 0x7f3f6819aa80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 --2- 172.21.15.31:0/3299522090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f68103b60 0x7f3f6819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 >> 172.21.15.31:0/3299522090 conn(0x7f3f6807b850 msgr2=0x7f3f681054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 shutdown_connections 2024-09-21T06:09:49.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:49.591+0000 7f3f6e075640 1 -- 172.21.15.31:0/3299522090 wait complete. 2024-09-21T06:09:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:49 smithi110 ceph-mon[34550]: pgmap v264: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-21T06:09:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:49 smithi110 ceph-mon[34550]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:49 smithi107 ceph-mon[34380]: pgmap v264: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-21T06:09:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:49 smithi107 ceph-mon[34380]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:50.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T06:09:50.534 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 -- 172.21.15.31:0/263576577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8109df0 msgr2=0x7fafb8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 --2- 172.21.15.31:0/263576577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8109df0 0x7fafb8110620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009ca0 tx=0x7fafb002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 -- 172.21.15.31:0/263576577 shutdown_connections 2024-09-21T06:09:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 --2- 172.21.15.31:0/263576577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8109df0 0x7fafb8110620 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 --2- 172.21.15.31:0/263576577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb81094a0 0x7fafb81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 --2- 172.21.15.31:0/263576577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8103b60 0x7fafb8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 -- 172.21.15.31:0/263576577 >> 172.21.15.31:0/263576577 conn(0x7fafb807b850 msgr2=0x7fafb807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 -- 172.21.15.31:0/263576577 shutdown_connections 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 -- 172.21.15.31:0/263576577 wait complete. 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.008+0000 7fafc03cc640 1 Processor -- start 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 -- start start 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 0x7fafb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb81094a0 0x7fafb8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8109df0 0x7fafb8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8113ab0 con 0x7fafb8103b60 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8113930 con 0x7fafb81094a0 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafc03cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8113c30 con 0x7fafb8109df0 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 0x7fafb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 0x7fafb8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52460/0 (socket says 172.21.15.31:52460) 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 -- 172.21.15.31:0/3884098858 learned_addr learned my addr 172.21.15.31:0/3884098858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe969640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8109df0 0x7fafb8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbd967640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb81094a0 0x7fafb8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 -- 172.21.15.31:0/3884098858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8109df0 msgr2=0x7fafb8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8109df0 0x7fafb8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 -- 172.21.15.31:0/3884098858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb81094a0 msgr2=0x7fafb8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb81094a0 0x7fafb8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.009+0000 7fafbe168640 1 -- 172.21.15.31:0/3884098858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb81a8700 con 0x7fafb8103b60 2024-09-21T06:09:51.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.010+0000 7fafbe168640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 0x7fafb8075710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fafb400d950 tx=0x7fafb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.010+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4017030 con 0x7fafb8103b60 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.010+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb81a89f0 con 0x7fafb8103b60 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.010+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafb4004020 con 0x7fafb8103b60 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.010+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40047a0 con 0x7fafb8103b60 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.010+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a8f80 con 0x7fafb8103b60 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.011+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb8103f70 con 0x7fafb8103b60 2024-09-21T06:09:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.013+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fafb4009a30 con 0x7fafb8103b60 2024-09-21T06:09:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.014+0000 7fafaf7fe640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fafa40b65f0 0x7fafa40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.014+0000 7fafbd967640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fafa40b65f0 0x7fafa40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.014+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fafb40e7560 con 0x7fafb8103b60 2024-09-21T06:09:51.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.014+0000 7fafbd967640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fafa40b65f0 0x7fafa40b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fafb8109190 tx=0x7fafa8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.017+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fafb40adcb0 con 0x7fafb8103b60 2024-09-21T06:09:51.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.237+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fafb8063800 con 0x7fafa40b65f0 2024-09-21T06:09:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.245+0000 7fafaf7fe640 1 -- 172.21.15.31:0/3884098858 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fafb8063800 con 0x7fafa40b65f0 2024-09-21T06:09:51.246 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_0/3dfc0d80-9312-4546-9b88-ad7aa8e234de 2024-09-21T06:09:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fafa40b65f0 msgr2=0x7fafa40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fafa40b65f0 0x7fafa40b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fafb8109190 tx=0x7fafa8005f90 comp rx=0 tx=0).stop 2024-09-21T06:09:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 msgr2=0x7fafb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:51.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 0x7fafb8075710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fafb400d950 tx=0x7fafb400de10 comp rx=0 tx=0).stop 2024-09-21T06:09:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 shutdown_connections 2024-09-21T06:09:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb8109df0 0x7fafb8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb81094a0 0x7fafb8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fafa40b65f0 0x7fafa40b8b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 --2- 172.21.15.31:0/3884098858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafb8103b60 0x7fafb8075710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:51.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 >> 172.21.15.31:0/3884098858 conn(0x7fafb807b850 msgr2=0x7fafb81074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:51.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 shutdown_connections 2024-09-21T06:09:51.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:51.247+0000 7fafc03cc640 1 -- 172.21.15.31:0/3884098858 wait complete. 2024-09-21T06:09:51.566 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:51 smithi031 ceph-mon[31237]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:51.566 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:51 smithi031 ceph-mon[31237]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T06:09:51.566 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:51 smithi031 ceph-mon[31237]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T06:09:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:51 smithi110 ceph-mon[34550]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:51 smithi110 ceph-mon[34550]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T06:09:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:51 smithi110 ceph-mon[34550]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T06:09:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:51 smithi107 ceph-mon[34380]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:51 smithi107 ceph-mon[34380]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-21T06:09:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:51 smithi107 ceph-mon[34380]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T06:09:51.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T06:09:52.153 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:52.530 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:52 smithi031 ceph-mon[31237]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 -- 172.21.15.31:0/74568999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 msgr2=0x7fd6d41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 --2- 172.21.15.31:0/74568999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d41098c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009900 tx=0x7fd6c802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 -- 172.21.15.31:0/74568999 shutdown_connections 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 --2- 172.21.15.31:0/74568999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4109e00 0x7fd6d4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 --2- 172.21.15.31:0/74568999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d41098c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 --2- 172.21.15.31:0/74568999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6d4103bc0 0x7fd6d4103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 -- 172.21.15.31:0/74568999 >> 172.21.15.31:0/74568999 conn(0x7fd6d407b910 msgr2=0x7fd6d407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 -- 172.21.15.31:0/74568999 shutdown_connections 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.626+0000 7fd6db008640 1 -- 172.21.15.31:0/74568999 wait complete. 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 Processor -- start 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 -- start start 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4103bc0 0x7fd6d419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6d4109e00 0x7fd6d419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4113ac0 con 0x7fd6d41094b0 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4113940 con 0x7fd6d4103bc0 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6db008640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d4113c40 con 0x7fd6d4109e00 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6d3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52466/0 (socket says 172.21.15.31:52466) 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6d3fff640 1 -- 172.21.15.31:0/1997126779 learned_addr learned my addr 172.21.15.31:0/1997126779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6d8da4640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4103bc0 0x7fd6d419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.627+0000 7fd6d95a5640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6d4109e00 0x7fd6d419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d3fff640 1 -- 172.21.15.31:0/1997126779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6d4109e00 msgr2=0x7fd6d419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d3fff640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6d4109e00 0x7fd6d419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d3fff640 1 -- 172.21.15.31:0/1997126779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4103bc0 msgr2=0x7fd6d419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d3fff640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4103bc0 0x7fd6d419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d3fff640 1 -- 172.21.15.31:0/1997126779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d41a0400 con 0x7fd6d41094b0 2024-09-21T06:09:52.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d8da4640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4103bc0 0x7fd6d419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d3fff640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d419aa30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd6c802a930 tx=0x7fd6c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c803f030 con 0x7fd6d41094b0 2024-09-21T06:09:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8003aa0 con 0x7fd6d41094b0 2024-09-21T06:09:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8035070 con 0x7fd6d41094b0 2024-09-21T06:09:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d41a0690 con 0x7fd6d41094b0 2024-09-21T06:09:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.628+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d41a6e50 con 0x7fd6d41094b0 2024-09-21T06:09:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.629+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d4104df0 con 0x7fd6d41094b0 2024-09-21T06:09:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.631+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd6c8042030 con 0x7fd6d41094b0 2024-09-21T06:09:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.631+0000 7fd6d1ffb640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd6bc0b6670 0x7fd6bc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:52.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.632+0000 7fd6d8da4640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd6bc0b6670 0x7fd6bc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:52.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.632+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fd6c8107c50 con 0x7fd6d41094b0 2024-09-21T06:09:52.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.632+0000 7fd6d8da4640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd6bc0b6670 0x7fd6bc0b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005d00 tx=0x7fd6c4005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:52.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.636+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6c80ce3a0 con 0x7fd6d41094b0 2024-09-21T06:09:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:52 smithi110 ceph-mon[34550]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:52 smithi107 ceph-mon[34380]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.851+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd6d4063800 con 0x7fd6bc0b6670 2024-09-21T06:09:52.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.883+0000 7fd6d1ffb640 1 -- 172.21.15.31:0/1997126779 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd6d4063800 con 0x7fd6bc0b6670 2024-09-21T06:09:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd6bc0b6670 msgr2=0x7fd6bc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd6bc0b6670 0x7fd6bc0b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4005d00 tx=0x7fd6c4005c70 comp rx=0 tx=0).stop 2024-09-21T06:09:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 msgr2=0x7fd6d419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d419aa30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd6c802a930 tx=0x7fd6c8004270 comp rx=0 tx=0).stop 2024-09-21T06:09:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 shutdown_connections 2024-09-21T06:09:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6d4109e00 0x7fd6d419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd6bc0b6670 0x7fd6bc0b8b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6d41094b0 0x7fd6d419aa30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 --2- 172.21.15.31:0/1997126779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6d4103bc0 0x7fd6d419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 >> 172.21.15.31:0/1997126779 conn(0x7fd6d407b910 msgr2=0x7fd6d41075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 shutdown_connections 2024-09-21T06:09:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:52.885+0000 7fd6db008640 1 -- 172.21.15.31:0/1997126779 wait complete. 2024-09-21T06:09:53.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:53 smithi031 ceph-mon[31237]: pgmap v266: 97 pgs: 97 active+clean; 601 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-21T06:09:53.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T06:09:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:53 smithi110 ceph-mon[34550]: pgmap v266: 97 pgs: 97 active+clean; 601 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-21T06:09:53.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:53 smithi107 ceph-mon[34380]: pgmap v266: 97 pgs: 97 active+clean; 601 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-21T06:09:53.808 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 -- 172.21.15.31:0/3550265385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 msgr2=0x7f6b5c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 --2- 172.21.15.31:0/3550265385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c1098b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009900 tx=0x7f6b5002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 -- 172.21.15.31:0/3550265385 shutdown_connections 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 --2- 172.21.15.31:0/3550265385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c109df0 0x7f6b5c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 --2- 172.21.15.31:0/3550265385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c1098b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 --2- 172.21.15.31:0/3550265385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b5c103b60 0x7f6b5c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 -- 172.21.15.31:0/3550265385 >> 172.21.15.31:0/3550265385 conn(0x7f6b5c07b850 msgr2=0x7f6b5c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 -- 172.21.15.31:0/3550265385 shutdown_connections 2024-09-21T06:09:54.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.268+0000 7f6b60af6640 1 -- 172.21.15.31:0/3550265385 wait complete. 2024-09-21T06:09:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 Processor -- start 2024-09-21T06:09:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 -- start start 2024-09-21T06:09:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c113af0 con 0x7f6b5c1094a0 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c113970 con 0x7f6b5c109df0 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b60af6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c113c70 con 0x7f6b5c103b60 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b59d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b59d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52484/0 (socket says 172.21.15.31:52484) 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b59d9b640 1 -- 172.21.15.31:0/1812439090 learned_addr learned my addr 172.21.15.31:0/1812439090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b5a59c640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.269+0000 7f6b5ad9d640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b59d9b640 1 -- 172.21.15.31:0/1812439090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b5c103b60 msgr2=0x7f6b5c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b59d9b640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b59d9b640 1 -- 172.21.15.31:0/1812439090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c109df0 msgr2=0x7f6b5c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b59d9b640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b59d9b640 1 -- 172.21.15.31:0/1812439090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c1a0450 con 0x7f6b5c1094a0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b59d9b640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19aa80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6b5002a930 tx=0x7f6b50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5003f030 con 0x7f6b5c1094a0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c1a06e0 con 0x7f6b5c1094a0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b50003aa0 con 0x7f6b5c1094a0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b50035070 con 0x7f6b5c1094a0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.270+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c1a6ea0 con 0x7f6b5c1094a0 2024-09-21T06:09:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.271+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c103f70 con 0x7f6b5c1094a0 2024-09-21T06:09:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.273+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6b50042030 con 0x7f6b5c1094a0 2024-09-21T06:09:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.273+0000 7f6b4b7fe640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b440b65f0 0x7f6b440b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.274+0000 7f6b5a59c640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b440b65f0 0x7f6b440b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.274+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f6b50107fd0 con 0x7f6b5c1094a0 2024-09-21T06:09:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.274+0000 7f6b5a59c640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b440b65f0 0x7f6b440b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c005fd0 tx=0x7f6b4c009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.277+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b500ce720 con 0x7f6b5c1094a0 2024-09-21T06:09:54.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.500+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f6b5c063800 con 0x7f6b440b65f0 2024-09-21T06:09:54.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.504+0000 7f6b4b7fe640 1 -- 172.21.15.31:0/1812439090 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6b5c063800 con 0x7f6b440b65f0 2024-09-21T06:09:54.506 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:54 smithi031 ceph-mon[31237]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:54.507 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/qa/sv_1/11ff3a3e-12f9-435b-bdf0-22b9c15d836e 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.506+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b440b65f0 msgr2=0x7f6b440b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b440b65f0 0x7f6b440b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c005fd0 tx=0x7f6b4c009680 comp rx=0 tx=0).stop 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 msgr2=0x7f6b5c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19aa80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6b5002a930 tx=0x7f6b50004270 comp rx=0 tx=0).stop 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 shutdown_connections 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b5c109df0 0x7f6b5c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b440b65f0 0x7f6b440b8b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b5c1094a0 0x7f6b5c19aa80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 --2- 172.21.15.31:0/1812439090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b5c103b60 0x7f6b5c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 >> 172.21.15.31:0/1812439090 conn(0x7f6b5c07b850 msgr2=0x7f6b5c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:54.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 shutdown_connections 2024-09-21T06:09:54.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:54.507+0000 7f6b60af6640 1 -- 172.21.15.31:0/1812439090 wait complete. 2024-09-21T06:09:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:54 smithi110 ceph-mon[34550]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:54 smithi107 ceph-mon[34380]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:55.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:55.552 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:55 smithi031 ceph-mon[31237]: pgmap v267: 97 pgs: 97 active+clean; 601 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 2 op/s 2024-09-21T06:09:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:55 smithi110 ceph-mon[34550]: pgmap v267: 97 pgs: 97 active+clean; 601 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 2 op/s 2024-09-21T06:09:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:55 smithi107 ceph-mon[34380]: pgmap v267: 97 pgs: 97 active+clean; 601 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 2 op/s 2024-09-21T06:09:55.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 -- 172.21.15.31:0/67494431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c108cf0 msgr2=0x7f4c9c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:55.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 --2- 172.21.15.31:0/67494431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c10f520 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009900 tx=0x7f4c8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 -- 172.21.15.31:0/67494431 shutdown_connections 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 --2- 172.21.15.31:0/67494431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c10f520 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 --2- 172.21.15.31:0/67494431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c9c1083a0 0x7f4c9c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 --2- 172.21.15.31:0/67494431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c102a60 0x7f4c9c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 -- 172.21.15.31:0/67494431 >> 172.21.15.31:0/67494431 conn(0x7f4c9c079760 msgr2=0x7f4c9c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 -- 172.21.15.31:0/67494431 shutdown_connections 2024-09-21T06:09:55.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.920+0000 7f4ca110d640 1 -- 172.21.15.31:0/67494431 wait complete. 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.921+0000 7f4ca110d640 1 Processor -- start 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.921+0000 7f4ca110d640 1 -- start start 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4ca110d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 0x7f4c9c0ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4ca110d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c9c1083a0 0x7f4c9c0ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4ca110d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c1000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4ca110d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1129b0 con 0x7f4c9c102a60 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4ca110d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c112830 con 0x7f4c9c108cf0 2024-09-21T06:09:55.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4ca110d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c112b30 con 0x7f4c9c1083a0 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 0x7f4c9c0ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 0x7f4c9c0ff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52502/0 (socket says 172.21.15.31:52502) 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 -- 172.21.15.31:0/1655526925 learned_addr learned my addr 172.21.15.31:0/1655526925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9a59c640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c9c1083a0 0x7f4c9c0ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9b59e640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c1000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 -- 172.21.15.31:0/1655526925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c9c1083a0 msgr2=0x7f4c9c0ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c9c1083a0 0x7f4c9c0ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 -- 172.21.15.31:0/1655526925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c108cf0 msgr2=0x7f4c9c1000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c1000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 -- 172.21.15.31:0/1655526925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c1a8660 con 0x7f4c9c102a60 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9b59e640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:55.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.922+0000 7f4c9ad9d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 0x7f4c9c0ff650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400cce0 tx=0x7f4c9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.923+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9400f650 con 0x7f4c9c102a60 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.923+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9400fca0 con 0x7f4c9c102a60 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.923+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c1a8950 con 0x7f4c9c102a60 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.923+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94018b40 con 0x7f4c9c102a60 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.923+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1a8e90 con 0x7f4c9c102a60 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.924+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c103ce0 con 0x7f4c9c102a60 2024-09-21T06:09:55.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.925+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4c9400f7f0 con 0x7f4c9c102a60 2024-09-21T06:09:55.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.926+0000 7f4c93fff640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b65f0 0x7f4c880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:55.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.927+0000 7f4c9a59c640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b65f0 0x7f4c880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:55.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.927+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f4c940e7a00 con 0x7f4c9c102a60 2024-09-21T06:09:55.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.927+0000 7f4c9a59c640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b65f0 0x7f4c880b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c1026e0 tx=0x7f4c8400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:55.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:55.931+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c94014030 con 0x7f4c9c102a60 2024-09-21T06:09:56.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.158+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c9c02e640 con 0x7f4c9c102a60 2024-09-21T06:09:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.159+0000 7f4c93fff640 1 -- 172.21.15.31:0/1655526925 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f4c940ae0f0 con 0x7f4c9c102a60 2024-09-21T06:09:56.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:56.161 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:09:56.161 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b65f0 msgr2=0x7f4c880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b65f0 0x7f4c880b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c1026e0 tx=0x7f4c8400b390 comp rx=0 tx=0).stop 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 msgr2=0x7f4c9c0ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 0x7f4c9c0ff650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400cce0 tx=0x7f4c9400a620 comp rx=0 tx=0).stop 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 shutdown_connections 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c9c108cf0 0x7f4c9c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c9c1083a0 0x7f4c9c0ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b65f0 0x7f4c880b8b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 --2- 172.21.15.31:0/1655526925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c9c102a60 0x7f4c9c0ff650 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:56.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 >> 172.21.15.31:0/1655526925 conn(0x7f4c9c079760 msgr2=0x7f4c9c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:56.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 shutdown_connections 2024-09-21T06:09:56.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:56.161+0000 7f4ca110d640 1 -- 172.21.15.31:0/1655526925 wait complete. 2024-09-21T06:09:56.535 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:56 smithi031 ceph-mon[31237]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:56.535 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:56 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1655526925' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:56 smithi110 ceph-mon[34550]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:56 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1655526925' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:56 smithi107 ceph-mon[34380]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:09:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:56 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1655526925' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:56.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:09:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:57.497 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:57 smithi031 ceph-mon[31237]: pgmap v268: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:09:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 -- 172.21.15.31:0/1805908546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c106c30 msgr2=0x7f607c110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 --2- 172.21.15.31:0/1805908546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c106c30 0x7f607c110680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6074009ca0 tx=0x7f607402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 -- 172.21.15.31:0/1805908546 shutdown_connections 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 --2- 172.21.15.31:0/1805908546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c106c30 0x7f607c110680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 --2- 172.21.15.31:0/1805908546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f607c1062e0 0x7f607c1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 --2- 172.21.15.31:0/1805908546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f607c102000 0x7f607c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.587+0000 7f6082437640 1 -- 172.21.15.31:0/1805908546 >> 172.21.15.31:0/1805908546 conn(0x7f607c079e40 msgr2=0x7f607c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.588+0000 7f6082437640 1 -- 172.21.15.31:0/1805908546 shutdown_connections 2024-09-21T06:09:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.588+0000 7f6082437640 1 -- 172.21.15.31:0/1805908546 wait complete. 2024-09-21T06:09:57.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.588+0000 7f6082437640 1 Processor -- start 2024-09-21T06:09:57.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.588+0000 7f6082437640 1 -- start start 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f6082437640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 0x7f607c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f6082437640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f607c1062e0 0x7f607c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f6082437640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f607c106c30 0x7f607c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f6082437640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f607c113ad0 con 0x7f607c102000 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f6082437640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f607c113950 con 0x7f607c106c30 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f6082437640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f607c113c50 con 0x7f607c1062e0 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 0x7f607c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 0x7f607c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52510/0 (socket says 172.21.15.31:52510) 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 -- 172.21.15.31:0/523735497 learned_addr learned my addr 172.21.15.31:0/523735497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607b7fe640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f607c1062e0 0x7f607c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f60809d4640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f607c106c30 0x7f607c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 -- 172.21.15.31:0/523735497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f607c1062e0 msgr2=0x7f607c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f607c1062e0 0x7f607c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 -- 172.21.15.31:0/523735497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f607c106c30 msgr2=0x7f607c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f607c106c30 0x7f607c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.589+0000 7f607bfff640 1 -- 172.21.15.31:0/523735497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c1a0370 con 0x7f607c102000 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.590+0000 7f607bfff640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 0x7f607c19a400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f606c00e9f0 tx=0x7f606c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.590+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c00cd80 con 0x7f607c102000 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.590+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c1a0660 con 0x7f607c102000 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.590+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f606c0108e0 con 0x7f607c102000 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.590+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c018980 con 0x7f607c102000 2024-09-21T06:09:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.590+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1a6dc0 con 0x7f607c102000 2024-09-21T06:09:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.591+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c102410 con 0x7f607c102000 2024-09-21T06:09:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.592+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f606c010430 con 0x7f607c102000 2024-09-21T06:09:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.593+0000 7f60797fa640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60640b65f0 0x7f60640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.593+0000 7f607b7fe640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60640b65f0 0x7f60640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.593+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f606c0e7c90 con 0x7f607c102000 2024-09-21T06:09:57.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.594+0000 7f607b7fe640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60640b65f0 0x7f60640b8b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f607c06b280 tx=0x7f607000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.596+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f606c0b27c0 con 0x7f607c102000 2024-09-21T06:09:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:57 smithi110 ceph-mon[34550]: pgmap v268: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:09:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.786+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f607c19b530 con 0x7f607c102000 2024-09-21T06:09:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:57 smithi107 ceph-mon[34380]: pgmap v268: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:09:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.788+0000 7f60797fa640 1 -- 172.21.15.31:0/523735497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22610 (secure 0 0 0) 0x7f606c0b7700 con 0x7f607c102000 2024-09-21T06:09:57.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:57.790 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:09:36.903462+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.790+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60640b65f0 msgr2=0x7f60640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60640b65f0 0x7f60640b8b00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f607c06b280 tx=0x7f607000b390 comp rx=0 tx=0).stop 2024-09-21T06:09:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 msgr2=0x7f607c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 0x7f607c19a400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f606c00e9f0 tx=0x7f606c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 shutdown_connections 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f607c106c30 0x7f607c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f607c1062e0 0x7f607c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60640b65f0 0x7f60640b8b00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 --2- 172.21.15.31:0/523735497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f607c102000 0x7f607c19a400 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 >> 172.21.15.31:0/523735497 conn(0x7f607c079e40 msgr2=0x7f607c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 shutdown_connections 2024-09-21T06:09:57.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:57.791+0000 7f6082437640 1 -- 172.21.15.31:0/523735497 wait complete. 2024-09-21T06:09:58.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:58 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/523735497' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:58.432 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:09:58.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:09:58.702 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:58 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/523735497' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:58.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:58 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/523735497' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 -- 172.21.15.31:0/1759221693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 msgr2=0x7f85a8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 --2- 172.21.15.31:0/1759221693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a8103f70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009900 tx=0x7f85a402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 -- 172.21.15.31:0/1759221693 shutdown_connections 2024-09-21T06:09:59.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 --2- 172.21.15.31:0/1759221693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 0x7f85a8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 --2- 172.21.15.31:0/1759221693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85a81094a0 0x7f85a81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 --2- 172.21.15.31:0/1759221693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a8103f70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 -- 172.21.15.31:0/1759221693 >> 172.21.15.31:0/1759221693 conn(0x7f85a807b850 msgr2=0x7f85a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 -- 172.21.15.31:0/1759221693 shutdown_connections 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 -- 172.21.15.31:0/1759221693 wait complete. 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.112+0000 7f85b0383640 1 Processor -- start 2024-09-21T06:09:59.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 -- start start 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85a81094a0 0x7f85a819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 0x7f85a819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113ab0 con 0x7f85a8103b60 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113930 con 0x7f85a8109df0 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85b0383640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8113c30 con 0x7f85a81094a0 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37124/0 (socket says 172.21.15.31:37124) 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 -- 172.21.15.31:0/3523046113 learned_addr learned my addr 172.21.15.31:0/3523046113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ad91e640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85a81094a0 0x7f85a819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae920640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 0x7f85a819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 -- 172.21.15.31:0/3523046113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85a81094a0 msgr2=0x7f85a819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85a81094a0 0x7f85a819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 -- 172.21.15.31:0/3523046113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 msgr2=0x7f85a819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 0x7f85a819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 -- 172.21.15.31:0/3523046113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a81a0470 con 0x7f85a8103b60 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae920640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 0x7f85a819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.113+0000 7f85ae11f640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a819a500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f85a4005940 tx=0x7f85a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.114+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a403f030 con 0x7f85a8103b60 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.114+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a81a0700 con 0x7f85a8103b60 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.114+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a4003cb0 con 0x7f85a8103b60 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.114+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81a6ec0 con 0x7f85a8103b60 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.114+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4035070 con 0x7f85a8103b60 2024-09-21T06:09:59.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.115+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8103f70 con 0x7f85a8103b60 2024-09-21T06:09:59.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.116+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f85a4042030 con 0x7f85a8103b60 2024-09-21T06:09:59.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.117+0000 7f859b7fe640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85940b65f0 0x7f85940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:59.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.117+0000 7f85ad91e640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85940b65f0 0x7f85940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:59.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.117+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f85a4108140 con 0x7f85a8103b60 2024-09-21T06:09:59.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.118+0000 7f85ad91e640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85940b65f0 0x7f85940b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f859c005fd0 tx=0x7f859c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:59.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.120+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f85a40ce890 con 0x7f85a8103b60 2024-09-21T06:09:59.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.304+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85a810bc20 con 0x7f85940b65f0 2024-09-21T06:09:59.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.310+0000 7f859b7fe640 1 -- 172.21.15.31:0/3523046113 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258622 (secure 0 0 0) 0x7f85a810bc20 con 0x7f85940b65f0 2024-09-21T06:09:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:09:59.315 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:09:59.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85940b65f0 msgr2=0x7f85940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85940b65f0 0x7f85940b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f859c005fd0 tx=0x7f859c005850 comp rx=0 tx=0).stop 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 msgr2=0x7f85a819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a819a500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f85a4005940 tx=0x7f85a40042f0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 shutdown_connections 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a8109df0 0x7f85a819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85a81094a0 0x7f85a819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85940b65f0 0x7f85940b8b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 --2- 172.21.15.31:0/3523046113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a8103b60 0x7f85a819a500 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 >> 172.21.15.31:0/3523046113 conn(0x7f85a807b850 msgr2=0x7f85a81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 shutdown_connections 2024-09-21T06:09:59.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:09:59.313+0000 7f85b0383640 1 -- 172.21.15.31:0/3523046113 wait complete. 2024-09-21T06:09:59.608 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:09:59 smithi031 ceph-mon[31237]: pgmap v269: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:09:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:09:59 smithi110 ceph-mon[34550]: pgmap v269: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:09:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:09:59 smithi107 ceph-mon[34380]: pgmap v269: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:09:59.982 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-21T06:09:57.598598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636603,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":115,"num_read_kb":132,"num_write":193,"num_write_kb":2218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381516,"kb_used_data":11436,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560308,"statfs":{"total":1151940427776,"available":1151549755392,"internally_reserved":0,"allocated":11710464,"data_stored":6686361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43825,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":54,"num_write":25,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003320"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297440+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:09:37.297440+0000","last_peered":"2024-09-21T06:09:37.297440+0000","last_clean":"2024-09-21T06:09:37.297440+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:09:37.297440+0000","last_undegraded":"2024-09-21T06:09:37.297440+0000","last_fullsized":"2024-09-21T06:09:37.297440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293503+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:09:37.293503+0000","last_peered":"2024-09-21T06:09:37.293503+0000","last_clean":"2024-09-21T06:09:37.293503+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:09:37.293503+0000","last_undegraded":"2024-09-21T06:09:37.293503+0000","last_fullsized":"2024-09-21T06:09:37.293503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293875+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:09:37.293875+0000","last_peered":"2024-09-21T06:09:37.293875+0000","last_clean":"2024-09-21T06:09:37.293875+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:09:37.293875+0000","last_undegraded":"2024-09-21T06:09:37.293875+0000","last_fullsized":"2024-09-21T06:09:37.293875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291553+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:09:37.291553+0000","last_peered":"2024-09-21T06:09:37.291553+0000","last_clean":"2024-09-21T06:09:37.291553+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:09:37.291553+0000","last_undegraded":"2024-09-21T06:09:37.291553+0000","last_fullsized":"2024-09-21T06:09:37.291553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290945+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:09:37.290945+0000","last_peered":"2024-09-21T06:09:37.290945+0000","last_clean":"2024-09-21T06:09:37.290945+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:09:37.290945+0000","last_undegraded":"2024-09-21T06:09:37.290945+0000","last_fullsized":"2024-09-21T06:09:37.290945+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278056+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:09:37.278056+0000","last_peered":"2024-09-21T06:09:37.278056+0000","last_clean":"2024-09-21T06:09:37.278056+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:09:37.278056+0000","last_undegraded":"2024-09-21T06:09:37.278056+0000","last_fullsized":"2024-09-21T06:09:37.278056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293059+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:09:37.293059+0000","last_peered":"2024-09-21T06:09:37.293059+0000","last_clean":"2024-09-21T06:09:37.293059+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:09:37.293059+0000","last_undegraded":"2024-09-21T06:09:37.293059+0000","last_fullsized":"2024-09-21T06:09:37.293059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290688+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:09:37.290688+0000","last_peered":"2024-09-21T06:09:37.290688+0000","last_clean":"2024-09-21T06:09:37.290688+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:09:37.290688+0000","last_undegraded":"2024-09-21T06:09:37.290688+0000","last_fullsized":"2024-09-21T06:09:37.290688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291984+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:09:37.291984+0000","last_peered":"2024-09-21T06:09:37.291984+0000","last_clean":"2024-09-21T06:09:37.291984+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:09:37.291984+0000","last_undegraded":"2024-09-21T06:09:37.291984+0000","last_fullsized":"2024-09-21T06:09:37.291984+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278569+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:09:37.278569+0000","last_peered":"2024-09-21T06:09:37.278569+0000","last_clean":"2024-09-21T06:09:37.278569+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:09:37.278569+0000","last_undegraded":"2024-09-21T06:09:37.278569+0000","last_fullsized":"2024-09-21T06:09:37.278569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293898+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:09:37.293898+0000","last_peered":"2024-09-21T06:09:37.293898+0000","last_clean":"2024-09-21T06:09:37.293898+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:09:37.293898+0000","last_undegraded":"2024-09-21T06:09:37.293898+0000","last_fullsized":"2024-09-21T06:09:37.293898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294902+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:09:37.294902+0000","last_peered":"2024-09-21T06:09:37.294902+0000","last_clean":"2024-09-21T06:09:37.294902+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:09:37.294902+0000","last_undegraded":"2024-09-21T06:09:37.294902+0000","last_fullsized":"2024-09-21T06:09:37.294902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277433+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:09:37.277433+0000","last_peered":"2024-09-21T06:09:37.277433+0000","last_clean":"2024-09-21T06:09:37.277433+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:09:37.277433+0000","last_undegraded":"2024-09-21T06:09:37.277433+0000","last_fullsized":"2024-09-21T06:09:37.277433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293710+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:09:37.293710+0000","last_peered":"2024-09-21T06:09:37.293710+0000","last_clean":"2024-09-21T06:09:37.293710+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:09:37.293710+0000","last_undegraded":"2024-09-21T06:09:37.293710+0000","last_fullsized":"2024-09-21T06:09:37.293710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293369+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:09:37.293369+0000","last_peered":"2024-09-21T06:09:37.293369+0000","last_clean":"2024-09-21T06:09:37.293369+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:09:37.293369+0000","last_undegraded":"2024-09-21T06:09:37.293369+0000","last_fullsized":"2024-09-21T06:09:37.293369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290846+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:09:37.290846+0000","last_peered":"2024-09-21T06:09:37.290846+0000","last_clean":"2024-09-21T06:09:37.290846+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:09:37.290846+0000","last_undegraded":"2024-09-21T06:09:37.290846+0000","last_fullsized":"2024-09-21T06:09:37.290846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297716+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:09:37.297716+0000","last_peered":"2024-09-21T06:09:37.297716+0000","last_clean":"2024-09-21T06:09:37.297716+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:09:37.297716+0000","last_undegraded":"2024-09-21T06:09:37.297716+0000","last_fullsized":"2024-09-21T06:09:37.297716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289137+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:09:37.289137+0000","last_peered":"2024-09-21T06:09:37.289137+0000","last_clean":"2024-09-21T06:09:37.289137+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:09:37.289137+0000","last_undegraded":"2024-09-21T06:09:37.289137+0000","last_fullsized":"2024-09-21T06:09:37.289137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277749+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:09:37.277749+0000","last_peered":"2024-09-21T06:09:37.277749+0000","last_clean":"2024-09-21T06:09:37.277749+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:09:37.277749+0000","last_undegraded":"2024-09-21T06:09:37.277749+0000","last_fullsized":"2024-09-21T06:09:37.277749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.296773+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:09:37.296773+0000","last_peered":"2024-09-21T06:09:37.296773+0000","last_clean":"2024-09-21T06:09:37.296773+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:09:37.296773+0000","last_undegraded":"2024-09-21T06:09:37.296773+0000","last_fullsized":"2024-09-21T06:09:37.296773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277563+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:09:37.277563+0000","last_peered":"2024-09-21T06:09:37.277563+0000","last_clean":"2024-09-21T06:09:37.277563+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:09:37.277563+0000","last_undegraded":"2024-09-21T06:09:37.277563+0000","last_fullsized":"2024-09-21T06:09:37.277563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291461+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:09:37.291461+0000","last_peered":"2024-09-21T06:09:37.291461+0000","last_clean":"2024-09-21T06:09:37.291461+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:09:37.291461+0000","last_undegraded":"2024-09-21T06:09:37.291461+0000","last_fullsized":"2024-09-21T06:09:37.291461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292709+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:09:37.292709+0000","last_peered":"2024-09-21T06:09:37.292709+0000","last_clean":"2024-09-21T06:09:37.292709+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:09:37.292709+0000","last_undegraded":"2024-09-21T06:09:37.292709+0000","last_fullsized":"2024-09-21T06:09:37.292709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290260+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:09:37.290260+0000","last_peered":"2024-09-21T06:09:37.290260+0000","last_clean":"2024-09-21T06:09:37.290260+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:09:37.290260+0000","last_undegraded":"2024-09-21T06:09:37.290260+0000","last_fullsized":"2024-09-21T06:09:37.290260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292847+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:09:37.292847+0000","last_peered":"2024-09-21T06:09:37.292847+0000","last_clean":"2024-09-21T06:09:37.292847+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:09:37.292847+0000","last_undegraded":"2024-09-21T06:09:37.292847+0000","last_fullsized":"2024-09-21T06:09:37.292847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278018+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:09:37.278018+0000","last_peered":"2024-09-21T06:09:37.278018+0000","last_clean":"2024-09-21T06:09:37.278018+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:09:37.278018+0000","last_undegraded":"2024-09-21T06:09:37.278018+0000","last_fullsized":"2024-09-21T06:09:37.278018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295794+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:09:37.295794+0000","last_peered":"2024-09-21T06:09:37.295794+0000","last_clean":"2024-09-21T06:09:37.295794+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:09:37.295794+0000","last_undegraded":"2024-09-21T06:09:37.295794+0000","last_fullsized":"2024-09-21T06:09:37.295794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277080+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:09:37.277080+0000","last_peered":"2024-09-21T06:09:37.277080+0000","last_clean":"2024-09-21T06:09:37.277080+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:09:37.277080+0000","last_undegraded":"2024-09-21T06:09:37.277080+0000","last_fullsized":"2024-09-21T06:09:37.277080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293801+0000","last_change":"2024-09-21T06:09:36.304825+0000","last_active":"2024-09-21T06:09:37.293801+0000","last_peered":"2024-09-21T06:09:37.293801+0000","last_clean":"2024-09-21T06:09:37.293801+0000","last_became_active":"2024-09-21T06:09:36.304293+0000","last_became_peered":"2024-09-21T06:09:36.304293+0000","last_unstale":"2024-09-21T06:09:37.293801+0000","last_undegraded":"2024-09-21T06:09:37.293801+0000","last_fullsized":"2024-09-21T06:09:37.293801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:28.534401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277521+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:09:37.277521+0000","last_peered":"2024-09-21T06:09:37.277521+0000","last_clean":"2024-09-21T06:09:37.277521+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:09:37.277521+0000","last_undegraded":"2024-09-21T06:09:37.277521+0000","last_fullsized":"2024-09-21T06:09:37.277521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277495+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:09:37.277495+0000","last_peered":"2024-09-21T06:09:37.277495+0000","last_clean":"2024-09-21T06:09:37.277495+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:09:37.277495+0000","last_undegraded":"2024-09-21T06:09:37.277495+0000","last_fullsized":"2024-09-21T06:09:37.277495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295966+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:09:37.295966+0000","last_peered":"2024-09-21T06:09:37.295966+0000","last_clean":"2024-09-21T06:09:37.295966+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:09:37.295966+0000","last_undegraded":"2024-09-21T06:09:37.295966+0000","last_fullsized":"2024-09-21T06:09:37.295966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290450+0000","last_change":"2024-09-21T06:09:36.294513+0000","last_active":"2024-09-21T06:09:37.290450+0000","last_peered":"2024-09-21T06:09:37.290450+0000","last_clean":"2024-09-21T06:09:37.290450+0000","last_became_active":"2024-09-21T06:09:36.294180+0000","last_became_peered":"2024-09-21T06:09:36.294180+0000","last_unstale":"2024-09-21T06:09:37.290450+0000","last_undegraded":"2024-09-21T06:09:37.290450+0000","last_fullsized":"2024-09-21T06:09:37.290450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:13.216106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291335+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:09:37.291335+0000","last_peered":"2024-09-21T06:09:37.291335+0000","last_clean":"2024-09-21T06:09:37.291335+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:09:37.291335+0000","last_undegraded":"2024-09-21T06:09:37.291335+0000","last_fullsized":"2024-09-21T06:09:37.291335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291084+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:09:37.291084+0000","last_peered":"2024-09-21T06:09:37.291084+0000","last_clean":"2024-09-21T06:09:37.291084+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:09:37.291084+0000","last_undegraded":"2024-09-21T06:09:37.291084+0000","last_fullsized":"2024-09-21T06:09:37.291084+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278143+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:09:37.278143+0000","last_peered":"2024-09-21T06:09:37.278143+0000","last_clean":"2024-09-21T06:09:37.278143+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:09:37.278143+0000","last_undegraded":"2024-09-21T06:09:37.278143+0000","last_fullsized":"2024-09-21T06:09:37.278143+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.276711+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:09:37.276711+0000","last_peered":"2024-09-21T06:09:37.276711+0000","last_clean":"2024-09-21T06:09:37.276711+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:09:37.276711+0000","last_undegraded":"2024-09-21T06:09:37.276711+0000","last_fullsized":"2024-09-21T06:09:37.276711+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291196+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:09:37.291196+0000","last_peered":"2024-09-21T06:09:37.291196+0000","last_clean":"2024-09-21T06:09:37.291196+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:09:37.291196+0000","last_undegraded":"2024-09-21T06:09:37.291196+0000","last_fullsized":"2024-09-21T06:09:37.291196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290481+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:09:37.290481+0000","last_peered":"2024-09-21T06:09:37.290481+0000","last_clean":"2024-09-21T06:09:37.290481+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:09:37.290481+0000","last_undegraded":"2024-09-21T06:09:37.290481+0000","last_fullsized":"2024-09-21T06:09:37.290481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289782+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:09:37.289782+0000","last_peered":"2024-09-21T06:09:37.289782+0000","last_clean":"2024-09-21T06:09:37.289782+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:09:37.289782+0000","last_undegraded":"2024-09-21T06:09:37.289782+0000","last_fullsized":"2024-09-21T06:09:37.289782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278357+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:09:37.278357+0000","last_peered":"2024-09-21T06:09:37.278357+0000","last_clean":"2024-09-21T06:09:37.278357+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:09:37.278357+0000","last_undegraded":"2024-09-21T06:09:37.278357+0000","last_fullsized":"2024-09-21T06:09:37.278357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291655+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:09:37.291655+0000","last_peered":"2024-09-21T06:09:37.291655+0000","last_clean":"2024-09-21T06:09:37.291655+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:09:37.291655+0000","last_undegraded":"2024-09-21T06:09:37.291655+0000","last_fullsized":"2024-09-21T06:09:37.291655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293666+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:09:37.293666+0000","last_peered":"2024-09-21T06:09:37.293666+0000","last_clean":"2024-09-21T06:09:37.293666+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:09:37.293666+0000","last_undegraded":"2024-09-21T06:09:37.293666+0000","last_fullsized":"2024-09-21T06:09:37.293666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294790+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:09:37.294790+0000","last_peered":"2024-09-21T06:09:37.294790+0000","last_clean":"2024-09-21T06:09:37.294790+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:09:37.294790+0000","last_undegraded":"2024-09-21T06:09:37.294790+0000","last_fullsized":"2024-09-21T06:09:37.294790+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298009+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:09:37.298009+0000","last_peered":"2024-09-21T06:09:37.298009+0000","last_clean":"2024-09-21T06:09:37.298009+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:09:37.298009+0000","last_undegraded":"2024-09-21T06:09:37.298009+0000","last_fullsized":"2024-09-21T06:09:37.298009+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291183+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:09:37.291183+0000","last_peered":"2024-09-21T06:09:37.291183+0000","last_clean":"2024-09-21T06:09:37.291183+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:09:37.291183+0000","last_undegraded":"2024-09-21T06:09:37.291183+0000","last_fullsized":"2024-09-21T06:09:37.291183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'19","reported_seq":66,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:54.338176+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:09:54.338176+0000","last_peered":"2024-09-21T06:09:54.338176+0000","last_clean":"2024-09-21T06:09:54.338176+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:09:54.338176+0000","last_undegraded":"2024-09-21T06:09:54.338176+0000","last_fullsized":"2024-09-21T06:09:54.338176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45203,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":28,"num_read_kb":57,"num_write":19,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291258+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:09:37.291258+0000","last_peered":"2024-09-21T06:09:37.291258+0000","last_clean":"2024-09-21T06:09:37.291258+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:09:37.291258+0000","last_undegraded":"2024-09-21T06:09:37.291258+0000","last_fullsized":"2024-09-21T06:09:37.291258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295239+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:09:37.295239+0000","last_peered":"2024-09-21T06:09:37.295239+0000","last_clean":"2024-09-21T06:09:37.295239+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:09:37.295239+0000","last_undegraded":"2024-09-21T06:09:37.295239+0000","last_fullsized":"2024-09-21T06:09:37.295239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.276779+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:09:37.276779+0000","last_peered":"2024-09-21T06:09:37.276779+0000","last_clean":"2024-09-21T06:09:37.276779+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:09:37.276779+0000","last_undegraded":"2024-09-21T06:09:37.276779+0000","last_fullsized":"2024-09-21T06:09:37.276779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292228+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:09:37.292228+0000","last_peered":"2024-09-21T06:09:37.292228+0000","last_clean":"2024-09-21T06:09:37.292228+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:09:37.292228+0000","last_undegraded":"2024-09-21T06:09:37.292228+0000","last_fullsized":"2024-09-21T06:09:37.292228+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293468+0000","last_change":"2024-09-21T06:09:36.297329+0000","last_active":"2024-09-21T06:09:37.293468+0000","last_peered":"2024-09-21T06:09:37.293468+0000","last_clean":"2024-09-21T06:09:37.293468+0000","last_became_active":"2024-09-21T06:09:36.296336+0000","last_became_peered":"2024-09-21T06:09:36.296336+0000","last_unstale":"2024-09-21T06:09:37.293468+0000","last_undegraded":"2024-09-21T06:09:37.293468+0000","last_fullsized":"2024-09-21T06:09:37.293468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:54.635062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294898+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:09:37.294898+0000","last_peered":"2024-09-21T06:09:37.294898+0000","last_clean":"2024-09-21T06:09:37.294898+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:09:37.294898+0000","last_undegraded":"2024-09-21T06:09:37.294898+0000","last_fullsized":"2024-09-21T06:09:37.294898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292976+0000","last_change":"2024-09-21T06:09:36.295123+0000","last_active":"2024-09-21T06:09:37.292976+0000","last_peered":"2024-09-21T06:09:37.292976+0000","last_clean":"2024-09-21T06:09:37.292976+0000","last_became_active":"2024-09-21T06:09:36.294745+0000","last_became_peered":"2024-09-21T06:09:36.294745+0000","last_unstale":"2024-09-21T06:09:37.292976+0000","last_undegraded":"2024-09-21T06:09:37.292976+0000","last_fullsized":"2024-09-21T06:09:37.292976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:14.735759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298639+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:09:37.298639+0000","last_peered":"2024-09-21T06:09:37.298639+0000","last_clean":"2024-09-21T06:09:37.298639+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:09:37.298639+0000","last_undegraded":"2024-09-21T06:09:37.298639+0000","last_fullsized":"2024-09-21T06:09:37.298639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290565+0000","last_change":"2024-09-21T06:09:36.302929+0000","last_active":"2024-09-21T06:09:37.290565+0000","last_peered":"2024-09-21T06:09:37.290565+0000","last_clean":"2024-09-21T06:09:37.290565+0000","last_became_active":"2024-09-21T06:09:36.301814+0000","last_became_peered":"2024-09-21T06:09:36.301814+0000","last_unstale":"2024-09-21T06:09:37.290565+0000","last_undegraded":"2024-09-21T06:09:37.290565+0000","last_fullsized":"2024-09-21T06:09:37.290565+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:57.281362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298726+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:09:37.298726+0000","last_peered":"2024-09-21T06:09:37.298726+0000","last_clean":"2024-09-21T06:09:37.298726+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:09:37.298726+0000","last_undegraded":"2024-09-21T06:09:37.298726+0000","last_fullsized":"2024-09-21T06:09:37.298726+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295772+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:09:37.295772+0000","last_peered":"2024-09-21T06:09:37.295772+0000","last_clean":"2024-09-21T06:09:37.295772+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:09:37.295772+0000","last_undegraded":"2024-09-21T06:09:37.295772+0000","last_fullsized":"2024-09-21T06:09:37.295772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290524+0000","last_change":"2024-09-21T06:09:34.271629+0000","last_active":"2024-09-21T06:09:37.290524+0000","last_peered":"2024-09-21T06:09:37.290524+0000","last_clean":"2024-09-21T06:09:37.290524+0000","last_became_active":"2024-09-21T06:09:34.271244+0000","last_became_peered":"2024-09-21T06:09:34.271244+0000","last_unstale":"2024-09-21T06:09:37.290524+0000","last_undegraded":"2024-09-21T06:09:37.290524+0000","last_fullsized":"2024-09-21T06:09:37.290524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:25.137668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294232+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:09:37.294232+0000","last_peered":"2024-09-21T06:09:37.294232+0000","last_clean":"2024-09-21T06:09:37.294232+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:09:37.294232+0000","last_undegraded":"2024-09-21T06:09:37.294232+0000","last_fullsized":"2024-09-21T06:09:37.294232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298514+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:09:37.298514+0000","last_peered":"2024-09-21T06:09:37.298514+0000","last_clean":"2024-09-21T06:09:37.298514+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:09:37.298514+0000","last_undegraded":"2024-09-21T06:09:37.298514+0000","last_fullsized":"2024-09-21T06:09:37.298514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293566+0000","last_change":"2024-09-21T06:09:36.293365+0000","last_active":"2024-09-21T06:09:37.293566+0000","last_peered":"2024-09-21T06:09:37.293566+0000","last_clean":"2024-09-21T06:09:37.293566+0000","last_became_active":"2024-09-21T06:09:36.292950+0000","last_became_peered":"2024-09-21T06:09:36.292950+0000","last_unstale":"2024-09-21T06:09:37.293566+0000","last_undegraded":"2024-09-21T06:09:37.293566+0000","last_fullsized":"2024-09-21T06:09:37.293566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:07.361475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277868+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:09:37.277868+0000","last_peered":"2024-09-21T06:09:37.277868+0000","last_clean":"2024-09-21T06:09:37.277868+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:09:37.277868+0000","last_undegraded":"2024-09-21T06:09:37.277868+0000","last_fullsized":"2024-09-21T06:09:37.277868+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293940+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:09:37.293940+0000","last_peered":"2024-09-21T06:09:37.293940+0000","last_clean":"2024-09-21T06:09:37.293940+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:09:37.293940+0000","last_undegraded":"2024-09-21T06:09:37.293940+0000","last_fullsized":"2024-09-21T06:09:37.293940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293176+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:09:37.293176+0000","last_peered":"2024-09-21T06:09:37.293176+0000","last_clean":"2024-09-21T06:09:37.293176+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:09:37.293176+0000","last_undegraded":"2024-09-21T06:09:37.293176+0000","last_fullsized":"2024-09-21T06:09:37.293176+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290380+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:09:37.290380+0000","last_peered":"2024-09-21T06:09:37.290380+0000","last_clean":"2024-09-21T06:09:37.290380+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:09:37.290380+0000","last_undegraded":"2024-09-21T06:09:37.290380+0000","last_fullsized":"2024-09-21T06:09:37.290380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292327+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:09:37.292327+0000","last_peered":"2024-09-21T06:09:37.292327+0000","last_clean":"2024-09-21T06:09:37.292327+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:09:37.292327+0000","last_undegraded":"2024-09-21T06:09:37.292327+0000","last_fullsized":"2024-09-21T06:09:37.292327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293135+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:09:37.293135+0000","last_peered":"2024-09-21T06:09:37.293135+0000","last_clean":"2024-09-21T06:09:37.293135+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:09:37.293135+0000","last_undegraded":"2024-09-21T06:09:37.293135+0000","last_fullsized":"2024-09-21T06:09:37.293135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.276705+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:09:37.276705+0000","last_peered":"2024-09-21T06:09:37.276705+0000","last_clean":"2024-09-21T06:09:37.276705+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:09:37.276705+0000","last_undegraded":"2024-09-21T06:09:37.276705+0000","last_fullsized":"2024-09-21T06:09:37.276705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277007+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:09:37.277007+0000","last_peered":"2024-09-21T06:09:37.277007+0000","last_clean":"2024-09-21T06:09:37.277007+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:09:37.277007+0000","last_undegraded":"2024-09-21T06:09:37.277007+0000","last_fullsized":"2024-09-21T06:09:37.277007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290318+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:09:37.290318+0000","last_peered":"2024-09-21T06:09:37.290318+0000","last_clean":"2024-09-21T06:09:37.290318+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:09:37.290318+0000","last_undegraded":"2024-09-21T06:09:37.290318+0000","last_fullsized":"2024-09-21T06:09:37.290318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294047+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:09:37.294047+0000","last_peered":"2024-09-21T06:09:37.294047+0000","last_clean":"2024-09-21T06:09:37.294047+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:09:37.294047+0000","last_undegraded":"2024-09-21T06:09:37.294047+0000","last_fullsized":"2024-09-21T06:09:37.294047+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278211+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:09:37.278211+0000","last_peered":"2024-09-21T06:09:37.278211+0000","last_clean":"2024-09-21T06:09:37.278211+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:09:37.278211+0000","last_undegraded":"2024-09-21T06:09:37.278211+0000","last_fullsized":"2024-09-21T06:09:37.278211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289472+0000","last_change":"2024-09-21T06:09:34.332630+0000","last_active":"2024-09-21T06:09:37.289472+0000","last_peered":"2024-09-21T06:09:37.289472+0000","last_clean":"2024-09-21T06:09:37.289472+0000","last_became_active":"2024-09-21T06:09:34.332195+0000","last_became_peered":"2024-09-21T06:09:34.332195+0000","last_unstale":"2024-09-21T06:09:37.289472+0000","last_undegraded":"2024-09-21T06:09:37.289472+0000","last_fullsized":"2024-09-21T06:09:37.289472+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:32.639600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:39.646448+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:09:39.646448+0000","last_peered":"2024-09-21T06:09:39.646448+0000","last_clean":"2024-09-21T06:09:39.646448+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:09:39.646448+0000","last_undegraded":"2024-09-21T06:09:39.646448+0000","last_fullsized":"2024-09-21T06:09:39.646448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291699+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:09:37.291699+0000","last_peered":"2024-09-21T06:09:37.291699+0000","last_clean":"2024-09-21T06:09:37.291699+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:09:37.291699+0000","last_undegraded":"2024-09-21T06:09:37.291699+0000","last_fullsized":"2024-09-21T06:09:37.291699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277253+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:09:37.277253+0000","last_peered":"2024-09-21T06:09:37.277253+0000","last_clean":"2024-09-21T06:09:37.277253+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:09:37.277253+0000","last_undegraded":"2024-09-21T06:09:37.277253+0000","last_fullsized":"2024-09-21T06:09:37.277253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'6","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:53.290449+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:09:53.290449+0000","last_peered":"2024-09-21T06:09:53.290449+0000","last_clean":"2024-09-21T06:09:53.290449+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:09:53.290449+0000","last_undegraded":"2024-09-21T06:09:53.290449+0000","last_fullsized":"2024-09-21T06:09:53.290449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293344+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:09:37.293344+0000","last_peered":"2024-09-21T06:09:37.293344+0000","last_clean":"2024-09-21T06:09:37.293344+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:09:37.293344+0000","last_undegraded":"2024-09-21T06:09:37.293344+0000","last_fullsized":"2024-09-21T06:09:37.293344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295306+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:09:37.295306+0000","last_peered":"2024-09-21T06:09:37.295306+0000","last_clean":"2024-09-21T06:09:37.295306+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:09:37.295306+0000","last_undegraded":"2024-09-21T06:09:37.295306+0000","last_fullsized":"2024-09-21T06:09:37.295306+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294593+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:09:37.294593+0000","last_peered":"2024-09-21T06:09:37.294593+0000","last_clean":"2024-09-21T06:09:37.294593+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:09:37.294593+0000","last_undegraded":"2024-09-21T06:09:37.294593+0000","last_fullsized":"2024-09-21T06:09:37.294593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291084+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:09:37.291084+0000","last_peered":"2024-09-21T06:09:37.291084+0000","last_clean":"2024-09-21T06:09:37.291084+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:09:37.291084+0000","last_undegraded":"2024-09-21T06:09:37.291084+0000","last_fullsized":"2024-09-21T06:09:37.291084+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289730+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:09:37.289730+0000","last_peered":"2024-09-21T06:09:37.289730+0000","last_clean":"2024-09-21T06:09:37.289730+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:09:37.289730+0000","last_undegraded":"2024-09-21T06:09:37.289730+0000","last_fullsized":"2024-09-21T06:09:37.289730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292053+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:09:37.292053+0000","last_peered":"2024-09-21T06:09:37.292053+0000","last_clean":"2024-09-21T06:09:37.292053+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:09:37.292053+0000","last_undegraded":"2024-09-21T06:09:37.292053+0000","last_fullsized":"2024-09-21T06:09:37.292053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.296230+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:09:37.296230+0000","last_peered":"2024-09-21T06:09:37.296230+0000","last_clean":"2024-09-21T06:09:37.296230+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:09:37.296230+0000","last_undegraded":"2024-09-21T06:09:37.296230+0000","last_fullsized":"2024-09-21T06:09:37.296230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277540+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:09:37.277540+0000","last_peered":"2024-09-21T06:09:37.277540+0000","last_clean":"2024-09-21T06:09:37.277540+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:09:37.277540+0000","last_undegraded":"2024-09-21T06:09:37.277540+0000","last_fullsized":"2024-09-21T06:09:37.277540+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290989+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:09:37.290989+0000","last_peered":"2024-09-21T06:09:37.290989+0000","last_clean":"2024-09-21T06:09:37.290989+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:09:37.290989+0000","last_undegraded":"2024-09-21T06:09:37.290989+0000","last_fullsized":"2024-09-21T06:09:37.290989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298233+0000","last_change":"2024-09-21T06:09:34.271214+0000","last_active":"2024-09-21T06:09:37.298233+0000","last_peered":"2024-09-21T06:09:37.298233+0000","last_clean":"2024-09-21T06:09:37.298233+0000","last_became_active":"2024-09-21T06:09:34.270756+0000","last_became_peered":"2024-09-21T06:09:34.270756+0000","last_unstale":"2024-09-21T06:09:37.298233+0000","last_undegraded":"2024-09-21T06:09:37.298233+0000","last_fullsized":"2024-09-21T06:09:37.298233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:38.175845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293584+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:09:37.293584+0000","last_peered":"2024-09-21T06:09:37.293584+0000","last_clean":"2024-09-21T06:09:37.293584+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:09:37.293584+0000","last_undegraded":"2024-09-21T06:09:37.293584+0000","last_fullsized":"2024-09-21T06:09:37.293584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293496+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:09:37.293496+0000","last_peered":"2024-09-21T06:09:37.293496+0000","last_clean":"2024-09-21T06:09:37.293496+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:09:37.293496+0000","last_undegraded":"2024-09-21T06:09:37.293496+0000","last_fullsized":"2024-09-21T06:09:37.293496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297666+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:09:37.297666+0000","last_peered":"2024-09-21T06:09:37.297666+0000","last_clean":"2024-09-21T06:09:37.297666+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:09:37.297666+0000","last_undegraded":"2024-09-21T06:09:37.297666+0000","last_fullsized":"2024-09-21T06:09:37.297666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291032+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:09:37.291032+0000","last_peered":"2024-09-21T06:09:37.291032+0000","last_clean":"2024-09-21T06:09:37.291032+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:09:37.291032+0000","last_undegraded":"2024-09-21T06:09:37.291032+0000","last_fullsized":"2024-09-21T06:09:37.291032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298128+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:09:37.298128+0000","last_peered":"2024-09-21T06:09:37.298128+0000","last_clean":"2024-09-21T06:09:37.298128+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:09:37.298128+0000","last_undegraded":"2024-09-21T06:09:37.298128+0000","last_fullsized":"2024-09-21T06:09:37.298128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294766+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:09:37.294766+0000","last_peered":"2024-09-21T06:09:37.294766+0000","last_clean":"2024-09-21T06:09:37.294766+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:09:37.294766+0000","last_undegraded":"2024-09-21T06:09:37.294766+0000","last_fullsized":"2024-09-21T06:09:37.294766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292737+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:09:37.292737+0000","last_peered":"2024-09-21T06:09:37.292737+0000","last_clean":"2024-09-21T06:09:37.292737+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:09:37.292737+0000","last_undegraded":"2024-09-21T06:09:37.292737+0000","last_fullsized":"2024-09-21T06:09:37.292737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291579+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:09:37.291579+0000","last_peered":"2024-09-21T06:09:37.291579+0000","last_clean":"2024-09-21T06:09:37.291579+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:09:37.291579+0000","last_undegraded":"2024-09-21T06:09:37.291579+0000","last_fullsized":"2024-09-21T06:09:37.291579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297752+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:09:37.297752+0000","last_peered":"2024-09-21T06:09:37.297752+0000","last_clean":"2024-09-21T06:09:37.297752+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:09:37.297752+0000","last_undegraded":"2024-09-21T06:09:37.297752+0000","last_fullsized":"2024-09-21T06:09:37.297752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46001,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":34,"num_read_kb":63,"num_write":68,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":138003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":806912,"data_stored":398046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1470464,"data_stored":1033928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103194,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":398628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266719,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":815104,"data_stored":398650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1409024,"data_stored":989113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":398163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1417216,"data_stored":988713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":811008,"data_stored":398068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":398384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378685,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40348,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704804,"statfs":{"total":95995035648,"available":95953719296,"internally_reserved":0,"allocated":864256,"data_stored":443283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":443249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":811008,"data_stored":398136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":45203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":45237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:09:59.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:10:00.247 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:00.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:00 smithi031 ceph-mon[31237]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:00.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:00 smithi031 ceph-mon[31237]: overall HEALTH_OK 2024-09-21T06:10:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:00 smithi110 ceph-mon[34550]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:00 smithi110 ceph-mon[34550]: overall HEALTH_OK 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 -- 172.21.15.31:0/15903563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 msgr2=0x7ff6a41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/15903563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a41087e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 -- 172.21.15.31:0/15903563 shutdown_connections 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/15903563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4108d20 0x7ff6a410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/15903563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a41087e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/15903563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4102a90 0x7ff6a4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 -- 172.21.15.31:0/15903563 >> 172.21.15.31:0/15903563 conn(0x7ff6a4079750 msgr2=0x7ff6a4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:00.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 -- 172.21.15.31:0/15903563 shutdown_connections 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.746+0000 7ff6a91cd640 1 -- 172.21.15.31:0/15903563 wait complete. 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 Processor -- start 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 -- start start 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4102a90 0x7ff6a410ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a4110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4108d20 0x7ff6a4110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4113ab0 con 0x7ff6a41083d0 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4113930 con 0x7ff6a4108d20 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a91cd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a4113c30 con 0x7ff6a4102a90 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff69a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a4110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff69a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a4110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37130/0 (socket says 172.21.15.31:37130) 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff69a59c640 1 -- 172.21.15.31:0/2767270600 learned_addr learned my addr 172.21.15.31:0/2767270600 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a2d9d640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4102a90 0x7ff6a410ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff69a59c640 1 -- 172.21.15.31:0/2767270600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4102a90 msgr2=0x7ff6a410ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.747+0000 7ff6a359e640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4108d20 0x7ff6a4110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff69a59c640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4102a90 0x7ff6a410ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff69a59c640 1 -- 172.21.15.31:0/2767270600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4108d20 msgr2=0x7ff6a4110a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff69a59c640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4108d20 0x7ff6a4110a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff69a59c640 1 -- 172.21.15.31:0/2767270600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a41a8740 con 0x7ff6a41083d0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a2d9d640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4102a90 0x7ff6a410ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a359e640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4108d20 0x7ff6a4110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff69a59c640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a4110520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff68c032fd0 tx=0x7ff68c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c03f030 con 0x7ff6a41083d0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a41a89d0 con 0x7ff6a41083d0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff68c004490 con 0x7ff6a41083d0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c035070 con 0x7ff6a41083d0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.748+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a41a8f00 con 0x7ff6a41083d0 2024-09-21T06:10:00.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.749+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a4103d10 con 0x7ff6a41083d0 2024-09-21T06:10:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.751+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff68c042030 con 0x7ff6a41083d0 2024-09-21T06:10:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.751+0000 7ff6a0d99640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff6900b65f0 0x7ff6900b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.751+0000 7ff6a2d9d640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff6900b65f0 0x7ff6900b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.751+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7ff68c107f10 con 0x7ff6a41083d0 2024-09-21T06:10:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.752+0000 7ff6a2d9d640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff6900b65f0 0x7ff6900b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff694005d00 tx=0x7ff694005ef0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:00.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.754+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff68c0ce660 con 0x7ff6a41083d0 2024-09-21T06:10:00.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:00 smithi107 ceph-mon[34380]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:00.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:00 smithi107 ceph-mon[34380]: overall HEALTH_OK 2024-09-21T06:10:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.940+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6a4063800 con 0x7ff6900b65f0 2024-09-21T06:10:00.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.945+0000 7ff6a0d99640 1 -- 172.21.15.31:0/2767270600 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258622 (secure 0 0 0) 0x7ff6a4063800 con 0x7ff6900b65f0 2024-09-21T06:10:00.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:00.949 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:10:00.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff6900b65f0 msgr2=0x7ff6900b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff6900b65f0 0x7ff6900b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff694005d00 tx=0x7ff694005ef0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 msgr2=0x7ff6a4110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a4110520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff68c032fd0 tx=0x7ff68c004270 comp rx=0 tx=0).stop 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 shutdown_connections 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6a4108d20 0x7ff6a4110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff6900b65f0 0x7ff6900b8b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6a41083d0 0x7ff6a4110520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 --2- 172.21.15.31:0/2767270600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6a4102a90 0x7ff6a410ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.947+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 >> 172.21.15.31:0/2767270600 conn(0x7ff6a4079750 msgr2=0x7ff6a4105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.948+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 shutdown_connections 2024-09-21T06:10:00.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:00.948+0000 7ff6a91cd640 1 -- 172.21.15.31:0/2767270600 wait complete. 2024-09-21T06:10:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:01 smithi031 ceph-mon[31237]: pgmap v270: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:10:01.755 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-21T06:09:59.599185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636603,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":115,"num_read_kb":132,"num_write":193,"num_write_kb":2218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381516,"kb_used_data":11436,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560308,"statfs":{"total":1151940427776,"available":1151549755392,"internally_reserved":0,"allocated":11710464,"data_stored":6686361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43825,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":54,"num_write":25,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003348"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297440+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:09:37.297440+0000","last_peered":"2024-09-21T06:09:37.297440+0000","last_clean":"2024-09-21T06:09:37.297440+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:09:37.297440+0000","last_undegraded":"2024-09-21T06:09:37.297440+0000","last_fullsized":"2024-09-21T06:09:37.297440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293503+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:09:37.293503+0000","last_peered":"2024-09-21T06:09:37.293503+0000","last_clean":"2024-09-21T06:09:37.293503+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:09:37.293503+0000","last_undegraded":"2024-09-21T06:09:37.293503+0000","last_fullsized":"2024-09-21T06:09:37.293503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293875+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:09:37.293875+0000","last_peered":"2024-09-21T06:09:37.293875+0000","last_clean":"2024-09-21T06:09:37.293875+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:09:37.293875+0000","last_undegraded":"2024-09-21T06:09:37.293875+0000","last_fullsized":"2024-09-21T06:09:37.293875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291553+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:09:37.291553+0000","last_peered":"2024-09-21T06:09:37.291553+0000","last_clean":"2024-09-21T06:09:37.291553+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:09:37.291553+0000","last_undegraded":"2024-09-21T06:09:37.291553+0000","last_fullsized":"2024-09-21T06:09:37.291553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290945+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:09:37.290945+0000","last_peered":"2024-09-21T06:09:37.290945+0000","last_clean":"2024-09-21T06:09:37.290945+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:09:37.290945+0000","last_undegraded":"2024-09-21T06:09:37.290945+0000","last_fullsized":"2024-09-21T06:09:37.290945+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278056+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:09:37.278056+0000","last_peered":"2024-09-21T06:09:37.278056+0000","last_clean":"2024-09-21T06:09:37.278056+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:09:37.278056+0000","last_undegraded":"2024-09-21T06:09:37.278056+0000","last_fullsized":"2024-09-21T06:09:37.278056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293059+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:09:37.293059+0000","last_peered":"2024-09-21T06:09:37.293059+0000","last_clean":"2024-09-21T06:09:37.293059+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:09:37.293059+0000","last_undegraded":"2024-09-21T06:09:37.293059+0000","last_fullsized":"2024-09-21T06:09:37.293059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290688+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:09:37.290688+0000","last_peered":"2024-09-21T06:09:37.290688+0000","last_clean":"2024-09-21T06:09:37.290688+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:09:37.290688+0000","last_undegraded":"2024-09-21T06:09:37.290688+0000","last_fullsized":"2024-09-21T06:09:37.290688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291984+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:09:37.291984+0000","last_peered":"2024-09-21T06:09:37.291984+0000","last_clean":"2024-09-21T06:09:37.291984+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:09:37.291984+0000","last_undegraded":"2024-09-21T06:09:37.291984+0000","last_fullsized":"2024-09-21T06:09:37.291984+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278569+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:09:37.278569+0000","last_peered":"2024-09-21T06:09:37.278569+0000","last_clean":"2024-09-21T06:09:37.278569+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:09:37.278569+0000","last_undegraded":"2024-09-21T06:09:37.278569+0000","last_fullsized":"2024-09-21T06:09:37.278569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293898+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:09:37.293898+0000","last_peered":"2024-09-21T06:09:37.293898+0000","last_clean":"2024-09-21T06:09:37.293898+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:09:37.293898+0000","last_undegraded":"2024-09-21T06:09:37.293898+0000","last_fullsized":"2024-09-21T06:09:37.293898+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294902+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:09:37.294902+0000","last_peered":"2024-09-21T06:09:37.294902+0000","last_clean":"2024-09-21T06:09:37.294902+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:09:37.294902+0000","last_undegraded":"2024-09-21T06:09:37.294902+0000","last_fullsized":"2024-09-21T06:09:37.294902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277433+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:09:37.277433+0000","last_peered":"2024-09-21T06:09:37.277433+0000","last_clean":"2024-09-21T06:09:37.277433+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:09:37.277433+0000","last_undegraded":"2024-09-21T06:09:37.277433+0000","last_fullsized":"2024-09-21T06:09:37.277433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293710+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:09:37.293710+0000","last_peered":"2024-09-21T06:09:37.293710+0000","last_clean":"2024-09-21T06:09:37.293710+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:09:37.293710+0000","last_undegraded":"2024-09-21T06:09:37.293710+0000","last_fullsized":"2024-09-21T06:09:37.293710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293369+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:09:37.293369+0000","last_peered":"2024-09-21T06:09:37.293369+0000","last_clean":"2024-09-21T06:09:37.293369+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:09:37.293369+0000","last_undegraded":"2024-09-21T06:09:37.293369+0000","last_fullsized":"2024-09-21T06:09:37.293369+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290846+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:09:37.290846+0000","last_peered":"2024-09-21T06:09:37.290846+0000","last_clean":"2024-09-21T06:09:37.290846+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:09:37.290846+0000","last_undegraded":"2024-09-21T06:09:37.290846+0000","last_fullsized":"2024-09-21T06:09:37.290846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297716+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:09:37.297716+0000","last_peered":"2024-09-21T06:09:37.297716+0000","last_clean":"2024-09-21T06:09:37.297716+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:09:37.297716+0000","last_undegraded":"2024-09-21T06:09:37.297716+0000","last_fullsized":"2024-09-21T06:09:37.297716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289137+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:09:37.289137+0000","last_peered":"2024-09-21T06:09:37.289137+0000","last_clean":"2024-09-21T06:09:37.289137+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:09:37.289137+0000","last_undegraded":"2024-09-21T06:09:37.289137+0000","last_fullsized":"2024-09-21T06:09:37.289137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277749+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:09:37.277749+0000","last_peered":"2024-09-21T06:09:37.277749+0000","last_clean":"2024-09-21T06:09:37.277749+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:09:37.277749+0000","last_undegraded":"2024-09-21T06:09:37.277749+0000","last_fullsized":"2024-09-21T06:09:37.277749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.296773+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:09:37.296773+0000","last_peered":"2024-09-21T06:09:37.296773+0000","last_clean":"2024-09-21T06:09:37.296773+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:09:37.296773+0000","last_undegraded":"2024-09-21T06:09:37.296773+0000","last_fullsized":"2024-09-21T06:09:37.296773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277563+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:09:37.277563+0000","last_peered":"2024-09-21T06:09:37.277563+0000","last_clean":"2024-09-21T06:09:37.277563+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:09:37.277563+0000","last_undegraded":"2024-09-21T06:09:37.277563+0000","last_fullsized":"2024-09-21T06:09:37.277563+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291461+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:09:37.291461+0000","last_peered":"2024-09-21T06:09:37.291461+0000","last_clean":"2024-09-21T06:09:37.291461+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:09:37.291461+0000","last_undegraded":"2024-09-21T06:09:37.291461+0000","last_fullsized":"2024-09-21T06:09:37.291461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292709+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:09:37.292709+0000","last_peered":"2024-09-21T06:09:37.292709+0000","last_clean":"2024-09-21T06:09:37.292709+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:09:37.292709+0000","last_undegraded":"2024-09-21T06:09:37.292709+0000","last_fullsized":"2024-09-21T06:09:37.292709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290260+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:09:37.290260+0000","last_peered":"2024-09-21T06:09:37.290260+0000","last_clean":"2024-09-21T06:09:37.290260+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:09:37.290260+0000","last_undegraded":"2024-09-21T06:09:37.290260+0000","last_fullsized":"2024-09-21T06:09:37.290260+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292847+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:09:37.292847+0000","last_peered":"2024-09-21T06:09:37.292847+0000","last_clean":"2024-09-21T06:09:37.292847+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:09:37.292847+0000","last_undegraded":"2024-09-21T06:09:37.292847+0000","last_fullsized":"2024-09-21T06:09:37.292847+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278018+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:09:37.278018+0000","last_peered":"2024-09-21T06:09:37.278018+0000","last_clean":"2024-09-21T06:09:37.278018+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:09:37.278018+0000","last_undegraded":"2024-09-21T06:09:37.278018+0000","last_fullsized":"2024-09-21T06:09:37.278018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295794+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:09:37.295794+0000","last_peered":"2024-09-21T06:09:37.295794+0000","last_clean":"2024-09-21T06:09:37.295794+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:09:37.295794+0000","last_undegraded":"2024-09-21T06:09:37.295794+0000","last_fullsized":"2024-09-21T06:09:37.295794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277080+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:09:37.277080+0000","last_peered":"2024-09-21T06:09:37.277080+0000","last_clean":"2024-09-21T06:09:37.277080+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:09:37.277080+0000","last_undegraded":"2024-09-21T06:09:37.277080+0000","last_fullsized":"2024-09-21T06:09:37.277080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293801+0000","last_change":"2024-09-21T06:09:36.304825+0000","last_active":"2024-09-21T06:09:37.293801+0000","last_peered":"2024-09-21T06:09:37.293801+0000","last_clean":"2024-09-21T06:09:37.293801+0000","last_became_active":"2024-09-21T06:09:36.304293+0000","last_became_peered":"2024-09-21T06:09:36.304293+0000","last_unstale":"2024-09-21T06:09:37.293801+0000","last_undegraded":"2024-09-21T06:09:37.293801+0000","last_fullsized":"2024-09-21T06:09:37.293801+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:28.534401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277521+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:09:37.277521+0000","last_peered":"2024-09-21T06:09:37.277521+0000","last_clean":"2024-09-21T06:09:37.277521+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:09:37.277521+0000","last_undegraded":"2024-09-21T06:09:37.277521+0000","last_fullsized":"2024-09-21T06:09:37.277521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277495+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:09:37.277495+0000","last_peered":"2024-09-21T06:09:37.277495+0000","last_clean":"2024-09-21T06:09:37.277495+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:09:37.277495+0000","last_undegraded":"2024-09-21T06:09:37.277495+0000","last_fullsized":"2024-09-21T06:09:37.277495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295966+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:09:37.295966+0000","last_peered":"2024-09-21T06:09:37.295966+0000","last_clean":"2024-09-21T06:09:37.295966+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:09:37.295966+0000","last_undegraded":"2024-09-21T06:09:37.295966+0000","last_fullsized":"2024-09-21T06:09:37.295966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290450+0000","last_change":"2024-09-21T06:09:36.294513+0000","last_active":"2024-09-21T06:09:37.290450+0000","last_peered":"2024-09-21T06:09:37.290450+0000","last_clean":"2024-09-21T06:09:37.290450+0000","last_became_active":"2024-09-21T06:09:36.294180+0000","last_became_peered":"2024-09-21T06:09:36.294180+0000","last_unstale":"2024-09-21T06:09:37.290450+0000","last_undegraded":"2024-09-21T06:09:37.290450+0000","last_fullsized":"2024-09-21T06:09:37.290450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:13.216106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291335+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:09:37.291335+0000","last_peered":"2024-09-21T06:09:37.291335+0000","last_clean":"2024-09-21T06:09:37.291335+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:09:37.291335+0000","last_undegraded":"2024-09-21T06:09:37.291335+0000","last_fullsized":"2024-09-21T06:09:37.291335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291084+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:09:37.291084+0000","last_peered":"2024-09-21T06:09:37.291084+0000","last_clean":"2024-09-21T06:09:37.291084+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:09:37.291084+0000","last_undegraded":"2024-09-21T06:09:37.291084+0000","last_fullsized":"2024-09-21T06:09:37.291084+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278143+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:09:37.278143+0000","last_peered":"2024-09-21T06:09:37.278143+0000","last_clean":"2024-09-21T06:09:37.278143+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:09:37.278143+0000","last_undegraded":"2024-09-21T06:09:37.278143+0000","last_fullsized":"2024-09-21T06:09:37.278143+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.276711+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:09:37.276711+0000","last_peered":"2024-09-21T06:09:37.276711+0000","last_clean":"2024-09-21T06:09:37.276711+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:09:37.276711+0000","last_undegraded":"2024-09-21T06:09:37.276711+0000","last_fullsized":"2024-09-21T06:09:37.276711+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291196+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:09:37.291196+0000","last_peered":"2024-09-21T06:09:37.291196+0000","last_clean":"2024-09-21T06:09:37.291196+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:09:37.291196+0000","last_undegraded":"2024-09-21T06:09:37.291196+0000","last_fullsized":"2024-09-21T06:09:37.291196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290481+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:09:37.290481+0000","last_peered":"2024-09-21T06:09:37.290481+0000","last_clean":"2024-09-21T06:09:37.290481+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:09:37.290481+0000","last_undegraded":"2024-09-21T06:09:37.290481+0000","last_fullsized":"2024-09-21T06:09:37.290481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289782+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:09:37.289782+0000","last_peered":"2024-09-21T06:09:37.289782+0000","last_clean":"2024-09-21T06:09:37.289782+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:09:37.289782+0000","last_undegraded":"2024-09-21T06:09:37.289782+0000","last_fullsized":"2024-09-21T06:09:37.289782+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278357+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:09:37.278357+0000","last_peered":"2024-09-21T06:09:37.278357+0000","last_clean":"2024-09-21T06:09:37.278357+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:09:37.278357+0000","last_undegraded":"2024-09-21T06:09:37.278357+0000","last_fullsized":"2024-09-21T06:09:37.278357+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291655+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:09:37.291655+0000","last_peered":"2024-09-21T06:09:37.291655+0000","last_clean":"2024-09-21T06:09:37.291655+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:09:37.291655+0000","last_undegraded":"2024-09-21T06:09:37.291655+0000","last_fullsized":"2024-09-21T06:09:37.291655+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293666+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:09:37.293666+0000","last_peered":"2024-09-21T06:09:37.293666+0000","last_clean":"2024-09-21T06:09:37.293666+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:09:37.293666+0000","last_undegraded":"2024-09-21T06:09:37.293666+0000","last_fullsized":"2024-09-21T06:09:37.293666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294790+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:09:37.294790+0000","last_peered":"2024-09-21T06:09:37.294790+0000","last_clean":"2024-09-21T06:09:37.294790+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:09:37.294790+0000","last_undegraded":"2024-09-21T06:09:37.294790+0000","last_fullsized":"2024-09-21T06:09:37.294790+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298009+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:09:37.298009+0000","last_peered":"2024-09-21T06:09:37.298009+0000","last_clean":"2024-09-21T06:09:37.298009+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:09:37.298009+0000","last_undegraded":"2024-09-21T06:09:37.298009+0000","last_fullsized":"2024-09-21T06:09:37.298009+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291183+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:09:37.291183+0000","last_peered":"2024-09-21T06:09:37.291183+0000","last_clean":"2024-09-21T06:09:37.291183+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:09:37.291183+0000","last_undegraded":"2024-09-21T06:09:37.291183+0000","last_fullsized":"2024-09-21T06:09:37.291183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'19","reported_seq":66,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:54.338176+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:09:54.338176+0000","last_peered":"2024-09-21T06:09:54.338176+0000","last_clean":"2024-09-21T06:09:54.338176+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:09:54.338176+0000","last_undegraded":"2024-09-21T06:09:54.338176+0000","last_fullsized":"2024-09-21T06:09:54.338176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45203,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":28,"num_read_kb":57,"num_write":19,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291258+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:09:37.291258+0000","last_peered":"2024-09-21T06:09:37.291258+0000","last_clean":"2024-09-21T06:09:37.291258+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:09:37.291258+0000","last_undegraded":"2024-09-21T06:09:37.291258+0000","last_fullsized":"2024-09-21T06:09:37.291258+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295239+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:09:37.295239+0000","last_peered":"2024-09-21T06:09:37.295239+0000","last_clean":"2024-09-21T06:09:37.295239+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:09:37.295239+0000","last_undegraded":"2024-09-21T06:09:37.295239+0000","last_fullsized":"2024-09-21T06:09:37.295239+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.276779+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:09:37.276779+0000","last_peered":"2024-09-21T06:09:37.276779+0000","last_clean":"2024-09-21T06:09:37.276779+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:09:37.276779+0000","last_undegraded":"2024-09-21T06:09:37.276779+0000","last_fullsized":"2024-09-21T06:09:37.276779+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292228+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:09:37.292228+0000","last_peered":"2024-09-21T06:09:37.292228+0000","last_clean":"2024-09-21T06:09:37.292228+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:09:37.292228+0000","last_undegraded":"2024-09-21T06:09:37.292228+0000","last_fullsized":"2024-09-21T06:09:37.292228+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293468+0000","last_change":"2024-09-21T06:09:36.297329+0000","last_active":"2024-09-21T06:09:37.293468+0000","last_peered":"2024-09-21T06:09:37.293468+0000","last_clean":"2024-09-21T06:09:37.293468+0000","last_became_active":"2024-09-21T06:09:36.296336+0000","last_became_peered":"2024-09-21T06:09:36.296336+0000","last_unstale":"2024-09-21T06:09:37.293468+0000","last_undegraded":"2024-09-21T06:09:37.293468+0000","last_fullsized":"2024-09-21T06:09:37.293468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:54.635062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294898+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:09:37.294898+0000","last_peered":"2024-09-21T06:09:37.294898+0000","last_clean":"2024-09-21T06:09:37.294898+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:09:37.294898+0000","last_undegraded":"2024-09-21T06:09:37.294898+0000","last_fullsized":"2024-09-21T06:09:37.294898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292976+0000","last_change":"2024-09-21T06:09:36.295123+0000","last_active":"2024-09-21T06:09:37.292976+0000","last_peered":"2024-09-21T06:09:37.292976+0000","last_clean":"2024-09-21T06:09:37.292976+0000","last_became_active":"2024-09-21T06:09:36.294745+0000","last_became_peered":"2024-09-21T06:09:36.294745+0000","last_unstale":"2024-09-21T06:09:37.292976+0000","last_undegraded":"2024-09-21T06:09:37.292976+0000","last_fullsized":"2024-09-21T06:09:37.292976+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:14.735759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298639+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:09:37.298639+0000","last_peered":"2024-09-21T06:09:37.298639+0000","last_clean":"2024-09-21T06:09:37.298639+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:09:37.298639+0000","last_undegraded":"2024-09-21T06:09:37.298639+0000","last_fullsized":"2024-09-21T06:09:37.298639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290565+0000","last_change":"2024-09-21T06:09:36.302929+0000","last_active":"2024-09-21T06:09:37.290565+0000","last_peered":"2024-09-21T06:09:37.290565+0000","last_clean":"2024-09-21T06:09:37.290565+0000","last_became_active":"2024-09-21T06:09:36.301814+0000","last_became_peered":"2024-09-21T06:09:36.301814+0000","last_unstale":"2024-09-21T06:09:37.290565+0000","last_undegraded":"2024-09-21T06:09:37.290565+0000","last_fullsized":"2024-09-21T06:09:37.290565+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:57.281362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298726+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:09:37.298726+0000","last_peered":"2024-09-21T06:09:37.298726+0000","last_clean":"2024-09-21T06:09:37.298726+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:09:37.298726+0000","last_undegraded":"2024-09-21T06:09:37.298726+0000","last_fullsized":"2024-09-21T06:09:37.298726+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295772+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:09:37.295772+0000","last_peered":"2024-09-21T06:09:37.295772+0000","last_clean":"2024-09-21T06:09:37.295772+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:09:37.295772+0000","last_undegraded":"2024-09-21T06:09:37.295772+0000","last_fullsized":"2024-09-21T06:09:37.295772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290524+0000","last_change":"2024-09-21T06:09:34.271629+0000","last_active":"2024-09-21T06:09:37.290524+0000","last_peered":"2024-09-21T06:09:37.290524+0000","last_clean":"2024-09-21T06:09:37.290524+0000","last_became_active":"2024-09-21T06:09:34.271244+0000","last_became_peered":"2024-09-21T06:09:34.271244+0000","last_unstale":"2024-09-21T06:09:37.290524+0000","last_undegraded":"2024-09-21T06:09:37.290524+0000","last_fullsized":"2024-09-21T06:09:37.290524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:25.137668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294232+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:09:37.294232+0000","last_peered":"2024-09-21T06:09:37.294232+0000","last_clean":"2024-09-21T06:09:37.294232+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:09:37.294232+0000","last_undegraded":"2024-09-21T06:09:37.294232+0000","last_fullsized":"2024-09-21T06:09:37.294232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298514+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:09:37.298514+0000","last_peered":"2024-09-21T06:09:37.298514+0000","last_clean":"2024-09-21T06:09:37.298514+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:09:37.298514+0000","last_undegraded":"2024-09-21T06:09:37.298514+0000","last_fullsized":"2024-09-21T06:09:37.298514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293566+0000","last_change":"2024-09-21T06:09:36.293365+0000","last_active":"2024-09-21T06:09:37.293566+0000","last_peered":"2024-09-21T06:09:37.293566+0000","last_clean":"2024-09-21T06:09:37.293566+0000","last_became_active":"2024-09-21T06:09:36.292950+0000","last_became_peered":"2024-09-21T06:09:36.292950+0000","last_unstale":"2024-09-21T06:09:37.293566+0000","last_undegraded":"2024-09-21T06:09:37.293566+0000","last_fullsized":"2024-09-21T06:09:37.293566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:07.361475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277868+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:09:37.277868+0000","last_peered":"2024-09-21T06:09:37.277868+0000","last_clean":"2024-09-21T06:09:37.277868+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:09:37.277868+0000","last_undegraded":"2024-09-21T06:09:37.277868+0000","last_fullsized":"2024-09-21T06:09:37.277868+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293940+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:09:37.293940+0000","last_peered":"2024-09-21T06:09:37.293940+0000","last_clean":"2024-09-21T06:09:37.293940+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:09:37.293940+0000","last_undegraded":"2024-09-21T06:09:37.293940+0000","last_fullsized":"2024-09-21T06:09:37.293940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":90,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293176+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:09:37.293176+0000","last_peered":"2024-09-21T06:09:37.293176+0000","last_clean":"2024-09-21T06:09:37.293176+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:09:37.293176+0000","last_undegraded":"2024-09-21T06:09:37.293176+0000","last_fullsized":"2024-09-21T06:09:37.293176+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290380+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:09:37.290380+0000","last_peered":"2024-09-21T06:09:37.290380+0000","last_clean":"2024-09-21T06:09:37.290380+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:09:37.290380+0000","last_undegraded":"2024-09-21T06:09:37.290380+0000","last_fullsized":"2024-09-21T06:09:37.290380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292327+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:09:37.292327+0000","last_peered":"2024-09-21T06:09:37.292327+0000","last_clean":"2024-09-21T06:09:37.292327+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:09:37.292327+0000","last_undegraded":"2024-09-21T06:09:37.292327+0000","last_fullsized":"2024-09-21T06:09:37.292327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293135+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:09:37.293135+0000","last_peered":"2024-09-21T06:09:37.293135+0000","last_clean":"2024-09-21T06:09:37.293135+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:09:37.293135+0000","last_undegraded":"2024-09-21T06:09:37.293135+0000","last_fullsized":"2024-09-21T06:09:37.293135+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.276705+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:09:37.276705+0000","last_peered":"2024-09-21T06:09:37.276705+0000","last_clean":"2024-09-21T06:09:37.276705+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:09:37.276705+0000","last_undegraded":"2024-09-21T06:09:37.276705+0000","last_fullsized":"2024-09-21T06:09:37.276705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277007+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:09:37.277007+0000","last_peered":"2024-09-21T06:09:37.277007+0000","last_clean":"2024-09-21T06:09:37.277007+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:09:37.277007+0000","last_undegraded":"2024-09-21T06:09:37.277007+0000","last_fullsized":"2024-09-21T06:09:37.277007+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290318+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:09:37.290318+0000","last_peered":"2024-09-21T06:09:37.290318+0000","last_clean":"2024-09-21T06:09:37.290318+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:09:37.290318+0000","last_undegraded":"2024-09-21T06:09:37.290318+0000","last_fullsized":"2024-09-21T06:09:37.290318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294047+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:09:37.294047+0000","last_peered":"2024-09-21T06:09:37.294047+0000","last_clean":"2024-09-21T06:09:37.294047+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:09:37.294047+0000","last_undegraded":"2024-09-21T06:09:37.294047+0000","last_fullsized":"2024-09-21T06:09:37.294047+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.278211+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:09:37.278211+0000","last_peered":"2024-09-21T06:09:37.278211+0000","last_clean":"2024-09-21T06:09:37.278211+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:09:37.278211+0000","last_undegraded":"2024-09-21T06:09:37.278211+0000","last_fullsized":"2024-09-21T06:09:37.278211+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289472+0000","last_change":"2024-09-21T06:09:34.332630+0000","last_active":"2024-09-21T06:09:37.289472+0000","last_peered":"2024-09-21T06:09:37.289472+0000","last_clean":"2024-09-21T06:09:37.289472+0000","last_became_active":"2024-09-21T06:09:34.332195+0000","last_became_peered":"2024-09-21T06:09:34.332195+0000","last_unstale":"2024-09-21T06:09:37.289472+0000","last_undegraded":"2024-09-21T06:09:37.289472+0000","last_fullsized":"2024-09-21T06:09:37.289472+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:32.639600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:39.646448+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:09:39.646448+0000","last_peered":"2024-09-21T06:09:39.646448+0000","last_clean":"2024-09-21T06:09:39.646448+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:09:39.646448+0000","last_undegraded":"2024-09-21T06:09:39.646448+0000","last_fullsized":"2024-09-21T06:09:39.646448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291699+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:09:37.291699+0000","last_peered":"2024-09-21T06:09:37.291699+0000","last_clean":"2024-09-21T06:09:37.291699+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:09:37.291699+0000","last_undegraded":"2024-09-21T06:09:37.291699+0000","last_fullsized":"2024-09-21T06:09:37.291699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277253+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:09:37.277253+0000","last_peered":"2024-09-21T06:09:37.277253+0000","last_clean":"2024-09-21T06:09:37.277253+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:09:37.277253+0000","last_undegraded":"2024-09-21T06:09:37.277253+0000","last_fullsized":"2024-09-21T06:09:37.277253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'6","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:53.290449+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:09:53.290449+0000","last_peered":"2024-09-21T06:09:53.290449+0000","last_clean":"2024-09-21T06:09:53.290449+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:09:53.290449+0000","last_undegraded":"2024-09-21T06:09:53.290449+0000","last_fullsized":"2024-09-21T06:09:53.290449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293344+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:09:37.293344+0000","last_peered":"2024-09-21T06:09:37.293344+0000","last_clean":"2024-09-21T06:09:37.293344+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:09:37.293344+0000","last_undegraded":"2024-09-21T06:09:37.293344+0000","last_fullsized":"2024-09-21T06:09:37.293344+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.295306+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:09:37.295306+0000","last_peered":"2024-09-21T06:09:37.295306+0000","last_clean":"2024-09-21T06:09:37.295306+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:09:37.295306+0000","last_undegraded":"2024-09-21T06:09:37.295306+0000","last_fullsized":"2024-09-21T06:09:37.295306+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294593+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:09:37.294593+0000","last_peered":"2024-09-21T06:09:37.294593+0000","last_clean":"2024-09-21T06:09:37.294593+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:09:37.294593+0000","last_undegraded":"2024-09-21T06:09:37.294593+0000","last_fullsized":"2024-09-21T06:09:37.294593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291084+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:09:37.291084+0000","last_peered":"2024-09-21T06:09:37.291084+0000","last_clean":"2024-09-21T06:09:37.291084+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:09:37.291084+0000","last_undegraded":"2024-09-21T06:09:37.291084+0000","last_fullsized":"2024-09-21T06:09:37.291084+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.289730+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:09:37.289730+0000","last_peered":"2024-09-21T06:09:37.289730+0000","last_clean":"2024-09-21T06:09:37.289730+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:09:37.289730+0000","last_undegraded":"2024-09-21T06:09:37.289730+0000","last_fullsized":"2024-09-21T06:09:37.289730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292053+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:09:37.292053+0000","last_peered":"2024-09-21T06:09:37.292053+0000","last_clean":"2024-09-21T06:09:37.292053+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:09:37.292053+0000","last_undegraded":"2024-09-21T06:09:37.292053+0000","last_fullsized":"2024-09-21T06:09:37.292053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.296230+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:09:37.296230+0000","last_peered":"2024-09-21T06:09:37.296230+0000","last_clean":"2024-09-21T06:09:37.296230+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:09:37.296230+0000","last_undegraded":"2024-09-21T06:09:37.296230+0000","last_fullsized":"2024-09-21T06:09:37.296230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.277540+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:09:37.277540+0000","last_peered":"2024-09-21T06:09:37.277540+0000","last_clean":"2024-09-21T06:09:37.277540+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:09:37.277540+0000","last_undegraded":"2024-09-21T06:09:37.277540+0000","last_fullsized":"2024-09-21T06:09:37.277540+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.290989+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:09:37.290989+0000","last_peered":"2024-09-21T06:09:37.290989+0000","last_clean":"2024-09-21T06:09:37.290989+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:09:37.290989+0000","last_undegraded":"2024-09-21T06:09:37.290989+0000","last_fullsized":"2024-09-21T06:09:37.290989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298233+0000","last_change":"2024-09-21T06:09:34.271214+0000","last_active":"2024-09-21T06:09:37.298233+0000","last_peered":"2024-09-21T06:09:37.298233+0000","last_clean":"2024-09-21T06:09:37.298233+0000","last_became_active":"2024-09-21T06:09:34.270756+0000","last_became_peered":"2024-09-21T06:09:34.270756+0000","last_unstale":"2024-09-21T06:09:37.298233+0000","last_undegraded":"2024-09-21T06:09:37.298233+0000","last_fullsized":"2024-09-21T06:09:37.298233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:38.175845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293584+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:09:37.293584+0000","last_peered":"2024-09-21T06:09:37.293584+0000","last_clean":"2024-09-21T06:09:37.293584+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:09:37.293584+0000","last_undegraded":"2024-09-21T06:09:37.293584+0000","last_fullsized":"2024-09-21T06:09:37.293584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.293496+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:09:37.293496+0000","last_peered":"2024-09-21T06:09:37.293496+0000","last_clean":"2024-09-21T06:09:37.293496+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:09:37.293496+0000","last_undegraded":"2024-09-21T06:09:37.293496+0000","last_fullsized":"2024-09-21T06:09:37.293496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297666+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:09:37.297666+0000","last_peered":"2024-09-21T06:09:37.297666+0000","last_clean":"2024-09-21T06:09:37.297666+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:09:37.297666+0000","last_undegraded":"2024-09-21T06:09:37.297666+0000","last_fullsized":"2024-09-21T06:09:37.297666+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291032+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:09:37.291032+0000","last_peered":"2024-09-21T06:09:37.291032+0000","last_clean":"2024-09-21T06:09:37.291032+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:09:37.291032+0000","last_undegraded":"2024-09-21T06:09:37.291032+0000","last_fullsized":"2024-09-21T06:09:37.291032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.298128+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:09:37.298128+0000","last_peered":"2024-09-21T06:09:37.298128+0000","last_clean":"2024-09-21T06:09:37.298128+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:09:37.298128+0000","last_undegraded":"2024-09-21T06:09:37.298128+0000","last_fullsized":"2024-09-21T06:09:37.298128+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.294766+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:09:37.294766+0000","last_peered":"2024-09-21T06:09:37.294766+0000","last_clean":"2024-09-21T06:09:37.294766+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:09:37.294766+0000","last_undegraded":"2024-09-21T06:09:37.294766+0000","last_fullsized":"2024-09-21T06:09:37.294766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.292737+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:09:37.292737+0000","last_peered":"2024-09-21T06:09:37.292737+0000","last_clean":"2024-09-21T06:09:37.292737+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:09:37.292737+0000","last_undegraded":"2024-09-21T06:09:37.292737+0000","last_fullsized":"2024-09-21T06:09:37.292737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.291579+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:09:37.291579+0000","last_peered":"2024-09-21T06:09:37.291579+0000","last_clean":"2024-09-21T06:09:37.291579+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:09:37.291579+0000","last_undegraded":"2024-09-21T06:09:37.291579+0000","last_fullsized":"2024-09-21T06:09:37.291579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T06:09:37.297752+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:09:37.297752+0000","last_peered":"2024-09-21T06:09:37.297752+0000","last_clean":"2024-09-21T06:09:37.297752+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:09:37.297752+0000","last_undegraded":"2024-09-21T06:09:37.297752+0000","last_fullsized":"2024-09-21T06:09:37.297752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":46001,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":34,"num_read_kb":63,"num_write":68,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":138003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":57,"ondisk_log_size":57,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":806912,"data_stored":398046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712532,"statfs":{"total":95995035648,"available":95961632768,"internally_reserved":0,"allocated":1470464,"data_stored":1033928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103194,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":398628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266719,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":815104,"data_stored":398650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1409024,"data_stored":989113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":398163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1417216,"data_stored":988713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":811008,"data_stored":398068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":398384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378685,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40348,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704804,"statfs":{"total":95995035648,"available":95953719296,"internally_reserved":0,"allocated":864256,"data_stored":443283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":443249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":811008,"data_stored":398136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":45203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":45237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":45397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:10:01.759 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:10:01.759 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T06:10:01.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T06:10:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:01 smithi110 ceph-mon[34550]: pgmap v270: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:10:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:01 smithi107 ceph-mon[34380]: pgmap v270: 97 pgs: 97 active+clean; 622 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-21T06:10:02.021 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:02.490 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:02 smithi031 ceph-mon[31237]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.487+0000 7f8e69bde640 1 -- 172.21.15.31:0/2432955449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 msgr2=0x7f8e641098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.487+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2432955449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e641098c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e5802a410 comp rx=0 tx=0).stop 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 -- 172.21.15.31:0/2432955449 shutdown_connections 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2432955449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64109e00 0x7f8e64110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2432955449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e641098c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2432955449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64103b70 0x7f8e64103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 -- 172.21.15.31:0/2432955449 >> 172.21.15.31:0/2432955449 conn(0x7f8e640ff5a0 msgr2=0x7f8e641019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 -- 172.21.15.31:0/2432955449 shutdown_connections 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 -- 172.21.15.31:0/2432955449 wait complete. 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.488+0000 7f8e69bde640 1 Processor -- start 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 -- start start 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64103b70 0x7f8e6419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e6419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64109e00 0x7f8e6419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64113ac0 con 0x7f8e641094b0 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64113940 con 0x7f8e64103b70 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e69bde640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e64113c40 con 0x7f8e64109e00 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e6419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e6419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37138/0 (socket says 172.21.15.31:37138) 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e62ffd640 1 -- 172.21.15.31:0/2993967586 learned_addr learned my addr 172.21.15.31:0/2993967586 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e63fff640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64109e00 0x7f8e6419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.489+0000 7f8e637fe640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64103b70 0x7f8e6419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e62ffd640 1 -- 172.21.15.31:0/2993967586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64109e00 msgr2=0x7f8e6419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e62ffd640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64109e00 0x7f8e6419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e62ffd640 1 -- 172.21.15.31:0/2993967586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64103b70 msgr2=0x7f8e6419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e62ffd640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64103b70 0x7f8e6419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e62ffd640 1 -- 172.21.15.31:0/2993967586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641a0460 con 0x7f8e641094b0 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e637fe640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64103b70 0x7f8e6419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e62ffd640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e6419aa90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8e5802a930 tx=0x7f8e58004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5803f030 con 0x7f8e641094b0 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58003aa0 con 0x7f8e641094b0 2024-09-21T06:10:02.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641a06f0 con 0x7f8e641094b0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58035070 con 0x7f8e641094b0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.490+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e641a6eb0 con 0x7f8e641094b0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.491+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e64104df0 con 0x7f8e641094b0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.493+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8e58042030 con 0x7f8e641094b0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.494+0000 7f8e60ff9640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8e4c0b65f0 0x7f8e4c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.494+0000 7f8e637fe640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8e4c0b65f0 0x7f8e4c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.494+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f8e58107fd0 con 0x7f8e641094b0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.494+0000 7f8e637fe640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8e4c0b65f0 0x7f8e4c0b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005fd0 tx=0x7f8e54005ea0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:02.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.497+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e580ce6c0 con 0x7f8e641094b0 2024-09-21T06:10:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.763+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8e6410bbd0 con 0x7f8e641094b0 2024-09-21T06:10:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.766+0000 7f8e60ff9640 1 -- 172.21.15.31:0/2993967586 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f8e580d5e20 con 0x7f8e641094b0 2024-09-21T06:10:02.768 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-21T06:10:02.768 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQA6Y+5mUQaaLRAAkkfWHURICZOU3k2+AAEeIQ== 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8e4c0b65f0 msgr2=0x7f8e4c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8e4c0b65f0 0x7f8e4c0b8b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005fd0 tx=0x7f8e54005ea0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 msgr2=0x7f8e6419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e6419aa90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8e5802a930 tx=0x7f8e58004270 comp rx=0 tx=0).stop 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 shutdown_connections 2024-09-21T06:10:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64109e00 0x7f8e6419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8e4c0b65f0 0x7f8e4c0b8b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e641094b0 0x7f8e6419aa90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 --2- 172.21.15.31:0/2993967586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e64103b70 0x7f8e6419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 >> 172.21.15.31:0/2993967586 conn(0x7f8e640ff5a0 msgr2=0x7f8e641019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 shutdown_connections 2024-09-21T06:10:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:02.769+0000 7f8e69bde640 1 -- 172.21.15.31:0/2993967586 wait complete. 2024-09-21T06:10:02.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:02 smithi110 ceph-mon[34550]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:02.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:02 smithi107 ceph-mon[34380]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:03.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:03 smithi031 ceph-mon[31237]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-21T06:10:03.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:03 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2993967586' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:10:03.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:03 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2993967586' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:10:03.496 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:10:03.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T06:10:03.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T06:10:03.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T06:10:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:03 smithi110 ceph-mon[34550]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-21T06:10:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:03 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2993967586' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:10:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:03 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2993967586' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:10:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:03 smithi107 ceph-mon[34380]: pgmap v271: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-21T06:10:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:03 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2993967586' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:10:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:03 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2993967586' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:10:03.830 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 -- 172.21.15.31:0/341426777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30102a60 msgr2=0x7f9e30102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 --2- 172.21.15.31:0/341426777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30102a60 0x7f9e30102e70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009900 tx=0x7f9e2002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 -- 172.21.15.31:0/341426777 shutdown_connections 2024-09-21T06:10:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 --2- 172.21.15.31:0/341426777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30108cf0 0x7f9e3010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 --2- 172.21.15.31:0/341426777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e301083a0 0x7f9e301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 --2- 172.21.15.31:0/341426777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30102a60 0x7f9e30102e70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 -- 172.21.15.31:0/341426777 >> 172.21.15.31:0/341426777 conn(0x7f9e30079730 msgr2=0x7f9e30079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 -- 172.21.15.31:0/341426777 shutdown_connections 2024-09-21T06:10:04.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.270+0000 7f9e36a56640 1 -- 172.21.15.31:0/341426777 wait complete. 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.271+0000 7f9e36a56640 1 Processor -- start 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.271+0000 7f9e36a56640 1 -- start start 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e36a56640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 0x7f9e30104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e36a56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e301083a0 0x7f9e30104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e36a56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30108cf0 0x7f9e30104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e36a56640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301129e0 con 0x7f9e30108cf0 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e36a56640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30112860 con 0x7f9e301083a0 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e36a56640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30112b60 con 0x7f9e30102a60 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 0x7f9e30104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e34ff3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30108cf0 0x7f9e30104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 0x7f9e30104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52938/0 (socket says 172.21.15.31:52938) 2024-09-21T06:10:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 -- 172.21.15.31:0/87292776 learned_addr learned my addr 172.21.15.31:0/87292776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2f7fe640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e301083a0 0x7f9e30104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 -- 172.21.15.31:0/87292776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e301083a0 msgr2=0x7f9e30104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e301083a0 0x7f9e30104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 -- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30108cf0 msgr2=0x7f9e30104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30108cf0 0x7f9e30104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e2ffff640 1 -- 172.21.15.31:0/87292776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e301a8760 con 0x7f9e30102a60 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.272+0000 7f9e34ff3640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30108cf0 0x7f9e30104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.273+0000 7f9e2ffff640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 0x7f9e30104250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005940 tx=0x7f9e200042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.273+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2003f030 con 0x7f9e30102a60 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.273+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e301a89f0 con 0x7f9e30102a60 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.273+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e301a8fa0 con 0x7f9e30102a60 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.273+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e20003cb0 con 0x7f9e30102a60 2024-09-21T06:10:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.273+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20049060 con 0x7f9e30102a60 2024-09-21T06:10:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.274+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c0052c0 con 0x7f9e30102a60 2024-09-21T06:10:04.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.276+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9e20042030 con 0x7f9e30102a60 2024-09-21T06:10:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.277+0000 7f9e2d7fa640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9e180b6670 0x7f9e180b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.277+0000 7f9e2f7fe640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9e180b6670 0x7f9e180b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.277+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f9e201079c0 con 0x7f9e30102a60 2024-09-21T06:10:04.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.277+0000 7f9e2f7fe640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9e180b6670 0x7f9e180b8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e30100b70 tx=0x7f9e24009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.280+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e200ce0b0 con 0x7f9e30102a60 2024-09-21T06:10:04.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.468+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f9e1c005880 con 0x7f9e30102a60 2024-09-21T06:10:04.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.469+0000 7f9e2d7fa640 1 -- 172.21.15.31:0/87292776 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7f9e200d2490 con 0x7f9e30102a60 2024-09-21T06:10:04.471 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9e180b6670 msgr2=0x7f9e180b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9e180b6670 0x7f9e180b8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e30100b70 tx=0x7f9e24009110 comp rx=0 tx=0).stop 2024-09-21T06:10:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 msgr2=0x7f9e30104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 0x7f9e30104250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005940 tx=0x7f9e200042f0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 shutdown_connections 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9e180b6670 0x7f9e180b8b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e30108cf0 0x7f9e30104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e301083a0 0x7f9e30104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 --2- 172.21.15.31:0/87292776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e30102a60 0x7f9e30104250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.471+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 >> 172.21.15.31:0/87292776 conn(0x7f9e30079730 msgr2=0x7f9e30101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.472+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 shutdown_connections 2024-09-21T06:10:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:04.472+0000 7f9e36a56640 1 -- 172.21.15.31:0/87292776 wait complete. 2024-09-21T06:10:05.146 INFO:teuthology.orchestra.run.smithi031.stdout:[{"version":56,"timestamp":"2024-09-21T06:09:27.137331+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T06:10:05.146 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T06:10:05.146 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T06:10:05.146 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T06:10:05.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T06:10:05.404 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:05 smithi031 ceph-mon[31237]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-21T06:10:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:05 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/87292776' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:10:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:05 smithi110 ceph-mon[34550]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-21T06:10:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:05 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/87292776' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:10:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:05 smithi107 ceph-mon[34380]: pgmap v272: 97 pgs: 97 active+clean; 638 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.8 KiB/s wr, 4 op/s 2024-09-21T06:10:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:05 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/87292776' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:10:05.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 -- 172.21.15.31:0/3605841291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd01094c0 msgr2=0x7fabd01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:05.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 --2- 172.21.15.31:0/3605841291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd01094c0 0x7fabd01098d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009900 tx=0x7fabc002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 -- 172.21.15.31:0/3605841291 shutdown_connections 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 --2- 172.21.15.31:0/3605841291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0109e10 0x7fabd0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 --2- 172.21.15.31:0/3605841291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd01094c0 0x7fabd01098d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 --2- 172.21.15.31:0/3605841291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd0101fe0 0x7fabd01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 -- 172.21.15.31:0/3605841291 >> 172.21.15.31:0/3605841291 conn(0x7fabd0077bc0 msgr2=0x7fabd0078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 -- 172.21.15.31:0/3605841291 shutdown_connections 2024-09-21T06:10:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.859+0000 7fabd770a640 1 -- 172.21.15.31:0/3605841291 wait complete. 2024-09-21T06:10:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 Processor -- start 2024-09-21T06:10:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 -- start start 2024-09-21T06:10:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0101fe0 0x7fabd019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd01094c0 0x7fabd019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 0x7fabd019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:05.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0113a80 con 0x7fabd0109e10 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0113900 con 0x7fabd0101fe0 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd770a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabd0113c00 con 0x7fabd01094c0 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.860+0000 7fabd5ca7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 0x7fabd019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 0x7fabd019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37160/0 (socket says 172.21.15.31:37160) 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 -- 172.21.15.31:0/2439820025 learned_addr learned my addr 172.21.15.31:0/2439820025 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd4ca5640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd01094c0 0x7fabd019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd54a6640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0101fe0 0x7fabd019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 -- 172.21.15.31:0/2439820025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd01094c0 msgr2=0x7fabd019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd01094c0 0x7fabd019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 -- 172.21.15.31:0/2439820025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0101fe0 msgr2=0x7fabd019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0101fe0 0x7fabd019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 -- 172.21.15.31:0/2439820025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd01a0340 con 0x7fabd0109e10 2024-09-21T06:10:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd54a6640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0101fe0 0x7fabd019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd5ca7640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 0x7fabd019fcd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fabc800cad0 tx=0x7fabc800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc800ae20 con 0x7fabd0109e10 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd01a0630 con 0x7fabd0109e10 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc8005020 con 0x7fabd0109e10 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc800f6b0 con 0x7fabd0109e10 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.861+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01a0af0 con 0x7fabd0109e10 2024-09-21T06:10:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.863+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd01023f0 con 0x7fabd0109e10 2024-09-21T06:10:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.864+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fabc80051d0 con 0x7fabd0109e10 2024-09-21T06:10:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.865+0000 7fabc67fc640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabbc0b6620 0x7fabbc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.865+0000 7fabd54a6640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabbc0b6620 0x7fabbc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.865+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fabc80e7510 con 0x7fabd0109e10 2024-09-21T06:10:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.865+0000 7fabd54a6640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabbc0b6620 0x7fabbc0b8b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00abe0 tx=0x7fabcc00a5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:05.869+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fabc8014030 con 0x7fabd0109e10 2024-09-21T06:10:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.114+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fabd010b700 con 0x7fabd0109e10 2024-09-21T06:10:06.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.119+0000 7fabc67fc640 1 -- 172.21.15.31:0/2439820025 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296453 (secure 0 0 0) 0x7fabc80adc60 con 0x7fabd0109e10 2024-09-21T06:10:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:06.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabbc0b6620 msgr2=0x7fabbc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:06.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabbc0b6620 0x7fabbc0b8b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00abe0 tx=0x7fabcc00a5c0 comp rx=0 tx=0).stop 2024-09-21T06:10:06.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 msgr2=0x7fabd019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 0x7fabd019fcd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fabc800cad0 tx=0x7fabc800cf90 comp rx=0 tx=0).stop 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 shutdown_connections 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabbc0b6620 0x7fabbc0b8b30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd0109e10 0x7fabd019fcd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd01094c0 0x7fabd019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 --2- 172.21.15.31:0/2439820025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd0101fe0 0x7fabd019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 >> 172.21.15.31:0/2439820025 conn(0x7fabd0077bc0 msgr2=0x7fabd0107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 shutdown_connections 2024-09-21T06:10:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:06.122+0000 7fabd770a640 1 -- 172.21.15.31:0/2439820025 wait complete. 2024-09-21T06:10:06.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-21T06:10:06.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-21T06:10:06.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:06 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2439820025' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:10:06.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-21T06:10:06.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T06:10:06.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-21T06:10:06.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:06 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2439820025' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:10:06.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-21T06:10:06.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T06:10:06.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T06:10:06.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-09-21T06:10:06.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T06:10:06.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-09-21T06:10:06.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T06:10:06.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-09-21T06:10:06.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]} : dispatch 2024-09-21T06:10:06.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:06 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2439820025' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:10:06.878 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"flags":0,"active_gid":14226,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2760018577},{"type":"v1","addr":"172.21.15.31:6801","nonce":2760018577}]},"active_addr":"172.21.15.31:6801/2760018577","active_change":"2024-09-21T06:03:05.499525+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24157,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24170,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3521725758}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2872515961}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1008423759}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":1742853890}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":314319969}]}]} 2024-09-21T06:10:06.884 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T06:10:06.884 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T06:10:06.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:10:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-21T06:10:07.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-21T06:10:07.546 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:07 smithi031 ceph-mon[31237]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 -- 172.21.15.31:0/1455248333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0754109df0 msgr2=0x7f0754110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 --2- 172.21.15.31:0/1455248333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0754109df0 0x7f0754110620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f074c009ca0 tx=0x7f074c02a8e0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 -- 172.21.15.31:0/1455248333 shutdown_connections 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 --2- 172.21.15.31:0/1455248333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0754109df0 0x7f0754110620 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 --2- 172.21.15.31:0/1455248333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07541094a0 0x7f07541098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 --2- 172.21.15.31:0/1455248333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0754103b60 0x7f0754103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.634+0000 7f075c79e640 1 -- 172.21.15.31:0/1455248333 >> 172.21.15.31:0/1455248333 conn(0x7f075407b850 msgr2=0x7f075407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.635+0000 7f075c79e640 1 -- 172.21.15.31:0/1455248333 shutdown_connections 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.635+0000 7f075c79e640 1 -- 172.21.15.31:0/1455248333 wait complete. 2024-09-21T06:10:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.635+0000 7f075c79e640 1 Processor -- start 2024-09-21T06:10:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.635+0000 7f075c79e640 1 -- start start 2024-09-21T06:10:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075c79e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0754103b60 0x7f075419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075c79e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 0x7f075419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:07.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075c79e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0754109df0 0x7f075419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075c79e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0754113af0 con 0x7f07541094a0 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075c79e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0754113970 con 0x7f0754109df0 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075c79e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0754113c70 con 0x7f0754103b60 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 0x7f075419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 0x7f075419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37186/0 (socket says 172.21.15.31:37186) 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 -- 172.21.15.31:0/1567238089 learned_addr learned my addr 172.21.15.31:0/1567238089 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075a53a640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0754103b60 0x7f075419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075ad3b640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0754109df0 0x7f075419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 -- 172.21.15.31:0/1567238089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0754103b60 msgr2=0x7f075419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0754103b60 0x7f075419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 -- 172.21.15.31:0/1567238089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0754109df0 msgr2=0x7f075419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:07.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0754109df0 0x7f075419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 -- 172.21.15.31:0/1567238089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07541a0430 con 0x7f07541094a0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f075ad3b640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0754109df0 0x7f075419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f0759d39640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 0x7f075419aa60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0744006fd0 tx=0x7f074400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.636+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744011830 con 0x7f07541094a0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.637+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0744016070 con 0x7f07541094a0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.637+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744010e00 con 0x7f07541094a0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.637+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07541a0720 con 0x7f07541094a0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.637+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0754075670 con 0x7f07541094a0 2024-09-21T06:10:07.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.638+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0754103f70 con 0x7f07541094a0 2024-09-21T06:10:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.639+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f074401b460 con 0x7f07541094a0 2024-09-21T06:10:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.640+0000 7f074b7fe640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f07400b6670 0x7f07400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.640+0000 7f075a53a640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f07400b6670 0x7f07400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.640+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f07440e7230 con 0x7f07541094a0 2024-09-21T06:10:07.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.641+0000 7f075a53a640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f07400b6670 0x7f07400b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07500058c0 tx=0x7f075000c850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:07.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.643+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07440ad820 con 0x7f07541094a0 2024-09-21T06:10:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-09-21T06:10:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-21T06:10:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-21T06:10:07.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:07 smithi110 ceph-mon[34550]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T06:10:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-21T06:10:07.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-09-21T06:10:07.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:07 smithi107 ceph-mon[34380]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:10:07.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.836+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f075419b5f0 con 0x7f07541094a0 2024-09-21T06:10:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.838+0000 7f074b7fe640 1 -- 172.21.15.31:0/1567238089 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23089 (secure 0 0 0) 0x7f075419b5f0 con 0x7f07541094a0 2024-09-21T06:10:07.840 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:07.840 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:10:07.387838+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f07400b6670 msgr2=0x7f07400b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f07400b6670 0x7f07400b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07500058c0 tx=0x7f075000c850 comp rx=0 tx=0).stop 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 msgr2=0x7f075419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 0x7f075419aa60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0744006fd0 tx=0x7f074400b8e0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 shutdown_connections 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0754109df0 0x7f075419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f07400b6670 0x7f07400b8b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07541094a0 0x7f075419aa60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 --2- 172.21.15.31:0/1567238089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0754103b60 0x7f075419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 >> 172.21.15.31:0/1567238089 conn(0x7f075407b850 msgr2=0x7f07541074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 shutdown_connections 2024-09-21T06:10:07.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:07.840+0000 7f075c79e640 1 -- 172.21.15.31:0/1567238089 wait complete. 2024-09-21T06:10:08.457 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T06:10:08.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:10:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:08.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:08 smithi110 ceph-mon[34550]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:10:08.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:08 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1567238089' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:08.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:08 smithi107 ceph-mon[34380]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:10:08.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:08 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1567238089' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:08 smithi031 ceph-mon[31237]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:10:08.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:08 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1567238089' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 -- 172.21.15.31:0/1049904039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc105310 msgr2=0x7f2ddc10d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 --2- 172.21.15.31:0/1049904039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc105310 0x7f2ddc10d860 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009900 tx=0x7f2dd002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 -- 172.21.15.31:0/1049904039 shutdown_connections 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 --2- 172.21.15.31:0/1049904039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc10de20 0x7f2ddc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 --2- 172.21.15.31:0/1049904039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc105310 0x7f2ddc10d860 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 --2- 172.21.15.31:0/1049904039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc1049c0 0x7f2ddc104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 -- 172.21.15.31:0/1049904039 >> 172.21.15.31:0/1049904039 conn(0x7f2ddc079730 msgr2=0x7f2ddc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 -- 172.21.15.31:0/1049904039 shutdown_connections 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.235+0000 7f2de189d640 1 -- 172.21.15.31:0/1049904039 wait complete. 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 Processor -- start 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 -- start start 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc1049c0 0x7f2ddc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc105310 0x7f2ddc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 0x7f2ddc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc113a80 con 0x7f2ddc10de20 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc113900 con 0x7f2ddc1049c0 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2de189d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc113c00 con 0x7f2ddc105310 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2ddb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 0x7f2ddc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2ddb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 0x7f2ddc19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33088/0 (socket says 172.21.15.31:33088) 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2ddb7fe640 1 -- 172.21.15.31:0/753344117 learned_addr learned my addr 172.21.15.31:0/753344117 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2dda7fc640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc105310 0x7f2ddc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.236+0000 7f2ddaffd640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc1049c0 0x7f2ddc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddb7fe640 1 -- 172.21.15.31:0/753344117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc105310 msgr2=0x7f2ddc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddb7fe640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc105310 0x7f2ddc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddb7fe640 1 -- 172.21.15.31:0/753344117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc1049c0 msgr2=0x7f2ddc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddb7fe640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc1049c0 0x7f2ddc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddb7fe640 1 -- 172.21.15.31:0/753344117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc1a0480 con 0x7f2ddc10de20 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddaffd640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc1049c0 0x7f2ddc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2dda7fc640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc105310 0x7f2ddc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2ddb7fe640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 0x7f2ddc19fd50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400d750 tx=0x7f2dd400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4017070 con 0x7f2ddc10de20 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd40054d0 con 0x7f2ddc10de20 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc1a0770 con 0x7f2ddc10de20 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4004c60 con 0x7f2ddc10de20 2024-09-21T06:10:09.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.237+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc1a6ed0 con 0x7f2ddc10de20 2024-09-21T06:10:09.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.238+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc06c1e0 con 0x7f2ddc10de20 2024-09-21T06:10:09.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.240+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2dd4009bd0 con 0x7f2ddc10de20 2024-09-21T06:10:09.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.241+0000 7f2de089b640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2dc40b65f0 0x7f2dc40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:09.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.241+0000 7f2ddaffd640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2dc40b65f0 0x7f2dc40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:09.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.241+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2dd40e7410 con 0x7f2ddc10de20 2024-09-21T06:10:09.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.241+0000 7f2ddaffd640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2dc40b65f0 0x7f2dc40b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc06c8e0 tx=0x7f2dcc009540 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.244+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2dd40ada60 con 0x7f2ddc10de20 2024-09-21T06:10:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.438+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ddc104dd0 con 0x7f2ddc10de20 2024-09-21T06:10:09.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.439+0000 7f2de089b640 1 -- 172.21.15.31:0/753344117 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23089 (secure 0 0 0) 0x7f2ddc104dd0 con 0x7f2ddc10de20 2024-09-21T06:10:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:09.441 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:10:07.387838+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:10:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2dc40b65f0 msgr2=0x7f2dc40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2dc40b65f0 0x7f2dc40b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc06c8e0 tx=0x7f2dcc009540 comp rx=0 tx=0).stop 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 msgr2=0x7f2ddc19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 0x7f2ddc19fd50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400d750 tx=0x7f2dd400dc10 comp rx=0 tx=0).stop 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 shutdown_connections 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2dc40b65f0 0x7f2dc40b8b00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ddc10de20 0x7f2ddc19fd50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.441+0000 7f2de189d640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ddc105310 0x7f2ddc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.442+0000 7f2de189d640 1 --2- 172.21.15.31:0/753344117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ddc1049c0 0x7f2ddc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.442+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 >> 172.21.15.31:0/753344117 conn(0x7f2ddc079730 msgr2=0x7f2ddc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.442+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 shutdown_connections 2024-09-21T06:10:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:09.442+0000 7f2de189d640 1 -- 172.21.15.31:0/753344117 wait complete. 2024-09-21T06:10:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:09 smithi110 ceph-mon[34550]: pgmap v276: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.1 KiB/s wr, 4 op/s 2024-09-21T06:10:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:09 smithi107 ceph-mon[34380]: pgmap v276: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.1 KiB/s wr, 4 op/s 2024-09-21T06:10:09.878 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:09 smithi031 ceph-mon[31237]: pgmap v276: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.1 KiB/s wr, 4 op/s 2024-09-21T06:10:10.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T06:10:10.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T06:10:10.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T06:10:10.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T06:10:10.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T06:10:10.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T06:10:10.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T06:10:10.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T06:10:10.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T06:10:10.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T06:10:10.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T06:10:10.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T06:10:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:10 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/753344117' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:10 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/753344117' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:10 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/753344117' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.109 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.112 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.172 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:11.510 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:11 smithi031 ceph-mon[31237]: pgmap v277: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 639 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:10:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:11 smithi110 ceph-mon[34550]: pgmap v277: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 639 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:10:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:11 smithi107 ceph-mon[34380]: pgmap v277: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 639 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T06:10:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 -- 172.21.15.31:0/1653141585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 msgr2=0x7fcda410fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/1653141585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda410fc70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009900 tx=0x7fcd9402a480 comp rx=0 tx=0).stop 2024-09-21T06:10:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 -- 172.21.15.31:0/1653141585 shutdown_connections 2024-09-21T06:10:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/1653141585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda41101b0 0x7fcda41125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/1653141585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda410fc70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/1653141585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda4102600 0x7fcda4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 -- 172.21.15.31:0/1653141585 >> 172.21.15.31:0/1653141585 conn(0x7fcda4079750 msgr2=0x7fcda4079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 -- 172.21.15.31:0/1653141585 shutdown_connections 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 -- 172.21.15.31:0/1653141585 wait complete. 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 Processor -- start 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.612+0000 7fcdab9b9640 1 -- start start 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcdab9b9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda4102600 0x7fcda419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcdab9b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcdab9b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda41101b0 0x7fcda41a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcdab9b9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4115ca0 con 0x7fcda4102f50 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcdab9b9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4115b20 con 0x7fcda41101b0 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcdab9b9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcda4115e20 con 0x7fcda4102600 2024-09-21T06:10:12.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33126/0 (socket says 172.21.15.31:33126) 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 -- 172.21.15.31:0/2983839201 learned_addr learned my addr 172.21.15.31:0/2983839201 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda9755640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda4102600 0x7fcda419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda9f56640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda41101b0 0x7fcda41a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 -- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda4102600 msgr2=0x7fcda419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcda4102600 0x7fcda419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 -- 172.21.15.31:0/2983839201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda41101b0 msgr2=0x7fcda41a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda41101b0 0x7fcda41a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 -- 172.21.15.31:0/2983839201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda41a2600 con 0x7fcda4102f50 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda9f56640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda41101b0 0x7fcda41a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.613+0000 7fcda8f54640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda419cc30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd94005940 tx=0x7fcd940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:12.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.614+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9403f030 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.614+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd94003cb0 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.614+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94035070 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.614+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda41a2890 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.614+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41a9050 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.615+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcda4109680 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.616+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fcd94042030 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.617+0000 7fcd9a7fc640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd900b6670 0x7fcd900b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.617+0000 7fcda9755640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd900b6670 0x7fcd900b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.617+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fcd94108090 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.617+0000 7fcd9a7fc640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] conn(0x7fcd900c0460 0x7fcd900c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.617+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 --> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd900c3020 con 0x7fcd900c0460 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.617+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fcd941084e0 con 0x7fcda4102f50 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.618+0000 7fcda9755640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd900b6670 0x7fcd900b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcda000bbe0 tx=0x7fcda000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:12.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.618+0000 7fcda9f56640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] conn(0x7fcd900c0460 0x7fcd900c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.618+0000 7fcda9f56640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] conn(0x7fcd900c0460 0x7fcd900c2950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.619+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== osd.8 v2:172.21.15.110:6816/4244140994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd900c3020 con 0x7fcd900c0460 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.633+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 --> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcda406b350 con 0x7fcd900c0460 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.633+0000 7fcd9a7fc640 1 -- 172.21.15.31:0/2983839201 <== osd.8 v2:172.21.15.110:6816/4244140994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcda406b350 con 0x7fcd900c0460 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] conn(0x7fcd900c0460 msgr2=0x7fcd900c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] conn(0x7fcd900c0460 0x7fcd900c2950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd900b6670 msgr2=0x7fcd900b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcd900b6670 0x7fcd900b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcda000bbe0 tx=0x7fcda000b390 comp rx=0 tx=0).stop 2024-09-21T06:10:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 msgr2=0x7fcda419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 --2- 172.21.15.31:0/2983839201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcda4102f50 0x7fcda419cc30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd94005940 tx=0x7fcd940042f0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcda9f56640 1 -- 172.21.15.31:0/2983839201 reap_dead start 2024-09-21T06:10:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 shutdown_connections 2024-09-21T06:10:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 >> 172.21.15.31:0/2983839201 conn(0x7fcda4079750 msgr2=0x7fcda4108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 shutdown_connections 2024-09-21T06:10:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.634+0000 7fcdab9b9640 1 -- 172.21.15.31:0/2983839201 wait complete. 2024-09-21T06:10:12.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 -- 172.21.15.31:0/1446162442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 msgr2=0x7f7828112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 --2- 172.21.15.31:0/1446162442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f7828112620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ca0 tx=0x7f782002a950 comp rx=0 tx=0).stop 2024-09-21T06:10:12.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 -- 172.21.15.31:0/1446162442 shutdown_connections 2024-09-21T06:10:12.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 --2- 172.21.15.31:0/1446162442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f7828112620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 --2- 172.21.15.31:0/1446162442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7828102fa0 0x7f782810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 --2- 172.21.15.31:0/1446162442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7828102650 0x7f7828102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 -- 172.21.15.31:0/1446162442 >> 172.21.15.31:0/1446162442 conn(0x7f7828079760 msgr2=0x7f7828079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 -- 172.21.15.31:0/1446162442 shutdown_connections 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 -- 172.21.15.31:0/1446162442 wait complete. 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.158+0000 7f782decd640 1 Processor -- start 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 -- start start 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7828102650 0x7f782819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7828102fa0 0x7f782819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f78281a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7828115ca0 con 0x7f7828110200 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7828115b20 con 0x7f7828102650 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f782decd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7828115e20 con 0x7f7828102fa0 2024-09-21T06:10:12.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f78281a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f78281a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33098/0 (socket says 172.21.15.31:33098) 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 -- 172.21.15.31:0/2464457210 learned_addr learned my addr 172.21.15.31:0/2464457210 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7826ffd640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7828102fa0 0x7f782819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f78277fe640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7828102650 0x7f782819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 -- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7828102fa0 msgr2=0x7f782819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7828102fa0 0x7f782819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 -- 172.21.15.31:0/2464457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7828102650 msgr2=0x7f782819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.159+0000 7f7827fff640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7828102650 0x7f782819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f7827fff640 1 -- 172.21.15.31:0/2464457210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78281a2580 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f78277fe640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7828102650 0x7f782819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f7827fff640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f78281a1e50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7820004af0 tx=0x7f7820031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820034040 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78200308f0 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78281a2810 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820004600 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.160+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78281a8fd0 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.161+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f782810a7e0 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.163+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7820037020 con 0x7f7828110200 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.163+0000 7f7824ff9640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78100b6670 0x7f78100b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.163+0000 7f78277fe640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78100b6670 0x7f78100b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.163+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f78201083b0 con 0x7f7828110200 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.164+0000 7f7824ff9640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] conn(0x7f78100c0460 0x7f78100c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.164+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 --> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f782000b150 con 0x7f78100c0460 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.164+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f7820108800 con 0x7f7828110200 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.164+0000 7f78277fe640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78100b6670 0x7f78100b8b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7818009c20 tx=0x7f7818005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.164+0000 7f7826ffd640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] conn(0x7f78100c0460 0x7f78100c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.164+0000 7f7826ffd640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] conn(0x7f78100c0460 0x7f78100c2950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.165+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== osd.5 v2:172.21.15.110:6808/3518155164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f782000b150 con 0x7f78100c0460 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.179+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 --> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f782806b350 con 0x7f78100c0460 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.179+0000 7f7824ff9640 1 -- 172.21.15.31:0/2464457210 <== osd.5 v2:172.21.15.110:6808/3518155164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f782806b350 con 0x7f78100c0460 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] conn(0x7f78100c0460 msgr2=0x7f78100c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] conn(0x7f78100c0460 0x7f78100c2950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:12.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78100b6670 msgr2=0x7f78100b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78100b6670 0x7f78100b8b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7818009c20 tx=0x7f7818005f90 comp rx=0 tx=0).stop 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 msgr2=0x7f78281a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 --2- 172.21.15.31:0/2464457210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7828110200 0x7f78281a1e50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7820004af0 tx=0x7f7820031f50 comp rx=0 tx=0).stop 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f7827fff640 1 -- 172.21.15.31:0/2464457210 reap_dead start 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 shutdown_connections 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 >> 172.21.15.31:0/2464457210 conn(0x7f7828079760 msgr2=0x7f7828109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 shutdown_connections 2024-09-21T06:10:12.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.180+0000 7f782decd640 1 -- 172.21.15.31:0/2464457210 wait complete. 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.266+0000 7fbe73e3a640 1 -- 172.21.15.31:0/1700774774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c105b10 msgr2=0x7fbe6c10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.266+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/1700774774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c105b10 0x7fbe6c10f580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009ca0 tx=0x7fbe6402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 -- 172.21.15.31:0/1700774774 shutdown_connections 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/1700774774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c105b10 0x7fbe6c10f580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/1700774774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1051c0 0x7fbe6c1055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/1700774774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe6c1031b0 0x7fbe6c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 -- 172.21.15.31:0/1700774774 >> 172.21.15.31:0/1700774774 conn(0x7fbe6c079f90 msgr2=0x7fbe6c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 -- 172.21.15.31:0/1700774774 shutdown_connections 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 -- 172.21.15.31:0/1700774774 wait complete. 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 Processor -- start 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.267+0000 7fbe73e3a640 1 -- start start 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe73e3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c1031b0 0x7fbe6c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe73e3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1051c0 0x7fbe6c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe73e3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe6c105b10 0x7fbe6c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe73e3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c112a10 con 0x7fbe6c1031b0 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe73e3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c112890 con 0x7fbe6c1051c0 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe73e3a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c112b90 con 0x7fbe6c105b10 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c1031b0 0x7fbe6c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c1031b0 0x7fbe6c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33116/0 (socket says 172.21.15.31:33116) 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 -- 172.21.15.31:0/2719971943 learned_addr learned my addr 172.21.15.31:0/2719971943 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe723d7640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe6c105b10 0x7fbe6c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:13.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe713d5640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1051c0 0x7fbe6c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 -- 172.21.15.31:0/2719971943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe6c105b10 msgr2=0x7fbe6c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe6c105b10 0x7fbe6c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 -- 172.21.15.31:0/2719971943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1051c0 msgr2=0x7fbe6c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1051c0 0x7fbe6c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 -- 172.21.15.31:0/2719971943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c1a0400 con 0x7fbe6c1031b0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe713d5640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1051c0 0x7fbe6c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.268+0000 7fbe71bd6640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c1031b0 0x7fbe6c19a490 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800d770 tx=0x7fbe6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.269+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68017070 con 0x7fbe6c1031b0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.269+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c1a06f0 con 0x7fbe6c1031b0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.269+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe680054d0 con 0x7fbe6c1031b0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.269+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68004c30 con 0x7fbe6c1031b0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.269+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c1a6e50 con 0x7fbe6c1031b0 2024-09-21T06:10:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.270+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe6c063b50 con 0x7fbe6c1031b0 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.271+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbe68004dd0 con 0x7fbe6c1031b0 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.272+0000 7fbe62ffd640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbe580b65f0 0x7fbe580b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.272+0000 7fbe713d5640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbe580b65f0 0x7fbe580b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.272+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbe680e74e0 con 0x7fbe6c1031b0 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.272+0000 7fbe62ffd640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] conn(0x7fbe580c03e0 0x7fbe580c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.272+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 --> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe68021e50 con 0x7fbe580c03e0 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.272+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fbe680e7930 con 0x7fbe6c1031b0 2024-09-21T06:10:13.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.273+0000 7fbe723d7640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] conn(0x7fbe580c03e0 0x7fbe580c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.273+0000 7fbe713d5640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbe580b65f0 0x7fbe580b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0058c0 tx=0x7fbe5c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.273+0000 7fbe723d7640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] conn(0x7fbe580c03e0 0x7fbe580c28d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.274+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== osd.3 v2:172.21.15.31:6810/1116793462 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe68021e50 con 0x7fbe580c03e0 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.288+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 --> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe6c063fd0 con 0x7fbe580c03e0 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.288+0000 7fbe62ffd640 1 -- 172.21.15.31:0/2719971943 <== osd.3 v2:172.21.15.31:6810/1116793462 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe6c063fd0 con 0x7fbe580c03e0 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.288+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] conn(0x7fbe580c03e0 msgr2=0x7fbe580c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.288+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] conn(0x7fbe580c03e0 0x7fbe580c28d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbe580b65f0 msgr2=0x7fbe580b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbe580b65f0 0x7fbe580b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0058c0 tx=0x7fbe5c005850 comp rx=0 tx=0).stop 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c1031b0 msgr2=0x7fbe6c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 --2- 172.21.15.31:0/2719971943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe6c1031b0 0x7fbe6c19a490 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800d770 tx=0x7fbe6800dc30 comp rx=0 tx=0).stop 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe723d7640 1 -- 172.21.15.31:0/2719971943 reap_dead start 2024-09-21T06:10:13.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 shutdown_connections 2024-09-21T06:10:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 >> 172.21.15.31:0/2719971943 conn(0x7fbe6c079f90 msgr2=0x7fbe6c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 shutdown_connections 2024-09-21T06:10:13.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:12.289+0000 7fbe73e3a640 1 -- 172.21.15.31:0/2719971943 wait complete. 2024-09-21T06:10:13.528 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:13.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:13 smithi031 ceph-mon[31237]: pgmap v278: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 895 B/s wr, 2 op/s; 14 B/s, 0 objects/s recovering 2024-09-21T06:10:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:13 smithi110 ceph-mon[34550]: pgmap v278: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 895 B/s wr, 2 op/s; 14 B/s, 0 objects/s recovering 2024-09-21T06:10:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:13 smithi107 ceph-mon[34380]: pgmap v278: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 895 B/s wr, 2 op/s; 14 B/s, 0 objects/s recovering 2024-09-21T06:10:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:227633266723 2024-09-21T06:10:14.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T06:10:14.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 -- 172.21.15.31:0/2089159982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 msgr2=0x7f192010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:14.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 --2- 172.21.15.31:0/2089159982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192010d8b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1910009900 tx=0x7f191002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 -- 172.21.15.31:0/2089159982 shutdown_connections 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 --2- 172.21.15.31:0/2089159982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192010ddf0 0x7f1920110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 --2- 172.21.15.31:0/2089159982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192010d8b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 --2- 172.21.15.31:0/2089159982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1920104840 0x7f1920104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 -- 172.21.15.31:0/2089159982 >> 172.21.15.31:0/2089159982 conn(0x7f1920079730 msgr2=0x7f1920079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 -- 172.21.15.31:0/2089159982 shutdown_connections 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.835+0000 7f1927f08640 1 -- 172.21.15.31:0/2089159982 wait complete. 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 Processor -- start 2024-09-21T06:10:14.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 -- start start 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1920104840 0x7f192019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192010ddf0 0x7f192019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1920113b50 con 0x7f1920105270 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19201139d0 con 0x7f192010ddf0 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1927f08640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1920113cd0 con 0x7f1920104840 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f19254a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f19254a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33138/0 (socket says 172.21.15.31:33138) 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f19254a3640 1 -- 172.21.15.31:0/3779628347 learned_addr learned my addr 172.21.15.31:0/3779628347 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:14.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f1925ca4640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1920104840 0x7f192019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.836+0000 7f19264a5640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192010ddf0 0x7f192019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19254a3640 1 -- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1920104840 msgr2=0x7f192019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19254a3640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1920104840 0x7f192019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19254a3640 1 -- 172.21.15.31:0/3779628347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192010ddf0 msgr2=0x7f192019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19254a3640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192010ddf0 0x7f192019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19254a3640 1 -- 172.21.15.31:0/3779628347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19201a0570 con 0x7f1920105270 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19264a5640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f192010ddf0 0x7f192019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f19254a3640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192019aba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f191002a930 tx=0x7f1910004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191003f030 con 0x7f1920105270 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19201a0800 con 0x7f1920105270 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1910003aa0 con 0x7f1920105270 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910035070 con 0x7f1920105270 2024-09-21T06:10:14.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.837+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19201a6fc0 con 0x7f1920105270 2024-09-21T06:10:14.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.838+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1920063b50 con 0x7f1920105270 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.840+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f1910042030 con 0x7f1920105270 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.840+0000 7f1916ffd640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f190c0b65f0 0x7f190c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f1925ca4640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f190c0b65f0 0x7f190c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f19101081f0 con 0x7f1920105270 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f1916ffd640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] conn(0x7f190c0c03e0 0x7f190c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 --> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f190c0c2fa0 con 0x7f190c0c03e0 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f1910108640 con 0x7f1920105270 2024-09-21T06:10:14.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f1925ca4640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f190c0b65f0 0x7f190c0b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f192006bc00 tx=0x7f191c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:14.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.841+0000 7f19264a5640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] conn(0x7f190c0c03e0 0x7f190c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:14.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.842+0000 7f19264a5640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] conn(0x7f190c0c03e0 0x7f190c0c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:14.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.843+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== osd.2 v2:172.21.15.110:6800/3771685673 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f190c0c2fa0 con 0x7f190c0c03e0 2024-09-21T06:10:14.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.859+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 --> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1920063f50 con 0x7f190c0c03e0 2024-09-21T06:10:14.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.860+0000 7f1916ffd640 1 -- 172.21.15.31:0/3779628347 <== osd.2 v2:172.21.15.110:6800/3771685673 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1920063f50 con 0x7f190c0c03e0 2024-09-21T06:10:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.860+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] conn(0x7f190c0c03e0 msgr2=0x7f190c0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:14.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.860+0000 7f1927f08640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] conn(0x7f190c0c03e0 0x7f190c0c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.860+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f190c0b65f0 msgr2=0x7f190c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.860+0000 7f1927f08640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f190c0b65f0 0x7f190c0b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f192006bc00 tx=0x7f191c009110 comp rx=0 tx=0).stop 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 msgr2=0x7f192019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f1927f08640 1 --2- 172.21.15.31:0/3779628347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1920105270 0x7f192019aba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f191002a930 tx=0x7f1910004270 comp rx=0 tx=0).stop 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f19264a5640 1 -- 172.21.15.31:0/3779628347 reap_dead start 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 shutdown_connections 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 >> 172.21.15.31:0/3779628347 conn(0x7f1920079730 msgr2=0x7f19201008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 shutdown_connections 2024-09-21T06:10:14.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:14.861+0000 7f1927f08640 1 -- 172.21.15.31:0/3779628347 wait complete. 2024-09-21T06:10:15.727 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:15 smithi031 ceph-mon[31237]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 255 B/s wr, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-21T06:10:15.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 -- 172.21.15.31:0/721549053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 msgr2=0x7f71cc110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:15.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/721549053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc110630 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009ca0 tx=0x7f71c402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 -- 172.21.15.31:0/721549053 shutdown_connections 2024-09-21T06:10:15.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/721549053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc110630 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/721549053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71cc1094b0 0x7f71cc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/721549053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71cc103bc0 0x7f71cc103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 -- 172.21.15.31:0/721549053 >> 172.21.15.31:0/721549053 conn(0x7f71cc07b910 msgr2=0x7f71cc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:15.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 -- 172.21.15.31:0/721549053 shutdown_connections 2024-09-21T06:10:15.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.735+0000 7f71d2f3d640 1 -- 172.21.15.31:0/721549053 wait complete. 2024-09-21T06:10:15.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 Processor -- start 2024-09-21T06:10:15.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 -- start start 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71cc103bc0 0x7f71cc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71cc1094b0 0x7f71cc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc113ac0 con 0x7f71cc109e00 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc113940 con 0x7f71cc103bc0 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d2f3d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc113c40 con 0x7f71cc1094b0 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d14da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d14da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33162/0 (socket says 172.21.15.31:33162) 2024-09-21T06:10:15.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d14da640 1 -- 172.21.15.31:0/4018238703 learned_addr learned my addr 172.21.15.31:0/4018238703 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:15.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.736+0000 7f71d0cd9640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71cc103bc0 0x7f71cc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:15.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71cbfff640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71cc1094b0 0x7f71cc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:15.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d14da640 1 -- 172.21.15.31:0/4018238703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71cc1094b0 msgr2=0x7f71cc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:15.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d14da640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71cc1094b0 0x7f71cc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d14da640 1 -- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71cc103bc0 msgr2=0x7f71cc19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d14da640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71cc103bc0 0x7f71cc19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d14da640 1 -- 172.21.15.31:0/4018238703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc1a03c0 con 0x7f71cc109e00 2024-09-21T06:10:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d14da640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc19fc90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71c40098d0 tx=0x7f71c40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4003b70 con 0x7f71cc109e00 2024-09-21T06:10:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4003d10 con 0x7f71cc109e00 2024-09-21T06:10:15.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c402fa20 con 0x7f71cc109e00 2024-09-21T06:10:15.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc1a0650 con 0x7f71cc109e00 2024-09-21T06:10:15.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.737+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a6e10 con 0x7f71cc109e00 2024-09-21T06:10:15.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.738+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71cc063b10 con 0x7f71cc109e00 2024-09-21T06:10:15.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.740+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f71c402fbe0 con 0x7f71cc109e00 2024-09-21T06:10:15.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71c9ffb640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71b40b6670 0x7f71b40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:15.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71d0cd9640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71b40b6670 0x7f71b40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:15.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f71c4108ba0 con 0x7f71cc109e00 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71c9ffb640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] conn(0x7f71b40c0460 0x7f71b40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 --> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71b40c3020 con 0x7f71b40c0460 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f71c4108f50 con 0x7f71cc109e00 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71d0cd9640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71b40b6670 0x7f71b40b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005fd0 tx=0x7f71bc00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.741+0000 7f71cbfff640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] conn(0x7f71b40c0460 0x7f71b40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.742+0000 7f71cbfff640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] conn(0x7f71b40c0460 0x7f71b40c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.742+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== osd.1 v2:172.21.15.107:6800/2850742213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71b40c3020 con 0x7f71b40c0460 2024-09-21T06:10:15.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.756+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 --> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71cc063fe0 con 0x7f71b40c0460 2024-09-21T06:10:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.756+0000 7f71c9ffb640 1 -- 172.21.15.31:0/4018238703 <== osd.1 v2:172.21.15.107:6800/2850742213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f71cc063fe0 con 0x7f71b40c0460 2024-09-21T06:10:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] conn(0x7f71b40c0460 msgr2=0x7f71b40c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] conn(0x7f71b40c0460 0x7f71b40c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71b40b6670 msgr2=0x7f71b40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71b40b6670 0x7f71b40b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71bc005fd0 tx=0x7f71bc00b390 comp rx=0 tx=0).stop 2024-09-21T06:10:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 msgr2=0x7f71cc19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 --2- 172.21.15.31:0/4018238703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71cc109e00 0x7f71cc19fc90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71c40098d0 tx=0x7f71c40317b0 comp rx=0 tx=0).stop 2024-09-21T06:10:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d14da640 1 -- 172.21.15.31:0/4018238703 reap_dead start 2024-09-21T06:10:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 shutdown_connections 2024-09-21T06:10:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 >> 172.21.15.31:0/4018238703 conn(0x7f71cc07b910 msgr2=0x7f71cc1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 shutdown_connections 2024-09-21T06:10:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:15.757+0000 7f71d2f3d640 1 -- 172.21.15.31:0/4018238703 wait complete. 2024-09-21T06:10:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:15 smithi110 ceph-mon[34550]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 255 B/s wr, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-21T06:10:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:15 smithi107 ceph-mon[34380]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 255 B/s wr, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-21T06:10:16.564 INFO:teuthology.orchestra.run.smithi031.stdout:60129542215 2024-09-21T06:10:16.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T06:10:16.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:16 smithi110 ceph-mon[34550]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 222 B/s wr, 2 op/s; 12 B/s, 0 objects/s recovering 2024-09-21T06:10:16.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:16 smithi107 ceph-mon[34380]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 222 B/s wr, 2 op/s; 12 B/s, 0 objects/s recovering 2024-09-21T06:10:16.845 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:16 smithi031 ceph-mon[31237]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 222 B/s wr, 2 op/s; 12 B/s, 0 objects/s recovering 2024-09-21T06:10:16.903 INFO:teuthology.orchestra.run.smithi031.stdout:111669149755 2024-09-21T06:10:16.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.071+0000 7f4e70504640 1 -- 172.21.15.31:0/4277384852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6810c040 msgr2=0x7f4e68112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.071+0000 7f4e70504640 1 --2- 172.21.15.31:0/4277384852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6810c040 0x7f4e68112870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009ca0 tx=0x7f4e6002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.072+0000 7f4e70504640 1 -- 172.21.15.31:0/4277384852 shutdown_connections 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.072+0000 7f4e70504640 1 --2- 172.21.15.31:0/4277384852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e6810c040 0x7f4e68112870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.072+0000 7f4e70504640 1 --2- 172.21.15.31:0/4277384852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6810b6f0 0x7f4e6810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.072+0000 7f4e70504640 1 --2- 172.21.15.31:0/4277384852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e68105db0 0x7f4e681061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.073+0000 7f4e70504640 1 -- 172.21.15.31:0/4277384852 >> 172.21.15.31:0/4277384852 conn(0x7f4e681017c0 msgr2=0x7f4e68103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.073+0000 7f4e70504640 1 -- 172.21.15.31:0/4277384852 shutdown_connections 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.073+0000 7f4e70504640 1 -- 172.21.15.31:0/4277384852 wait complete. 2024-09-21T06:10:17.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 Processor -- start 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 -- start start 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105db0 0x7f4e6819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6810b6f0 0x7f4e6819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6810c040 0x7f4e681a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68115d00 con 0x7f4e68105db0 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68115b80 con 0x7f4e6810b6f0 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e70504640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68115e80 con 0x7f4e6810c040 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e6e2a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105db0 0x7f4e6819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e6e2a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105db0 0x7f4e6819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33182/0 (socket says 172.21.15.31:33182) 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e6e2a0640 1 -- 172.21.15.31:0/2058099508 learned_addr learned my addr 172.21.15.31:0/2058099508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:17.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.074+0000 7f4e6eaa1640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6810c040 0x7f4e681a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6da9f640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6810b6f0 0x7f4e6819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6eaa1640 1 -- 172.21.15.31:0/2058099508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6810b6f0 msgr2=0x7f4e6819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6eaa1640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6810b6f0 0x7f4e6819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6eaa1640 1 -- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105db0 msgr2=0x7f4e6819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6eaa1640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105db0 0x7f4e6819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6eaa1640 1 -- 172.21.15.31:0/2058099508 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e681a2680 con 0x7f4e6810c040 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e6eaa1640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6810c040 0x7f4e681a1f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e6002ade0 tx=0x7f4e60031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60003b70 con 0x7f4e6810c040 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e60031910 con 0x7f4e6810c040 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.075+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e681a2910 con 0x7f4e6810c040 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.076+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e681a90d0 con 0x7f4e6810c040 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.076+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6002f950 con 0x7f4e6810c040 2024-09-21T06:10:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.077+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e68063ad0 con 0x7f4e6810c040 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.079+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4e6002faf0 con 0x7f4e6810c040 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.080+0000 7f4e5f7fe640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4e540b6670 0x7f4e540b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e6e2a0640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4e540b6670 0x7f4e540b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4e60108890 con 0x7f4e6810c040 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e5f7fe640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] conn(0x7f4e540c0460 0x7f4e540c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 --> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e540c3020 con 0x7f4e540c0460 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4e60108c40 con 0x7f4e6810c040 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e6da9f640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] conn(0x7f4e540c0460 0x7f4e540c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.081+0000 7f4e6e2a0640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4e540b6670 0x7f4e540b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009cd0 tx=0x7f4e64005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.082+0000 7f4e6da9f640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] conn(0x7f4e540c0460 0x7f4e540c2950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.083+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== osd.6 v2:172.21.15.31:6818/3870004119 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e540c3020 con 0x7f4e540c0460 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.112+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 --> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e68063f10 con 0x7f4e540c0460 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e5f7fe640 1 -- 172.21.15.31:0/2058099508 <== osd.6 v2:172.21.15.31:6818/3870004119 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4e68063f10 con 0x7f4e540c0460 2024-09-21T06:10:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] conn(0x7f4e540c0460 msgr2=0x7f4e540c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e70504640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] conn(0x7f4e540c0460 0x7f4e540c2950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4e540b6670 msgr2=0x7f4e540b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e70504640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4e540b6670 0x7f4e540b8b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009cd0 tx=0x7f4e64005f90 comp rx=0 tx=0).stop 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6810c040 msgr2=0x7f4e681a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.113+0000 7f4e70504640 1 --2- 172.21.15.31:0/2058099508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e6810c040 0x7f4e681a1f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e6002ade0 tx=0x7f4e60031660 comp rx=0 tx=0).stop 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.114+0000 7f4e6eaa1640 1 -- 172.21.15.31:0/2058099508 reap_dead start 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.114+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 shutdown_connections 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.114+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 >> 172.21.15.31:0/2058099508 conn(0x7f4e681017c0 msgr2=0x7f4e68108c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.114+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 shutdown_connections 2024-09-21T06:10:17.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.114+0000 7f4e70504640 1 -- 172.21.15.31:0/2058099508 wait complete. 2024-09-21T06:10:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 -- 172.21.15.31:0/94222934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac0102a60 msgr2=0x7f9ac0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/94222934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac0102a60 0x7f9ac0102e70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 -- 172.21.15.31:0/94222934 shutdown_connections 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/94222934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac0108cf0 0x7f9ac010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/94222934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac01083a0 0x7f9ac01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/94222934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac0102a60 0x7f9ac0102e70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 -- 172.21.15.31:0/94222934 >> 172.21.15.31:0/94222934 conn(0x7f9ac0079730 msgr2=0x7f9ac0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 -- 172.21.15.31:0/94222934 shutdown_connections 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.967+0000 7f9ac7e96640 1 -- 172.21.15.31:0/94222934 wait complete. 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.968+0000 7f9ac7e96640 1 Processor -- start 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.968+0000 7f9ac7e96640 1 -- start start 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac7e96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac0102a60 0x7f9ac019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac7e96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac01083a0 0x7f9ac019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac7e96640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac0108cf0 0x7f9ac019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac7e96640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0112960 con 0x7f9ac01083a0 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac7e96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac01127e0 con 0x7f9ac0102a60 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac7e96640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac0112ae0 con 0x7f9ac0108cf0 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac01083a0 0x7f9ac019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac01083a0 0x7f9ac019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33220/0 (socket says 172.21.15.31:33220) 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 -- 172.21.15.31:0/1390766320 learned_addr learned my addr 172.21.15.31:0/1390766320 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac6433640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac0108cf0 0x7f9ac019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5c32640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac0102a60 0x7f9ac019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 -- 172.21.15.31:0/1390766320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac0108cf0 msgr2=0x7f9ac019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ac0108cf0 0x7f9ac019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 -- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac0102a60 msgr2=0x7f9ac019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac0102a60 0x7f9ac019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 -- 172.21.15.31:0/1390766320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac01a0310 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ac5431640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac01083a0 0x7f9ac019aa00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0005fd0 tx=0x7f9ab000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.969+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0017070 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.970+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab0009c30 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.970+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac01a0600 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.970+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0003d30 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.970+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac01a6e20 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.970+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ac0101520 con 0x7f9ac01083a0 2024-09-21T06:10:18.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.972+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9ab0003ed0 con 0x7f9ac01083a0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ab6ffd640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9aac0b6620 0x7f9aac0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ac5c32640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9aac0b6620 0x7f9aac0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f9ab00e7620 con 0x7f9ac01083a0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ab6ffd640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] conn(0x7f9aac0c0410 0x7f9aac0c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 --> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ab001ce50 con 0x7f9aac0c0410 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9ab00e79d0 con 0x7f9ac01083a0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.973+0000 7f9ac5c32640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9aac0b6620 0x7f9aac0b8b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9abc031440 tx=0x7f9abc009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.974+0000 7f9ac6433640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] conn(0x7f9aac0c0410 0x7f9aac0c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.974+0000 7f9ac6433640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] conn(0x7f9aac0c0410 0x7f9aac0c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.975+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== osd.4 v2:172.21.15.107:6808/888650447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ab001ce50 con 0x7f9aac0c0410 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.988+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 --> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ac0103ce0 con 0x7f9aac0c0410 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.989+0000 7f9ab6ffd640 1 -- 172.21.15.31:0/1390766320 <== osd.4 v2:172.21.15.107:6808/888650447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ac0103ce0 con 0x7f9aac0c0410 2024-09-21T06:10:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.989+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] conn(0x7f9aac0c0410 msgr2=0x7f9aac0c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.989+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] conn(0x7f9aac0c0410 0x7f9aac0c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9aac0b6620 msgr2=0x7f9aac0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9aac0b6620 0x7f9aac0b8b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9abc031440 tx=0x7f9abc009560 comp rx=0 tx=0).stop 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac01083a0 msgr2=0x7f9ac019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 --2- 172.21.15.31:0/1390766320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ac01083a0 0x7f9ac019aa00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0005fd0 tx=0x7f9ab000dde0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac6433640 1 -- 172.21.15.31:0/1390766320 reap_dead start 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 shutdown_connections 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 >> 172.21.15.31:0/1390766320 conn(0x7f9ac0079730 msgr2=0x7f9ac0106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 shutdown_connections 2024-09-21T06:10:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.990+0000 7f9ac7e96640 1 -- 172.21.15.31:0/1390766320 wait complete. 2024-09-21T06:10:18.685 INFO:teuthology.orchestra.run.smithi031.stdout:133143986231 2024-09-21T06:10:18.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T06:10:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 -- 172.21.15.31:0/851243300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc109e10 msgr2=0x7faadc110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 --2- 172.21.15.31:0/851243300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc109e10 0x7faadc110640 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faad4009ca0 tx=0x7faad402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 -- 172.21.15.31:0/851243300 shutdown_connections 2024-09-21T06:10:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 --2- 172.21.15.31:0/851243300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc109e10 0x7faadc110640 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 --2- 172.21.15.31:0/851243300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc1094c0 0x7faadc1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 --2- 172.21.15.31:0/851243300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faadc103b80 0x7faadc103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 -- 172.21.15.31:0/851243300 >> 172.21.15.31:0/851243300 conn(0x7faadc0ff610 msgr2=0x7faadc101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 -- 172.21.15.31:0/851243300 shutdown_connections 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.857+0000 7faae2104640 1 -- 172.21.15.31:0/851243300 wait complete. 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 Processor -- start 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 -- start start 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faadc103b80 0x7faadc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc1094c0 0x7faadc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc109e10 0x7faadc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faadc113b30 con 0x7faadc1094c0 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faadc1139b0 con 0x7faadc109e10 2024-09-21T06:10:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faae2104640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faadc113cb0 con 0x7faadc103b80 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faadaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc1094c0 0x7faadc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faadaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc1094c0 0x7faadc19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33246/0 (socket says 172.21.15.31:33246) 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.858+0000 7faadaffd640 1 -- 172.21.15.31:0/3642233979 learned_addr learned my addr 172.21.15.31:0/3642233979 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadbfff640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc109e10 0x7faadc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadb7fe640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faadc103b80 0x7faadc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadaffd640 1 -- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faadc103b80 msgr2=0x7faadc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadaffd640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faadc103b80 0x7faadc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadaffd640 1 -- 172.21.15.31:0/3642233979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc109e10 msgr2=0x7faadc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadaffd640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc109e10 0x7faadc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadaffd640 1 -- 172.21.15.31:0/3642233979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc1a0410 con 0x7faadc1094c0 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faadaffd640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc1094c0 0x7faadc19aa40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faad000d950 tx=0x7faad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad0017070 con 0x7faadc1094c0 2024-09-21T06:10:18.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc1a0700 con 0x7faadc1094c0 2024-09-21T06:10:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad0009c30 con 0x7faadc1094c0 2024-09-21T06:10:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad00047a0 con 0x7faadc1094c0 2024-09-21T06:10:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.859+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1a6e60 con 0x7faadc1094c0 2024-09-21T06:10:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.860+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faadc063b50 con 0x7faadc1094c0 2024-09-21T06:10:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.862+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7faad0004020 con 0x7faadc1094c0 2024-09-21T06:10:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faad8ff9640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faac40b65f0 0x7faac40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faadb7fe640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faac40b65f0 0x7faac40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7faad00e7950 con 0x7faadc1094c0 2024-09-21T06:10:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faad8ff9640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] conn(0x7faac40c03e0 0x7faac40c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 --> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faac40c2fa0 con 0x7faac40c03e0 2024-09-21T06:10:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7faad00e7da0 con 0x7faadc1094c0 2024-09-21T06:10:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.863+0000 7faadbfff640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] conn(0x7faac40c03e0 0x7faac40c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.864+0000 7faadb7fe640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faac40b65f0 0x7faac40b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faacc009890 tx=0x7faacc008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.864+0000 7faadbfff640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] conn(0x7faac40c03e0 0x7faac40c28d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.865+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== osd.11 v2:172.21.15.110:6824/2652663870 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faac40c2fa0 con 0x7faac40c03e0 2024-09-21T06:10:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.879+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 --> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faadc063f80 con 0x7faac40c03e0 2024-09-21T06:10:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.879+0000 7faad8ff9640 1 -- 172.21.15.31:0/3642233979 <== osd.11 v2:172.21.15.110:6824/2652663870 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faadc063f80 con 0x7faac40c03e0 2024-09-21T06:10:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] conn(0x7faac40c03e0 msgr2=0x7faac40c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] conn(0x7faac40c03e0 0x7faac40c28d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faac40b65f0 msgr2=0x7faac40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faac40b65f0 0x7faac40b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faacc009890 tx=0x7faacc008040 comp rx=0 tx=0).stop 2024-09-21T06:10:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc1094c0 msgr2=0x7faadc19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 --2- 172.21.15.31:0/3642233979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faadc1094c0 0x7faadc19aa40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faad000d950 tx=0x7faad000de10 comp rx=0 tx=0).stop 2024-09-21T06:10:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faadbfff640 1 -- 172.21.15.31:0/3642233979 reap_dead start 2024-09-21T06:10:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 shutdown_connections 2024-09-21T06:10:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 >> 172.21.15.31:0/3642233979 conn(0x7faadc0ff610 msgr2=0x7faadc108ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 shutdown_connections 2024-09-21T06:10:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:18.880+0000 7faae2104640 1 -- 172.21.15.31:0/3642233979 wait complete. 2024-09-21T06:10:19.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:18 smithi110 ceph-mon[34550]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 200 B/s wr, 2 op/s; 11 B/s, 0 objects/s recovering 2024-09-21T06:10:19.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:18 smithi107 ceph-mon[34380]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 200 B/s wr, 2 op/s; 11 B/s, 0 objects/s recovering 2024-09-21T06:10:19.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:18 smithi031 ceph-mon[31237]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 200 B/s wr, 2 op/s; 11 B/s, 0 objects/s recovering 2024-09-21T06:10:19.540 INFO:teuthology.orchestra.run.smithi031.stdout:180388626478 2024-09-21T06:10:19.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T06:10:19.668 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:19.668 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 -- 172.21.15.31:0/2436430607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 msgr2=0x7fb788110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 --2- 172.21.15.31:0/2436430607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb788110330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ca0 tx=0x7fb78002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 -- 172.21.15.31:0/2436430607 shutdown_connections 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 --2- 172.21.15.31:0/2436430607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb788110330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 --2- 172.21.15.31:0/2436430607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb788105380 0x7fb78810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 --2- 172.21.15.31:0/2436430607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788104a30 0x7fb788104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 -- 172.21.15.31:0/2436430607 >> 172.21.15.31:0/2436430607 conn(0x7fb7880797b0 msgr2=0x7fb788079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 -- 172.21.15.31:0/2436430607 shutdown_connections 2024-09-21T06:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.674+0000 7fb78e593640 1 -- 172.21.15.31:0/2436430607 wait complete. 2024-09-21T06:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.675+0000 7fb78e593640 1 Processor -- start 2024-09-21T06:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.675+0000 7fb78e593640 1 -- start start 2024-09-21T06:10:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78e593640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb788104a30 0x7fb78819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78e593640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788105380 0x7fb78819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78e593640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb78819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78e593640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb788113ac0 con 0x7fb78810de60 2024-09-21T06:10:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78e593640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb788113940 con 0x7fb788105380 2024-09-21T06:10:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78e593640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb788113c40 con 0x7fb788104a30 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78cb30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb78819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78cb30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb78819fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59152/0 (socket says 172.21.15.31:59152) 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb78cb30640 1 -- 172.21.15.31:0/2640547770 learned_addr learned my addr 172.21.15.31:0/2640547770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.676+0000 7fb787fff640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb788104a30 0x7fb78819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78cb30640 1 -- 172.21.15.31:0/2640547770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb788104a30 msgr2=0x7fb78819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78cb30640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb788104a30 0x7fb78819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78cb30640 1 -- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788105380 msgr2=0x7fb78819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78cb30640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788105380 0x7fb78819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78cb30640 1 -- 172.21.15.31:0/2640547770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7881a0420 con 0x7fb78810de60 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78cb30640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb78819fcf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb780005cc0 tx=0x7fb78002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780003bf0 con 0x7fb78810de60 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb780003d90 con 0x7fb78810de60 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881a06b0 con 0x7fb78810de60 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.677+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002fbc0 con 0x7fb78810de60 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.678+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb788102650 con 0x7fb78810de60 2024-09-21T06:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.679+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb788100f80 con 0x7fb78810de60 2024-09-21T06:10:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.682+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb78002fd60 con 0x7fb78810de60 2024-09-21T06:10:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.683+0000 7fb7857fa640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb7700b6670 0x7fb7700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.683+0000 7fb787fff640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb7700b6670 0x7fb7700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.684+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb780108d40 con 0x7fb78810de60 2024-09-21T06:10:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.684+0000 7fb7857fa640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] conn(0x7fb7700c0460 0x7fb7700c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.684+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 --> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7700c3020 con 0x7fb7700c0460 2024-09-21T06:10:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.684+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fb780109190 con 0x7fb78810de60 2024-09-21T06:10:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.684+0000 7fb787fff640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb7700b6670 0x7fb7700b8b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb778005fd0 tx=0x7fb778005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.684+0000 7fb7877fe640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] conn(0x7fb7700c0460 0x7fb7700c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.685+0000 7fb7877fe640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] conn(0x7fb7700c0460 0x7fb7700c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.686+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== osd.7 v2:172.21.15.107:6816/1767013823 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7700c3020 con 0x7fb7700c0460 2024-09-21T06:10:19.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.715+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 --> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb788101190 con 0x7fb7700c0460 2024-09-21T06:10:19.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb7857fa640 1 -- 172.21.15.31:0/2640547770 <== osd.7 v2:172.21.15.107:6816/1767013823 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb788101190 con 0x7fb7700c0460 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] conn(0x7fb7700c0460 msgr2=0x7fb7700c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] conn(0x7fb7700c0460 0x7fb7700c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb7700b6670 msgr2=0x7fb7700b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb7700b6670 0x7fb7700b8b80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb778005fd0 tx=0x7fb778005850 comp rx=0 tx=0).stop 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 msgr2=0x7fb78819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 --2- 172.21.15.31:0/2640547770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb78810de60 0x7fb78819fcf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb780005cc0 tx=0x7fb78002f380 comp rx=0 tx=0).stop 2024-09-21T06:10:19.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78cb30640 1 -- 172.21.15.31:0/2640547770 reap_dead start 2024-09-21T06:10:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 shutdown_connections 2024-09-21T06:10:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 >> 172.21.15.31:0/2640547770 conn(0x7fb7880797b0 msgr2=0x7fb7880ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 shutdown_connections 2024-09-21T06:10:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:19.716+0000 7fb78e593640 1 -- 172.21.15.31:0/2640547770 wait complete. 2024-09-21T06:10:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 -- 172.21.15.31:0/2684336084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d01083a0 msgr2=0x7fe6d01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 --2- 172.21.15.31:0/2684336084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d01083a0 0x7fe6d01087b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00d910 tx=0x7fe6bc02c380 comp rx=0 tx=0).stop 2024-09-21T06:10:19.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 -- 172.21.15.31:0/2684336084 shutdown_connections 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 --2- 172.21.15.31:0/2684336084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6d0108cf0 0x7fe6d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 --2- 172.21.15.31:0/2684336084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d01083a0 0x7fe6d01087b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 --2- 172.21.15.31:0/2684336084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6d0102a60 0x7fe6d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 -- 172.21.15.31:0/2684336084 >> 172.21.15.31:0/2684336084 conn(0x7fe6d0079730 msgr2=0x7fe6d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 -- 172.21.15.31:0/2684336084 shutdown_connections 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.169+0000 7fe6d7155640 1 -- 172.21.15.31:0/2684336084 wait complete. 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.170+0000 7fe6d7155640 1 Processor -- start 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.170+0000 7fe6d7155640 1 -- start start 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d7155640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6d0102a60 0x7fe6d0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d7155640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6d01083a0 0x7fe6d0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d7155640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d0108cf0 0x7fe6d019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d7155640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0111860 con 0x7fe6d0108cf0 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d7155640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d01116e0 con 0x7fe6d0102a60 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d7155640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d01119e0 con 0x7fe6d01083a0 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d0108cf0 0x7fe6d019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d0108cf0 0x7fe6d019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33198/0 (socket says 172.21.15.31:33198) 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 -- 172.21.15.31:0/3072637324 learned_addr learned my addr 172.21.15.31:0/3072637324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6cffff640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6d01083a0 0x7fe6d0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d4ef1640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6d0102a60 0x7fe6d0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 -- 172.21.15.31:0/3072637324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6d01083a0 msgr2=0x7fe6d0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6d01083a0 0x7fe6d0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 -- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6d0102a60 msgr2=0x7fe6d0198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6d0102a60 0x7fe6d0198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6d56f2640 1 -- 172.21.15.31:0/3072637324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d019e1e0 con 0x7fe6d0108cf0 2024-09-21T06:10:19.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.171+0000 7fe6cffff640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6d01083a0 0x7fe6d0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.172+0000 7fe6d56f2640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d0108cf0 0x7fe6d019dab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800eb10 tx=0x7fe6c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.172+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8004b20 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.172+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d019e4d0 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.172+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c8004cc0 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.172+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8010690 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.172+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d01a9040 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.173+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6d0063b50 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.175+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe6c8005000 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.175+0000 7fe6cdffb640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe6c00b65f0 0x7fe6c00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6d4ef1640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe6c00b65f0 0x7fe6c00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe6c80e7c30 con 0x7fe6d0108cf0 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6cdffb640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] conn(0x7fe6c00c03e0 0x7fe6c00c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:19.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 --> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6c8019e70 con 0x7fe6c00c03e0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe6c80ec2a0 con 0x7fe6d0108cf0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6d4ef1640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe6c00b65f0 0x7fe6c00b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009bd0 tx=0x7fe6c4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.176+0000 7fe6cffff640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] conn(0x7fe6c00c03e0 0x7fe6c00c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.177+0000 7fe6cffff640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] conn(0x7fe6c00c03e0 0x7fe6c00c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.178+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== osd.10 v2:172.21.15.107:6824/3727307895 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6c8019e70 con 0x7fe6c00c03e0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.193+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 --> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe6d0064000 con 0x7fe6c00c03e0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.193+0000 7fe6cdffb640 1 -- 172.21.15.31:0/3072637324 <== osd.10 v2:172.21.15.107:6824/3727307895 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe6d0064000 con 0x7fe6c00c03e0 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] conn(0x7fe6c00c03e0 msgr2=0x7fe6c00c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] conn(0x7fe6c00c03e0 0x7fe6c00c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe6c00b65f0 msgr2=0x7fe6c00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe6c00b65f0 0x7fe6c00b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009bd0 tx=0x7fe6c4005f90 comp rx=0 tx=0).stop 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d0108cf0 msgr2=0x7fe6d019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 --2- 172.21.15.31:0/3072637324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6d0108cf0 0x7fe6d019dab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800eb10 tx=0x7fe6c800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d56f2640 1 -- 172.21.15.31:0/3072637324 reap_dead start 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 shutdown_connections 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 >> 172.21.15.31:0/3072637324 conn(0x7fe6d0079730 msgr2=0x7fe6d0104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 shutdown_connections 2024-09-21T06:10:19.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:17.194+0000 7fe6d7155640 1 -- 172.21.15.31:0/3072637324 wait complete. 2024-09-21T06:10:20.557 INFO:teuthology.orchestra.run.smithi031.stdout:274877906970 2024-09-21T06:10:20.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T06:10:20.776 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:20 smithi031 ceph-mon[31237]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T06:10:21.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:20 smithi110 ceph-mon[34550]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T06:10:21.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:20 smithi107 ceph-mon[34380]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T06:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout:296352743445 2024-09-21T06:10:21.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 -- 172.21.15.31:0/3519655788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89981094a0 msgr2=0x7f89981098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 --2- 172.21.15.31:0/3519655788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89981094a0 0x7f89981098b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f898c009900 tx=0x7f898c02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 -- 172.21.15.31:0/3519655788 shutdown_connections 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 --2- 172.21.15.31:0/3519655788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8998109df0 0x7f8998110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 --2- 172.21.15.31:0/3519655788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89981094a0 0x7f89981098b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 --2- 172.21.15.31:0/3519655788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 0x7f8998103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 -- 172.21.15.31:0/3519655788 >> 172.21.15.31:0/3519655788 conn(0x7f899807b850 msgr2=0x7f899807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 -- 172.21.15.31:0/3519655788 shutdown_connections 2024-09-21T06:10:21.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.897+0000 7f899d52a640 1 -- 172.21.15.31:0/3519655788 wait complete. 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.898+0000 7f899d52a640 1 Processor -- start 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.898+0000 7f899d52a640 1 -- start start 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f899d52a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 0x7f899819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f899d52a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89981094a0 0x7f899819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f899d52a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998109df0 0x7f899819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f899d52a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8998113ab0 con 0x7f8998109df0 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f899d52a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8998113930 con 0x7f89981094a0 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f899d52a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8998113c30 con 0x7f8998103b60 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998109df0 0x7f899819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 0x7f899819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998109df0 0x7f899819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59170/0 (socket says 172.21.15.31:59170) 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 0x7f899819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38656/0 (socket says 172.21.15.31:38656) 2024-09-21T06:10:21.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 -- 172.21.15.31:0/3755899655 learned_addr learned my addr 172.21.15.31:0/3755899655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89967fc640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89981094a0 0x7f899819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 -- 172.21.15.31:0/3755899655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 msgr2=0x7f899819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 0x7f899819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 -- 172.21.15.31:0/3755899655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89981094a0 msgr2=0x7f899819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89981094a0 0x7f899819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 -- 172.21.15.31:0/3755899655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89981a0350 con 0x7f8998109df0 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f8996ffd640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8998103b60 0x7f899819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89967fc640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89981094a0 0x7f899819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.899+0000 7f89977fe640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998109df0 0x7f899819fce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f899000cce0 tx=0x7f899000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.900+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899000f450 con 0x7f8998109df0 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.900+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89981a0640 con 0x7f8998109df0 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.900+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f899000faa0 con 0x7f8998109df0 2024-09-21T06:10:21.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.900+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990018b70 con 0x7f8998109df0 2024-09-21T06:10:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.900+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a6e60 con 0x7f8998109df0 2024-09-21T06:10:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.900+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8998063b50 con 0x7f8998109df0 2024-09-21T06:10:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.902+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8990018d10 con 0x7f8998109df0 2024-09-21T06:10:21.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.903+0000 7f897ffff640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89800b6620 0x7f89800b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.903+0000 7f8996ffd640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89800b6620 0x7f89800b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.903+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f89900e7de0 con 0x7f8998109df0 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.904+0000 7f897ffff640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] conn(0x7f89800c0410 0x7f89800c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.904+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 --> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89800c2fd0 con 0x7f89800c0410 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.904+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f89900e8230 con 0x7f8998109df0 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.904+0000 7f89967fc640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] conn(0x7f89800c0410 0x7f89800c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.904+0000 7f8996ffd640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89800b6620 0x7f89800b8b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8988005d00 tx=0x7f898800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.904+0000 7f89967fc640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] conn(0x7f89800c0410 0x7f89800c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:21.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.905+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== osd.0 v2:172.21.15.31:6802/215542336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89800c2fd0 con 0x7f89800c0410 2024-09-21T06:10:21.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.919+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 --> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8998063f50 con 0x7f89800c0410 2024-09-21T06:10:21.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.919+0000 7f897ffff640 1 -- 172.21.15.31:0/3755899655 <== osd.0 v2:172.21.15.31:6802/215542336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8998063f50 con 0x7f89800c0410 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] conn(0x7f89800c0410 msgr2=0x7f89800c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] conn(0x7f89800c0410 0x7f89800c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89800b6620 msgr2=0x7f89800b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89800b6620 0x7f89800b8b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8988005d00 tx=0x7f898800b410 comp rx=0 tx=0).stop 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998109df0 msgr2=0x7f899819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 --2- 172.21.15.31:0/3755899655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998109df0 0x7f899819fce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f899000cce0 tx=0x7f899000a620 comp rx=0 tx=0).stop 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f89977fe640 1 -- 172.21.15.31:0/3755899655 reap_dead start 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 shutdown_connections 2024-09-21T06:10:21.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 >> 172.21.15.31:0/3755899655 conn(0x7f899807b850 msgr2=0x7f8998105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:21.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 shutdown_connections 2024-09-21T06:10:21.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.920+0000 7f899d52a640 1 -- 172.21.15.31:0/3755899655 wait complete. 2024-09-21T06:10:22.546 INFO:teuthology.orchestra.run.smithi031.stdout:154618822705 2024-09-21T06:10:22.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T06:10:22.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 -- 172.21.15.31:0/2781994044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c103b70 msgr2=0x7f599c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:22.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 --2- 172.21.15.31:0/2781994044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c103b70 0x7f599c103f80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f598c009900 tx=0x7f598c02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:22.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 -- 172.21.15.31:0/2781994044 shutdown_connections 2024-09-21T06:10:22.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 --2- 172.21.15.31:0/2781994044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c109e00 0x7f599c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 --2- 172.21.15.31:0/2781994044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599c1094b0 0x7f599c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 --2- 172.21.15.31:0/2781994044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c103b70 0x7f599c103f80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.814+0000 7f59a2815640 1 -- 172.21.15.31:0/2781994044 >> 172.21.15.31:0/2781994044 conn(0x7f599c0ff5e0 msgr2=0x7f599c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.815+0000 7f59a2815640 1 -- 172.21.15.31:0/2781994044 shutdown_connections 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.815+0000 7f59a2815640 1 -- 172.21.15.31:0/2781994044 wait complete. 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.815+0000 7f59a2815640 1 Processor -- start 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 -- start start 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599c103b70 0x7f599c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1094b0 0x7f599c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c109e00 0x7f599c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f599c113ac0 con 0x7f599c109e00 2024-09-21T06:10:22.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f599c113940 con 0x7f599c103b70 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a2815640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f599c113c40 con 0x7f599c1094b0 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a0db2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c109e00 0x7f599c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a0db2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c109e00 0x7f599c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59166/0 (socket says 172.21.15.31:59166) 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f59a0db2640 1 -- 172.21.15.31:0/918374632 learned_addr learned my addr 172.21.15.31:0/918374632 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f599bfff640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599c103b70 0x7f599c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.816+0000 7f599b7fe640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1094b0 0x7f599c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a0db2640 1 -- 172.21.15.31:0/918374632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1094b0 msgr2=0x7f599c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:22.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a0db2640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f599c1094b0 0x7f599c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a0db2640 1 -- 172.21.15.31:0/918374632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599c103b70 msgr2=0x7f599c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a0db2640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599c103b70 0x7f599c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a0db2640 1 -- 172.21.15.31:0/918374632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c1a0400 con 0x7f599c109e00 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a0db2640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c109e00 0x7f599c19fcd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f599400eaa0 tx=0x7f599400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599400cdb0 con 0x7f599c109e00 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5994005020 con 0x7f599c109e00 2024-09-21T06:10:22.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c1a06f0 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59940116d0 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.817+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c1a6e50 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.818+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f599c063b50 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.820+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5994004b20 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f59997fa640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59840b65f0 0x7f59840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f599bfff640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59840b65f0 0x7f59840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f59940e7cd0 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f59997fa640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] conn(0x7f59840c03e0 0x7f59840c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 --> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5994014ec0 con 0x7f59840c03e0 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f59940e8080 con 0x7f599c109e00 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.821+0000 7f599b7fe640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] conn(0x7f59840c03e0 0x7f59840c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.822+0000 7f599bfff640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59840b65f0 0x7f59840b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f598c031440 tx=0x7f598c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:22.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.822+0000 7f599b7fe640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] conn(0x7f59840c03e0 0x7f59840c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.823+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== osd.9 v2:172.21.15.31:6826/1883621951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5994014ec0 con 0x7f59840c03e0 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.836+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 --> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f599c063f50 con 0x7f59840c03e0 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59997fa640 1 -- 172.21.15.31:0/918374632 <== osd.9 v2:172.21.15.31:6826/1883621951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f599c063f50 con 0x7f59840c03e0 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] conn(0x7f59840c03e0 msgr2=0x7f59840c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] conn(0x7f59840c03e0 0x7f59840c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59840b65f0 msgr2=0x7f59840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59840b65f0 0x7f59840b8b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f598c031440 tx=0x7f598c009560 comp rx=0 tx=0).stop 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c109e00 msgr2=0x7f599c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 --2- 172.21.15.31:0/918374632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c109e00 0x7f599c19fcd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f599400eaa0 tx=0x7f599400ef60 comp rx=0 tx=0).stop 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a0db2640 1 -- 172.21.15.31:0/918374632 reap_dead start 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 shutdown_connections 2024-09-21T06:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 >> 172.21.15.31:0/918374632 conn(0x7f599c0ff5e0 msgr2=0x7f599c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 shutdown_connections 2024-09-21T06:10:22.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:21.837+0000 7f59a2815640 1 -- 172.21.15.31:0/918374632 wait complete. 2024-09-21T06:10:22.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:22 smithi031 ceph-mon[31237]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T06:10:23.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:22 smithi110 ceph-mon[34550]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T06:10:23.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:22 smithi107 ceph-mon[34380]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s; 9 B/s, 0 objects/s recovering 2024-09-21T06:10:23.259 INFO:teuthology.orchestra.run.smithi031.stdout:249108103200 2024-09-21T06:10:23.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T06:10:24.054 INFO:teuthology.orchestra.run.smithi031.stdout:38654705741 2024-09-21T06:10:24.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T06:10:24.573 INFO:teuthology.orchestra.run.smithi031.stdout:201863462952 2024-09-21T06:10:24.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T06:10:24.617 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:24.617 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:24.623 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:24.645 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:24.821 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:24 smithi031 ceph-mon[31237]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:25.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:24 smithi110 ceph-mon[34550]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:25.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:24 smithi107 ceph-mon[34380]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:25.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.409+0000 7f5b5391b640 1 -- 172.21.15.31:0/1024944355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c102a60 msgr2=0x7f5b4c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.409+0000 7f5b5391b640 1 --2- 172.21.15.31:0/1024944355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c102a60 0x7f5b4c102e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009900 tx=0x7f5b4802a410 comp rx=0 tx=0).stop 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 -- 172.21.15.31:0/1024944355 shutdown_connections 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 --2- 172.21.15.31:0/1024944355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 --2- 172.21.15.31:0/1024944355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c1083a0 0x7f5b4c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 --2- 172.21.15.31:0/1024944355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c102a60 0x7f5b4c102e70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 -- 172.21.15.31:0/1024944355 >> 172.21.15.31:0/1024944355 conn(0x7f5b4c079730 msgr2=0x7f5b4c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 -- 172.21.15.31:0/1024944355 shutdown_connections 2024-09-21T06:10:25.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 -- 172.21.15.31:0/1024944355 wait complete. 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 Processor -- start 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.410+0000 7f5b5391b640 1 -- start start 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b5391b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c102a60 0x7f5b4c104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b5391b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b4c1083a0 0x7f5b4c104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b5391b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b5391b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112960 con 0x7f5b4c108cf0 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b5391b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1127e0 con 0x7f5b4c1083a0 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b5391b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112ae0 con 0x7f5b4c102a60 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59190/0 (socket says 172.21.15.31:59190) 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 -- 172.21.15.31:0/3894953166 learned_addr learned my addr 172.21.15.31:0/3894953166 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b516b7640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c102a60 0x7f5b4c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b50eb6640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b4c1083a0 0x7f5b4c104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 -- 172.21.15.31:0/3894953166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c102a60 msgr2=0x7f5b4c104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c102a60 0x7f5b4c104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 -- 172.21.15.31:0/3894953166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b4c1083a0 msgr2=0x7f5b4c104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b4c1083a0 0x7f5b4c104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 -- 172.21.15.31:0/3894953166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c1a8630 con 0x7f5b4c108cf0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.411+0000 7f5b51eb8640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c104cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400eaa0 tx=0x7f5b4400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.412+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44004d20 con 0x7f5b4c108cf0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.412+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c1a8920 con 0x7f5b4c108cf0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.412+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b44005020 con 0x7f5b4c108cf0 2024-09-21T06:10:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.412+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440116b0 con 0x7f5b4c108cf0 2024-09-21T06:10:25.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.412+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1a8e10 con 0x7f5b4c108cf0 2024-09-21T06:10:25.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.413+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c102e70 con 0x7f5b4c108cf0 2024-09-21T06:10:25.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.414+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5b440051d0 con 0x7f5b4c108cf0 2024-09-21T06:10:25.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.416+0000 7f5b427fc640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b380b65f0 0x7f5b380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.416+0000 7f5b516b7640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b380b65f0 0x7f5b380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.416+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f5b440e7b70 con 0x7f5b4c108cf0 2024-09-21T06:10:25.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.416+0000 7f5b516b7640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b380b65f0 0x7f5b380b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b48000c00 tx=0x7f5b48031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:25.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.419+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5b44016030 con 0x7f5b4c108cf0 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3492490694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c105300 msgr2=0x7ff79c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3492490694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c105300 0x7ff79c10d850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3492490694 shutdown_connections 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3492490694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c10de10 0x7ff79c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3492490694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c105300 0x7ff79c10d850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3492490694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff79c1049b0 0x7ff79c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3492490694 >> 172.21.15.31:0/3492490694 conn(0x7ff79c079730 msgr2=0x7ff79c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:25.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3492490694 shutdown_connections 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.546+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3492490694 wait complete. 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 Processor -- start 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 -- start start 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 0x7ff79c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c105300 0x7ff79c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff79c10de10 0x7ff79c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c113a70 con 0x7ff79c1049b0 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c1138f0 con 0x7ff79c105300 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a3ca7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c113bf0 con 0x7ff79c10de10 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.547+0000 7ff7a1a43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 0x7ff79c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 0x7ff79c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59214/0 (socket says 172.21.15.31:59214) 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 -- 172.21.15.31:0/3370246382 learned_addr learned my addr 172.21.15.31:0/3370246382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:25.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a2244640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff79c10de10 0x7ff79c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1242640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c105300 0x7ff79c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 -- 172.21.15.31:0/3370246382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff79c10de10 msgr2=0x7ff79c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff79c10de10 0x7ff79c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 -- 172.21.15.31:0/3370246382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c105300 msgr2=0x7ff79c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c105300 0x7ff79c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 -- 172.21.15.31:0/3370246382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c1a0430 con 0x7ff79c1049b0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1242640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c105300 0x7ff79c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a1a43640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 0x7ff79c19a4c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff79800d770 tx=0x7ff79800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798017070 con 0x7ff79c1049b0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79c1a0720 con 0x7ff79c1049b0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff798009c30 con 0x7ff79c1049b0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798003cc0 con 0x7ff79c1049b0 2024-09-21T06:10:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.548+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79c1a6e80 con 0x7ff79c1049b0 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.549+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79c104dc0 con 0x7ff79c1049b0 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.551+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff798003e60 con 0x7ff79c1049b0 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.552+0000 7ff792ffd640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7880b6670 0x7ff7880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.552+0000 7ff7a1242640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7880b6670 0x7ff7880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.552+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff7980e73e0 con 0x7ff79c1049b0 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.552+0000 7ff7a1242640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7880b6670 0x7ff7880b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009230 tx=0x7ff78c000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.555+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff798009de0 con 0x7ff79c1049b0 2024-09-21T06:10:25.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.615+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5b4c1a9200 con 0x7f5b4c108cf0 2024-09-21T06:10:25.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.615+0000 7f5b427fc640 1 -- 172.21.15.31:0/3894953166 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5b440ae1c0 con 0x7f5b4c108cf0 2024-09-21T06:10:25.618 INFO:teuthology.orchestra.run.smithi031.stdout:227633266725 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b380b65f0 msgr2=0x7f5b380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b380b65f0 0x7f5b380b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b48000c00 tx=0x7f5b48031400 comp rx=0 tx=0).stop 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 msgr2=0x7f5b4c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c104cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400eaa0 tx=0x7f5b4400eea0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 shutdown_connections 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b380b65f0 0x7f5b380b8b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b4c108cf0 0x7f5b4c104cd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b4c1083a0 0x7f5b4c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 --2- 172.21.15.31:0/3894953166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b4c102a60 0x7f5b4c104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 >> 172.21.15.31:0/3894953166 conn(0x7f5b4c079730 msgr2=0x7f5b4c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 shutdown_connections 2024-09-21T06:10:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.618+0000 7f5b5391b640 1 -- 172.21.15.31:0/3894953166 wait complete. 2024-09-21T06:10:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.755+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff79c108980 con 0x7ff79c1049b0 2024-09-21T06:10:25.758 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:25 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3894953166' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:10:25.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.755+0000 7ff792ffd640 1 -- 172.21.15.31:0/3370246382 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff7980ad9d0 con 0x7ff79c1049b0 2024-09-21T06:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout:133143986233 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7880b6670 msgr2=0x7ff7880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7880b6670 0x7ff7880b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009230 tx=0x7ff78c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 msgr2=0x7ff79c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 0x7ff79c19a4c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff79800d770 tx=0x7ff79800dc30 comp rx=0 tx=0).stop 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 shutdown_connections 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff79c10de10 0x7ff79c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79c105300 0x7ff79c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7880b6670 0x7ff7880b8b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 --2- 172.21.15.31:0/3370246382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff79c1049b0 0x7ff79c19a4c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 >> 172.21.15.31:0/3370246382 conn(0x7ff79c079730 msgr2=0x7ff79c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 shutdown_connections 2024-09-21T06:10:25.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:25.757+0000 7ff7a3ca7640 1 -- 172.21.15.31:0/3370246382 wait complete. 2024-09-21T06:10:26.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:25 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3894953166' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:10:26.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:25 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3894953166' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:10:26.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266723 got 227633266725 for osd.8 2024-09-21T06:10:26.566 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:26.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 -- 172.21.15.31:0/792139088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a41083b0 msgr2=0x7f28a41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:26.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 --2- 172.21.15.31:0/792139088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a41083b0 0x7f28a41087c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2898009900 tx=0x7f289802a410 comp rx=0 tx=0).stop 2024-09-21T06:10:26.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 -- 172.21.15.31:0/792139088 shutdown_connections 2024-09-21T06:10:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 --2- 172.21.15.31:0/792139088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28a4108d00 0x7f28a410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 --2- 172.21.15.31:0/792139088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a41083b0 0x7f28a41087c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 --2- 172.21.15.31:0/792139088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4102a70 0x7f28a4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 -- 172.21.15.31:0/792139088 >> 172.21.15.31:0/792139088 conn(0x7f28a4079710 msgr2=0x7f28a4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 -- 172.21.15.31:0/792139088 shutdown_connections 2024-09-21T06:10:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.730+0000 7f28ab300640 1 -- 172.21.15.31:0/792139088 wait complete. 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 Processor -- start 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 -- start start 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 0x7f28a419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28a41083b0 0x7f28a419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4108d00 0x7f28a419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28a41129c0 con 0x7f28a4102a70 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4112840 con 0x7f28a4108d00 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28ab300640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4112b40 con 0x7f28a41083b0 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28a909c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 0x7f28a419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28a909c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 0x7f28a419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59232/0 (socket says 172.21.15.31:59232) 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.731+0000 7f28a909c640 1 -- 172.21.15.31:0/500460912 learned_addr learned my addr 172.21.15.31:0/500460912 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a889b640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28a41083b0 0x7f28a419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a909c640 1 -- 172.21.15.31:0/500460912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28a41083b0 msgr2=0x7f28a419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a909c640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28a41083b0 0x7f28a419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a909c640 1 -- 172.21.15.31:0/500460912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4108d00 msgr2=0x7f28a419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a989d640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4108d00 0x7f28a419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a909c640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4108d00 0x7f28a419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a909c640 1 -- 172.21.15.31:0/500460912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a41a0510 con 0x7f28a4102a70 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a989d640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4108d00 0x7f28a419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a909c640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 0x7f28a419a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f289400eab0 tx=0x7f289400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289400ce40 con 0x7f28a4102a70 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2894011920 con 0x7f28a4102a70 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894010a30 con 0x7f28a4102a70 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a41a0800 con 0x7f28a4102a70 2024-09-21T06:10:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.732+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a41a6f60 con 0x7f28a4102a70 2024-09-21T06:10:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.733+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a4103cf0 con 0x7f28a4102a70 2024-09-21T06:10:26.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.736+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2894010bd0 con 0x7f28a4102a70 2024-09-21T06:10:26.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.737+0000 7f28a27fc640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f288c0b6670 0x7f288c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.737+0000 7f28a889b640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f288c0b6670 0x7f288c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.737+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f28940e7fd0 con 0x7f28a4102a70 2024-09-21T06:10:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.738+0000 7f28a889b640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f288c0b6670 0x7f288c0b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2898009230 tx=0x7f2898031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:26.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.740+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2894016030 con 0x7f28a4102a70 2024-09-21T06:10:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:26 smithi031 ceph-mon[31237]: pgmap v285: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:26.851 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:26 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3370246382' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:10:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.932+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f28a4102e80 con 0x7f28a4102a70 2024-09-21T06:10:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.932+0000 7f28a27fc640 1 -- 172.21.15.31:0/500460912 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28940ae6a0 con 0x7f28a4102a70 2024-09-21T06:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout:60129542217 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.934+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f288c0b6670 msgr2=0x7f288c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.934+0000 7f28ab300640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f288c0b6670 0x7f288c0b8b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2898009230 tx=0x7f2898031400 comp rx=0 tx=0).stop 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 msgr2=0x7f28a419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 0x7f28a419a5a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f289400eab0 tx=0x7f289400ef70 comp rx=0 tx=0).stop 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 shutdown_connections 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a4108d00 0x7f28a419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28a41083b0 0x7f28a419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f288c0b6670 0x7f288c0b8b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 --2- 172.21.15.31:0/500460912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28a4102a70 0x7f28a419a5a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:26.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 >> 172.21.15.31:0/500460912 conn(0x7f28a4079710 msgr2=0x7f28a4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 shutdown_connections 2024-09-21T06:10:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:26.935+0000 7f28ab300640 1 -- 172.21.15.31:0/500460912 wait complete. 2024-09-21T06:10:27.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:26 smithi110 ceph-mon[34550]: pgmap v285: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:27.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:26 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3370246382' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:10:27.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:26 smithi107 ceph-mon[34380]: pgmap v285: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:27.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:26 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3370246382' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:10:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 -- 172.21.15.31:0/2960152992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104920 msgr2=0x7efc7c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 --2- 172.21.15.31:0/2960152992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104920 0x7efc7c104d30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009ca0 tx=0x7efc6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 -- 172.21.15.31:0/2960152992 shutdown_connections 2024-09-21T06:10:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 --2- 172.21.15.31:0/2960152992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc7c10ded0 0x7efc7c110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 --2- 172.21.15.31:0/2960152992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c105350 0x7efc7c10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 --2- 172.21.15.31:0/2960152992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c104920 0x7efc7c104d30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 -- 172.21.15.31:0/2960152992 >> 172.21.15.31:0/2960152992 conn(0x7efc7c079770 msgr2=0x7efc7c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 -- 172.21.15.31:0/2960152992 shutdown_connections 2024-09-21T06:10:27.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.417+0000 7efc80fad640 1 -- 172.21.15.31:0/2960152992 wait complete. 2024-09-21T06:10:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 Processor -- start 2024-09-21T06:10:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 -- start start 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c104920 0x7efc7c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc7c105350 0x7efc7c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 0x7efc7c1a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c113bc0 con 0x7efc7c10ded0 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c113a40 con 0x7efc7c105350 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc80fad640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c113d40 con 0x7efc7c104920 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc7ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 0x7efc7c1a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc7ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 0x7efc7c1a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59238/0 (socket says 172.21.15.31:59238) 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc7ad9d640 1 -- 172.21.15.31:0/3442564805 learned_addr learned my addr 172.21.15.31:0/3442564805 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc79d9b640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc7c105350 0x7efc7c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.418+0000 7efc7a59c640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c104920 0x7efc7c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:27.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7ad9d640 1 -- 172.21.15.31:0/3442564805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c104920 msgr2=0x7efc7c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7ad9d640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c104920 0x7efc7c19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7ad9d640 1 -- 172.21.15.31:0/3442564805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc7c105350 msgr2=0x7efc7c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7ad9d640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc7c105350 0x7efc7c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7ad9d640 1 -- 172.21.15.31:0/3442564805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c1a2740 con 0x7efc7c10ded0 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7a59c640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c104920 0x7efc7c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc7ad9d640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 0x7efc7c1a2010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc74009fd0 tx=0x7efc7400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74015020 con 0x7efc7c10ded0 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc74011ce0 con 0x7efc7c10ded0 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c1a2a30 con 0x7efc7c10ded0 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74010dc0 con 0x7efc7c10ded0 2024-09-21T06:10:27.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.419+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c1a9190 con 0x7efc7c10ded0 2024-09-21T06:10:27.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.420+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c06c1a0 con 0x7efc7c10ded0 2024-09-21T06:10:27.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.422+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7efc74018020 con 0x7efc7c10ded0 2024-09-21T06:10:27.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.422+0000 7efc6b7fe640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc640b65f0 0x7efc640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.423+0000 7efc7a59c640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc640b65f0 0x7efc640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.423+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efc740e6d80 con 0x7efc7c10ded0 2024-09-21T06:10:27.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.423+0000 7efc7a59c640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc640b65f0 0x7efc640b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc6c000f80 tx=0x7efc6c02f3a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:27.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.427+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efc7401f020 con 0x7efc7c10ded0 2024-09-21T06:10:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.617+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efc7c19d900 con 0x7efc7c10ded0 2024-09-21T06:10:27.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.617+0000 7efc6b7fe640 1 -- 172.21.15.31:0/3442564805 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc740ad3d0 con 0x7efc7c10ded0 2024-09-21T06:10:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:154618822708 2024-09-21T06:10:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.619+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc640b65f0 msgr2=0x7efc640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.619+0000 7efc80fad640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc640b65f0 0x7efc640b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc6c000f80 tx=0x7efc6c02f3a0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.619+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 msgr2=0x7efc7c1a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.619+0000 7efc80fad640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 0x7efc7c1a2010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc74009fd0 tx=0x7efc7400b820 comp rx=0 tx=0).stop 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 shutdown_connections 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc640b65f0 0x7efc640b8b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc7c10ded0 0x7efc7c1a2010 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc7c105350 0x7efc7c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 --2- 172.21.15.31:0/3442564805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc7c104920 0x7efc7c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 >> 172.21.15.31:0/3442564805 conn(0x7efc7c079770 msgr2=0x7efc7c109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 shutdown_connections 2024-09-21T06:10:27.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:27.620+0000 7efc80fad640 1 -- 172.21.15.31:0/3442564805 wait complete. 2024-09-21T06:10:27.805 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:27.805 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:27.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:27 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/500460912' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:10:27.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:27 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3442564805' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:10:28.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:27 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/500460912' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:10:28.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:27 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3442564805' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:10:28.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:27 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/500460912' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:10:28.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:27 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3442564805' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:10:28.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986231 got 133143986233 for osd.4 2024-09-21T06:10:28.582 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:28 smithi031 ceph-mon[31237]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:29.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:28 smithi110 ceph-mon[34550]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:28 smithi107 ceph-mon[34380]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:29.243 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.434+0000 7f200c93d640 1 -- 172.21.15.31:0/3370177329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20081094a0 msgr2=0x7f20081098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.434+0000 7f200c93d640 1 --2- 172.21.15.31:0/3370177329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20081094a0 0x7f20081098b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009900 tx=0x7f1ffc02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 -- 172.21.15.31:0/3370177329 shutdown_connections 2024-09-21T06:10:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 --2- 172.21.15.31:0/3370177329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2008109df0 0x7f2008110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 --2- 172.21.15.31:0/3370177329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20081094a0 0x7f20081098b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 --2- 172.21.15.31:0/3370177329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2008103b60 0x7f2008103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 -- 172.21.15.31:0/3370177329 >> 172.21.15.31:0/3370177329 conn(0x7f200807b850 msgr2=0x7f200807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 -- 172.21.15.31:0/3370177329 shutdown_connections 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.435+0000 7f200c93d640 1 -- 172.21.15.31:0/3370177329 wait complete. 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 Processor -- start 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 -- start start 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 0x7f200819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20081094a0 0x7f200819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2008109df0 0x7f200819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2008113ad0 con 0x7f2008103b60 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2008113950 con 0x7f20081094a0 2024-09-21T06:10:29.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200c93d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2008113c50 con 0x7f2008109df0 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 0x7f200819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 0x7f200819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52914/0 (socket says 172.21.15.31:52914) 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 -- 172.21.15.31:0/3244891193 learned_addr learned my addr 172.21.15.31:0/3244891193 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f2005d9b640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20081094a0 0x7f200819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f2006d9d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2008109df0 0x7f200819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 -- 172.21.15.31:0/3244891193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2008109df0 msgr2=0x7f200819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2008109df0 0x7f200819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 -- 172.21.15.31:0/3244891193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20081094a0 msgr2=0x7f200819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20081094a0 0x7f200819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.436+0000 7f200659c640 1 -- 172.21.15.31:0/3244891193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20081a0370 con 0x7f2008103b60 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.437+0000 7f200659c640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 0x7f200819a4c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c920 tx=0x7f1ff800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:29.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.437+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8007c70 con 0x7f2008103b60 2024-09-21T06:10:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.437+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20081a0660 con 0x7f2008103b60 2024-09-21T06:10:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.437+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff8007e10 con 0x7f2008103b60 2024-09-21T06:10:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.437+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8018820 con 0x7f2008103b60 2024-09-21T06:10:29.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.437+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081a6e80 con 0x7f2008103b60 2024-09-21T06:10:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.438+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2008103f70 con 0x7f2008103b60 2024-09-21T06:10:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.439+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f1ff800f450 con 0x7f2008103b60 2024-09-21T06:10:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.440+0000 7f1ff77fe640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ff00b6620 0x7f1ff00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.440+0000 7f2005d9b640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ff00b6620 0x7f1ff00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.440+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1ff80e7aa0 con 0x7f2008103b60 2024-09-21T06:10:29.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.441+0000 7f2005d9b640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ff00b6620 0x7f1ff00b8b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009230 tx=0x7f1ffc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.445+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ff8014030 con 0x7f2008103b60 2024-09-21T06:10:29.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.668+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f200810bc40 con 0x7f2008103b60 2024-09-21T06:10:29.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.669+0000 7f1ff77fe640 1 -- 172.21.15.31:0/3244891193 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ff80ae120 con 0x7f2008103b60 2024-09-21T06:10:29.671 INFO:teuthology.orchestra.run.smithi031.stdout:249108103201 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ff00b6620 msgr2=0x7f1ff00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ff00b6620 0x7f1ff00b8b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009230 tx=0x7f1ffc031400 comp rx=0 tx=0).stop 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 msgr2=0x7f200819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 0x7f200819a4c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c920 tx=0x7f1ff800cde0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 shutdown_connections 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2008109df0 0x7f200819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20081094a0 0x7f200819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ff00b6620 0x7f1ff00b8b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 --2- 172.21.15.31:0/3244891193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2008103b60 0x7f200819a4c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 >> 172.21.15.31:0/3244891193 conn(0x7f200807b850 msgr2=0x7f2008107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 shutdown_connections 2024-09-21T06:10:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:29.671+0000 7f200c93d640 1 -- 172.21.15.31:0/3244891193 wait complete. 2024-09-21T06:10:29.852 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:29 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3244891193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:10:29.879 INFO:teuthology.orchestra.run.smithi031.stdout:81604378689 2024-09-21T06:10:29.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T06:10:30.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:29 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3244891193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:10:30.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:29 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3244891193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:10:30.455 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103201 for osd.9 2024-09-21T06:10:30.456 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 -- 172.21.15.31:0/3414252072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 msgr2=0x7fc93810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 --2- 172.21.15.31:0/3414252072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc93810f530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc930009ca0 tx=0x7fc93002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 -- 172.21.15.31:0/3414252072 shutdown_connections 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 --2- 172.21.15.31:0/3414252072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc93810f530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 --2- 172.21.15.31:0/3414252072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 0x7fc9381087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 --2- 172.21.15.31:0/3414252072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 0x7fc938102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 -- 172.21.15.31:0/3414252072 >> 172.21.15.31:0/3414252072 conn(0x7fc938079730 msgr2=0x7fc938079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 -- 172.21.15.31:0/3414252072 shutdown_connections 2024-09-21T06:10:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.662+0000 7fc93e451640 1 -- 172.21.15.31:0/3414252072 wait complete. 2024-09-21T06:10:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.663+0000 7fc93e451640 1 Processor -- start 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.663+0000 7fc93e451640 1 -- start start 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.663+0000 7fc93e451640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 0x7fc93810fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.663+0000 7fc93e451640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 0x7fc938110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.663+0000 7fc93e451640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc938110840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93e451640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113a80 con 0x7fc938108d00 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93e451640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113900 con 0x7fc938102a70 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93e451640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc938113c00 con 0x7fc9381083b0 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93c9ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc938110840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc937fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 0x7fc93810fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 0x7fc938110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93c9ee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc938110840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52942/0 (socket says 172.21.15.31:52942) 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc937fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 0x7fc93810fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48854/0 (socket says 172.21.15.31:48854) 2024-09-21T06:10:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93c9ee640 1 -- 172.21.15.31:0/3547221701 learned_addr learned my addr 172.21.15.31:0/3547221701 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 -- 172.21.15.31:0/3547221701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 msgr2=0x7fc93810fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 0x7fc93810fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 -- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 msgr2=0x7fc938110840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc938110840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 -- 172.21.15.31:0/3547221701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9381a8710 con 0x7fc9381083b0 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc93c9ee640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc938110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.664+0000 7fc92ffff640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 0x7fc938110300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c940 tx=0x7fc92000ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.665+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920007bb0 con 0x7fc9381083b0 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.665+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9381a8a00 con 0x7fc9381083b0 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.665+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc920007d50 con 0x7fc9381083b0 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.665+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920004db0 con 0x7fc9381083b0 2024-09-21T06:10:30.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.665+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9381a8ef0 con 0x7fc9381083b0 2024-09-21T06:10:30.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.666+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc938102e80 con 0x7fc9381083b0 2024-09-21T06:10:30.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.669+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc920005000 con 0x7fc9381083b0 2024-09-21T06:10:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.670+0000 7fc935ffb640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9240b65f0 0x7fc9240b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.670+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc9200e75f0 con 0x7fc9381083b0 2024-09-21T06:10:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.670+0000 7fc937fff640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9240b65f0 0x7fc9240b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.671+0000 7fc937fff640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9240b65f0 0x7fc9240b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9280058c0 tx=0x7fc92800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:30.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.672+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc920014030 con 0x7fc9381083b0 2024-09-21T06:10:30.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.866+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc9381110c0 con 0x7fc9381083b0 2024-09-21T06:10:30.868 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:30 smithi031 ceph-mon[31237]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s 2024-09-21T06:10:30.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.866+0000 7fc935ffb640 1 -- 172.21.15.31:0/3547221701 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc9200b2020 con 0x7fc9381083b0 2024-09-21T06:10:30.868 INFO:teuthology.orchestra.run.smithi031.stdout:111669149758 2024-09-21T06:10:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9240b65f0 msgr2=0x7fc9240b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9240b65f0 0x7fc9240b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc9280058c0 tx=0x7fc92800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 msgr2=0x7fc938110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 0x7fc938110300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc92000c940 tx=0x7fc92000ce00 comp rx=0 tx=0).stop 2024-09-21T06:10:30.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 shutdown_connections 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9240b65f0 0x7fc9240b8b00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc938108d00 0x7fc938110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9381083b0 0x7fc938110300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 --2- 172.21.15.31:0/3547221701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc938102a70 0x7fc93810fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 >> 172.21.15.31:0/3547221701 conn(0x7fc938079730 msgr2=0x7fc9381043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 shutdown_connections 2024-09-21T06:10:30.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:30.869+0000 7fc93e451640 1 -- 172.21.15.31:0/3547221701 wait complete. 2024-09-21T06:10:31.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:30 smithi110 ceph-mon[34550]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s 2024-09-21T06:10:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:30 smithi107 ceph-mon[34380]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1 op/s 2024-09-21T06:10:31.535 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542215 got 60129542217 for osd.1 2024-09-21T06:10:31.535 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:31.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:31 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3547221701' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:10:31.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149755 got 111669149758 for osd.3 2024-09-21T06:10:31.936 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:32.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:31 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3547221701' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:10:32.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:31 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3547221701' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:10:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 -- 172.21.15.31:0/1553930808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c1083a0 msgr2=0x7fe07c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 --2- 172.21.15.31:0/1553930808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c1083a0 0x7fe07c1087b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe070009900 tx=0x7fe07002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 -- 172.21.15.31:0/1553930808 shutdown_connections 2024-09-21T06:10:32.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 --2- 172.21.15.31:0/1553930808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe07c108cf0 0x7fe07c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 --2- 172.21.15.31:0/1553930808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c1083a0 0x7fe07c1087b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 --2- 172.21.15.31:0/1553930808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c102a60 0x7fe07c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 -- 172.21.15.31:0/1553930808 >> 172.21.15.31:0/1553930808 conn(0x7fe07c079730 msgr2=0x7fe07c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 -- 172.21.15.31:0/1553930808 shutdown_connections 2024-09-21T06:10:32.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.177+0000 7fe082979640 1 -- 172.21.15.31:0/1553930808 wait complete. 2024-09-21T06:10:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.178+0000 7fe082979640 1 Processor -- start 2024-09-21T06:10:32.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.178+0000 7fe082979640 1 -- start start 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.178+0000 7fe082979640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe07c102a60 0x7fe07c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe082979640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c1083a0 0x7fe07c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe082979640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 0x7fe07c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe082979640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c112960 con 0x7fe07c108cf0 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe082979640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c1127e0 con 0x7fe07c1083a0 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe082979640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c112ae0 con 0x7fe07c102a60 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 0x7fe07c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe07bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe07c102a60 0x7fe07c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 0x7fe07c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52954/0 (socket says 172.21.15.31:52954) 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 -- 172.21.15.31:0/149468534 learned_addr learned my addr 172.21.15.31:0/149468534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe07b7fe640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c1083a0 0x7fe07c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 -- 172.21.15.31:0/149468534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe07c102a60 msgr2=0x7fe07c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe07c102a60 0x7fe07c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 -- 172.21.15.31:0/149468534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c1083a0 msgr2=0x7fe07c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c1083a0 0x7fe07c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 -- 172.21.15.31:0/149468534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c1a0410 con 0x7fe07c108cf0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe080f16640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 0x7fe07c19fce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe074009fd0 tx=0x7fe07400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.179+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074017070 con 0x7fe07c108cf0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.180+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe074003e20 con 0x7fe07c108cf0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.180+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c1a0700 con 0x7fe07c108cf0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.180+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074010460 con 0x7fe07c108cf0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.180+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c1a6e60 con 0x7fe07c108cf0 2024-09-21T06:10:32.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.181+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c102e70 con 0x7fe07c108cf0 2024-09-21T06:10:32.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.182+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe074003970 con 0x7fe07c108cf0 2024-09-21T06:10:32.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.183+0000 7fe0797fa640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0640b65f0 0x7fe0640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:32.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.183+0000 7fe07bfff640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0640b65f0 0x7fe0640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:32.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.183+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe0740e72e0 con 0x7fe07c108cf0 2024-09-21T06:10:32.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.184+0000 7fe07bfff640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0640b65f0 0x7fe0640b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0058c0 tx=0x7fe06c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:32.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.186+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0740b1d10 con 0x7fe07c108cf0 2024-09-21T06:10:32.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.382+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe07c10bad0 con 0x7fe07c108cf0 2024-09-21T06:10:32.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.382+0000 7fe0797fa640 1 -- 172.21.15.31:0/149468534 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe0740b6c50 con 0x7fe07c108cf0 2024-09-21T06:10:32.384 INFO:teuthology.orchestra.run.smithi031.stdout:296352743447 2024-09-21T06:10:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0640b65f0 msgr2=0x7fe0640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0640b65f0 0x7fe0640b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0058c0 tx=0x7fe06c00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 msgr2=0x7fe07c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:32.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 0x7fe07c19fce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe074009fd0 tx=0x7fe07400b260 comp rx=0 tx=0).stop 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 shutdown_connections 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0640b65f0 0x7fe0640b8b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe07c108cf0 0x7fe07c19fce0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c1083a0 0x7fe07c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 --2- 172.21.15.31:0/149468534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe07c102a60 0x7fe07c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 >> 172.21.15.31:0/149468534 conn(0x7fe07c079730 msgr2=0x7fe07c06b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.384+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 shutdown_connections 2024-09-21T06:10:32.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:32.385+0000 7fe082979640 1 -- 172.21.15.31:0/149468534 wait complete. 2024-09-21T06:10:33.016 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:32 smithi110 ceph-mon[34550]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-21T06:10:33.016 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:32 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:33.016 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:32 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/149468534' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:10:33.029 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:32 smithi031 ceph-mon[31237]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-21T06:10:33.029 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:32 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:33.030 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:32 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/149468534' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:10:33.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:32 smithi107 ceph-mon[34380]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-21T06:10:33.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:32 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:10:33.032 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:32 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/149468534' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:10:33.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822705 got 154618822708 for osd.5 2024-09-21T06:10:33.060 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 -- 172.21.15.31:0/1880238147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6524109e40 msgr2=0x7f6524110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 --2- 172.21.15.31:0/1880238147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6524109e40 0x7f6524110670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f651c009ca0 tx=0x7f651c02a8e0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 -- 172.21.15.31:0/1880238147 shutdown_connections 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 --2- 172.21.15.31:0/1880238147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6524109e40 0x7f6524110670 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 --2- 172.21.15.31:0/1880238147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65241094f0 0x7f6524109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 --2- 172.21.15.31:0/1880238147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6524103bb0 0x7f6524103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 -- 172.21.15.31:0/1880238147 >> 172.21.15.31:0/1880238147 conn(0x7f65240ff600 msgr2=0x7f6524101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.252+0000 7f652b605640 1 -- 172.21.15.31:0/1880238147 shutdown_connections 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.253+0000 7f652b605640 1 -- 172.21.15.31:0/1880238147 wait complete. 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.253+0000 7f652b605640 1 Processor -- start 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.253+0000 7f652b605640 1 -- start start 2024-09-21T06:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.253+0000 7f652b605640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524103bb0 0x7f652419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.253+0000 7f652b605640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 0x7f652419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.253+0000 7f652b605640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6524109e40 0x7f652419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f652b605640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6524113bc0 con 0x7f65241094f0 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f652b605640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6524113a40 con 0x7f6524109e40 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f652b605640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6524113d40 con 0x7f6524103bb0 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 0x7f652419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 0x7f652419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52972/0 (socket says 172.21.15.31:52972) 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f65293a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524103bb0 0x7f652419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 -- 172.21.15.31:0/1241438807 learned_addr learned my addr 172.21.15.31:0/1241438807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6529ba2640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6524109e40 0x7f652419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 -- 172.21.15.31:0/1241438807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524103bb0 msgr2=0x7f652419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524103bb0 0x7f652419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 -- 172.21.15.31:0/1241438807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6524109e40 msgr2=0x7f652419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6524109e40 0x7f652419fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 -- 172.21.15.31:0/1241438807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65241a05a0 con 0x7f65241094f0 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f6528ba0640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 0x7f652419abd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6514006fd0 tx=0x7f651400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514015020 con 0x7f65241094f0 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.254+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65241a0890 con 0x7f65241094f0 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.255+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6514011ce0 con 0x7f65241094f0 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.255+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65241a6ff0 con 0x7f65241094f0 2024-09-21T06:10:33.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.255+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514010d90 con 0x7f65241094f0 2024-09-21T06:10:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.256+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65080052c0 con 0x7f65241094f0 2024-09-21T06:10:33.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.258+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6514018020 con 0x7f65241094f0 2024-09-21T06:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.259+0000 7f651a7fc640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f65100b65f0 0x7f65100b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.259+0000 7f65293a1640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f65100b65f0 0x7f65100b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.259+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f65140e70d0 con 0x7f65241094f0 2024-09-21T06:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.259+0000 7f65293a1640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f65100b65f0 0x7f65100b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6520009990 tx=0x7f6520009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:33.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.262+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f651401f020 con 0x7f65241094f0 2024-09-21T06:10:33.295 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:33.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.467+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6508005880 con 0x7f65241094f0 2024-09-21T06:10:33.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.468+0000 7f651a7fc640 1 -- 172.21.15.31:0/1241438807 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65140b4e20 con 0x7f65241094f0 2024-09-21T06:10:33.470 INFO:teuthology.orchestra.run.smithi031.stdout:38654705743 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f65100b65f0 msgr2=0x7f65100b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f65100b65f0 0x7f65100b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6520009990 tx=0x7f6520009110 comp rx=0 tx=0).stop 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 msgr2=0x7f652419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 0x7f652419abd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6514006fd0 tx=0x7f651400b820 comp rx=0 tx=0).stop 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 shutdown_connections 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6524109e40 0x7f652419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f65100b65f0 0x7f65100b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65241094f0 0x7f652419abd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 --2- 172.21.15.31:0/1241438807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6524103bb0 0x7f652419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 >> 172.21.15.31:0/1241438807 conn(0x7f65240ff600 msgr2=0x7f6524101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:33.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 shutdown_connections 2024-09-21T06:10:33.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:33.470+0000 7f652b605640 1 -- 172.21.15.31:0/1241438807 wait complete. 2024-09-21T06:10:33.770 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743447 for osd.11 2024-09-21T06:10:33.771 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:33.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:33 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1241438807' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:10:33.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:33.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:33 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:33 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1241438807' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:10:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:33 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:33 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1241438807' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:10:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:33 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 -- 172.21.15.31:0/804486574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20108d70 msgr2=0x7fdd2010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 --2- 172.21.15.31:0/804486574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20108d70 0x7fdd2010f5a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009ca0 tx=0x7fdd1802a8e0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 -- 172.21.15.31:0/804486574 shutdown_connections 2024-09-21T06:10:34.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 --2- 172.21.15.31:0/804486574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20108d70 0x7fdd2010f5a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 --2- 172.21.15.31:0/804486574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd20108420 0x7fdd20108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 --2- 172.21.15.31:0/804486574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd20102ae0 0x7fdd20102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 -- 172.21.15.31:0/804486574 >> 172.21.15.31:0/804486574 conn(0x7fdd20079770 msgr2=0x7fdd20079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:34.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 -- 172.21.15.31:0/804486574 shutdown_connections 2024-09-21T06:10:34.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.184+0000 7fdd253a6640 1 -- 172.21.15.31:0/804486574 wait complete. 2024-09-21T06:10:34.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 Processor -- start 2024-09-21T06:10:34.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 -- start start 2024-09-21T06:10:34.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 0x7fdd2019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd20108420 0x7fdd2019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd20108d70 0x7fdd2019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20112a70 con 0x7fdd20102ae0 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201128f0 con 0x7fdd20108420 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd253a6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20112bf0 con 0x7fdd20108d70 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.185+0000 7fdd1effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 0x7fdd2019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 0x7fdd2019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52984/0 (socket says 172.21.15.31:52984) 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 -- 172.21.15.31:0/1464653592 learned_addr learned my addr 172.21.15.31:0/1464653592 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1f7fe640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd20108d70 0x7fdd2019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1e7fc640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd20108420 0x7fdd2019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 -- 172.21.15.31:0/1464653592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd20108d70 msgr2=0x7fdd2019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd20108d70 0x7fdd2019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 -- 172.21.15.31:0/1464653592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd20108420 msgr2=0x7fdd2019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd20108420 0x7fdd2019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 -- 172.21.15.31:0/1464653592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd201a0520 con 0x7fdd20102ae0 2024-09-21T06:10:34.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd1effd640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 0x7fdd2019a5b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000d950 tx=0x7fdd1000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:34.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10017070 con 0x7fdd20102ae0 2024-09-21T06:10:34.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd201a0810 con 0x7fdd20102ae0 2024-09-21T06:10:34.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1000de80 con 0x7fdd20102ae0 2024-09-21T06:10:34.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.186+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1000f620 con 0x7fdd20102ae0 2024-09-21T06:10:34.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.187+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201a6f70 con 0x7fdd20102ae0 2024-09-21T06:10:34.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.187+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd20103d60 con 0x7fdd20102ae0 2024-09-21T06:10:34.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.189+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fdd10009c30 con 0x7fdd20102ae0 2024-09-21T06:10:34.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.190+0000 7fdd07fff640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd080b65f0 0x7fdd080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:34.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.190+0000 7fdd1e7fc640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd080b65f0 0x7fdd080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:34.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.190+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdd100e7100 con 0x7fdd20102ae0 2024-09-21T06:10:34.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.191+0000 7fdd1e7fc640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd080b65f0 0x7fdd080b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd20105370 tx=0x7fdd14009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:34.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.193+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd100ad750 con 0x7fdd20102ae0 2024-09-21T06:10:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.385+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdd20102ef0 con 0x7fdd20102ae0 2024-09-21T06:10:34.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.385+0000 7fdd07fff640 1 -- 172.21.15.31:0/1464653592 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdd100b1b30 con 0x7fdd20102ae0 2024-09-21T06:10:34.387 INFO:teuthology.orchestra.run.smithi031.stdout:201863462955 2024-09-21T06:10:34.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.387+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd080b65f0 msgr2=0x7fdd080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.387+0000 7fdd253a6640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd080b65f0 0x7fdd080b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd20105370 tx=0x7fdd14009110 comp rx=0 tx=0).stop 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 msgr2=0x7fdd2019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 0x7fdd2019a5b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000d950 tx=0x7fdd1000dd50 comp rx=0 tx=0).stop 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 shutdown_connections 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd20108d70 0x7fdd2019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd20108420 0x7fdd2019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd080b65f0 0x7fdd080b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 --2- 172.21.15.31:0/1464653592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd20102ae0 0x7fdd2019a5b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 >> 172.21.15.31:0/1464653592 conn(0x7fdd20079770 msgr2=0x7fdd20101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 shutdown_connections 2024-09-21T06:10:34.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:34.388+0000 7fdd253a6640 1 -- 172.21.15.31:0/1464653592 wait complete. 2024-09-21T06:10:34.417 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:34.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:34 smithi031 ceph-mon[31237]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:34.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:34 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:34.992 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:34 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1464653592' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:10:35.025 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462952 got 201863462955 for osd.7 2024-09-21T06:10:35.025 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:34 smithi110 ceph-mon[34550]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:34 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:34 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1464653592' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:10:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:34 smithi107 ceph-mon[34380]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:34 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:34 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1464653592' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 -- 172.21.15.31:0/3767357724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 msgr2=0x7f44ec102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 --2- 172.21.15.31:0/3767357724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec102ef0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009900 tx=0x7f44dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 -- 172.21.15.31:0/3767357724 shutdown_connections 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 --2- 172.21.15.31:0/3767357724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44ec108d20 0x7f44ec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 --2- 172.21.15.31:0/3767357724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44ec1083d0 0x7f44ec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 --2- 172.21.15.31:0/3767357724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec102ef0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 -- 172.21.15.31:0/3767357724 >> 172.21.15.31:0/3767357724 conn(0x7f44ec079760 msgr2=0x7f44ec079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:35.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 -- 172.21.15.31:0/3767357724 shutdown_connections 2024-09-21T06:10:35.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.756+0000 7f44f1332640 1 -- 172.21.15.31:0/3767357724 wait complete. 2024-09-21T06:10:35.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 Processor -- start 2024-09-21T06:10:35.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 -- start start 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44ec1083d0 0x7f44ec199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44ec108d20 0x7f44ec19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec112990 con 0x7f44ec102ae0 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec112810 con 0x7f44ec108d20 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44f1332640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec112b10 con 0x7f44ec1083d0 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53004/0 (socket says 172.21.15.31:53004) 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 -- 172.21.15.31:0/3291850275 learned_addr learned my addr 172.21.15.31:0/3291850275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 -- 172.21.15.31:0/3291850275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44ec1083d0 msgr2=0x7f44ec199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44e3fff640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44ec1083d0 0x7f44ec199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44ec1083d0 0x7f44ec199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 -- 172.21.15.31:0/3291850275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44ec108d20 msgr2=0x7f44ec19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eb7fe640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44ec108d20 0x7f44ec19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.757+0000 7f44eaffd640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44ec108d20 0x7f44ec19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44eaffd640 1 -- 172.21.15.31:0/3291850275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec19f2f0 con 0x7f44ec102ae0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44eaffd640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec199380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005940 tx=0x7f44dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc03f030 con 0x7f44ec102ae0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec19f580 con 0x7f44ec102ae0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44dc003cb0 con 0x7f44ec102ae0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc035070 con 0x7f44ec102ae0 2024-09-21T06:10:35.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.758+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec1a5d40 con 0x7f44ec102ae0 2024-09-21T06:10:35.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.759+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec103d10 con 0x7f44ec102ae0 2024-09-21T06:10:35.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.761+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f44dc042030 con 0x7f44ec102ae0 2024-09-21T06:10:35.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.762+0000 7f44e8ff9640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f44d80b65f0 0x7f44d80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:35.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.762+0000 7f44e3fff640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f44d80b65f0 0x7f44d80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:35.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.762+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f44dc108200 con 0x7f44ec102ae0 2024-09-21T06:10:35.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.762+0000 7f44e3fff640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f44d80b65f0 0x7f44d80b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44ec100b70 tx=0x7f44d4005c60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:35.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.765+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f44dc0ce7f0 con 0x7f44ec102ae0 2024-09-21T06:10:35.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.957+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f44ec19a4b0 con 0x7f44ec102ae0 2024-09-21T06:10:35.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.957+0000 7f44e8ff9640 1 -- 172.21.15.31:0/3291850275 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f44ec19a4b0 con 0x7f44ec102ae0 2024-09-21T06:10:35.959 INFO:teuthology.orchestra.run.smithi031.stdout:180388626481 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f44d80b65f0 msgr2=0x7f44d80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f44d80b65f0 0x7f44d80b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44ec100b70 tx=0x7f44d4005c60 comp rx=0 tx=0).stop 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 msgr2=0x7f44ec199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec199380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44dc005940 tx=0x7f44dc0042f0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 shutdown_connections 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44ec108d20 0x7f44ec19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44ec1083d0 0x7f44ec199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f44d80b65f0 0x7f44d80b8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 --2- 172.21.15.31:0/3291850275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44ec102ae0 0x7f44ec199380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 >> 172.21.15.31:0/3291850275 conn(0x7f44ec079760 msgr2=0x7f44ec101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 shutdown_connections 2024-09-21T06:10:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:35.960+0000 7f44f1332640 1 -- 172.21.15.31:0/3291850275 wait complete. 2024-09-21T06:10:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 -- 172.21.15.31:0/3967692422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720104a30 msgr2=0x7f4720104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 --2- 172.21.15.31:0/3967692422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720104a30 0x7f4720104e40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4718009ca0 tx=0x7f471802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 -- 172.21.15.31:0/3967692422 shutdown_connections 2024-09-21T06:10:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 --2- 172.21.15.31:0/3967692422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472010de60 0x7f4720110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 --2- 172.21.15.31:0/3967692422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4720105380 0x7f472010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 --2- 172.21.15.31:0/3967692422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720104a30 0x7f4720104e40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 -- 172.21.15.31:0/3967692422 >> 172.21.15.31:0/3967692422 conn(0x7f47200797b0 msgr2=0x7f4720079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 -- 172.21.15.31:0/3967692422 shutdown_connections 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.007+0000 7f4726b01640 1 -- 172.21.15.31:0/3967692422 wait complete. 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 Processor -- start 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 -- start start 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720104a30 0x7f472019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 0x7f472019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472010de60 0x7f472019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4720113aa0 con 0x7f4720105380 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4720113920 con 0x7f4720104a30 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f4726b01640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4720113c20 con 0x7f472010de60 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f471ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 0x7f472019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:36.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f471ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 0x7f472019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53020/0 (socket says 172.21.15.31:53020) 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.008+0000 7f471ffff640 1 -- 172.21.15.31:0/1876214499 learned_addr learned my addr 172.21.15.31:0/1876214499 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f472509e640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472010de60 0x7f472019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f472489d640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720104a30 0x7f472019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471ffff640 1 -- 172.21.15.31:0/1876214499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472010de60 msgr2=0x7f472019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471ffff640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472010de60 0x7f472019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471ffff640 1 -- 172.21.15.31:0/1876214499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720104a30 msgr2=0x7f472019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471ffff640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720104a30 0x7f472019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471ffff640 1 -- 172.21.15.31:0/1876214499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47201a0420 con 0x7f4720105380 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471ffff640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 0x7f472019aa50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f471400d950 tx=0x7f471400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714017070 con 0x7f4720105380 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4714009c30 con 0x7f4720105380 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47201a0710 con 0x7f4720105380 2024-09-21T06:10:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140047a0 con 0x7f4720105380 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.009+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47201a6e70 con 0x7f4720105380 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.010+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4720104e40 con 0x7f4720105380 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.012+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4714004020 con 0x7f4720105380 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.013+0000 7f471dffb640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f47080b6670 0x7f47080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.013+0000 7f472489d640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f47080b6670 0x7f47080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.013+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f47140e75a0 con 0x7f4720105380 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.014+0000 7f472489d640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f47080b6670 0x7f47080b8b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4718005e30 tx=0x7f4718009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.016+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4714020020 con 0x7f4720105380 2024-09-21T06:10:36.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.206+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f472019b5e0 con 0x7f4720105380 2024-09-21T06:10:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.206+0000 7f471dffb640 1 -- 172.21.15.31:0/1876214499 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f47140adc70 con 0x7f4720105380 2024-09-21T06:10:36.208 INFO:teuthology.orchestra.run.smithi031.stdout:274877906974 2024-09-21T06:10:36.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.208+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f47080b6670 msgr2=0x7f47080b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.208+0000 7f4726b01640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f47080b6670 0x7f47080b8b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4718005e30 tx=0x7f4718009360 comp rx=0 tx=0).stop 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.208+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 msgr2=0x7f472019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.208+0000 7f4726b01640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 0x7f472019aa50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f471400d950 tx=0x7f471400de10 comp rx=0 tx=0).stop 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 shutdown_connections 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f472010de60 0x7f472019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f47080b6670 0x7f47080b8b80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4720105380 0x7f472019aa50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 --2- 172.21.15.31:0/1876214499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4720104a30 0x7f472019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 >> 172.21.15.31:0/1876214499 conn(0x7f47200797b0 msgr2=0x7f4720100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 shutdown_connections 2024-09-21T06:10:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:36.209+0000 7f4726b01640 1 -- 172.21.15.31:0/1876214499 wait complete. 2024-09-21T06:10:36.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626478 got 180388626481 for osd.6 2024-09-21T06:10:36.674 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:36 smithi031 ceph-mon[31237]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3291850275' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:10:36.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1876214499' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:10:37.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:36 smithi110 ceph-mon[34550]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:37.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3291850275' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:10:37.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1876214499' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:10:37.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:36 smithi107 ceph-mon[34380]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:37.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3291850275' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:10:37.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1876214499' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:10:37.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906974 for osd.10 2024-09-21T06:10:37.303 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:38.001 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705741 got 38654705743 for osd.0 2024-09-21T06:10:38.001 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 -- 172.21.15.31:0/2758649373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 msgr2=0x7fbd88110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2758649373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd88110330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009ca0 tx=0x7fbd8002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 -- 172.21.15.31:0/2758649373 shutdown_connections 2024-09-21T06:10:38.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2758649373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd88110330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2758649373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 0x7fbd8810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2758649373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 0x7fbd88104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 -- 172.21.15.31:0/2758649373 >> 172.21.15.31:0/2758649373 conn(0x7fbd88079760 msgr2=0x7fbd88079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 -- 172.21.15.31:0/2758649373 shutdown_connections 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.783+0000 7fbd8e739640 1 -- 172.21.15.31:0/2758649373 wait complete. 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.784+0000 7fbd8e739640 1 Processor -- start 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.784+0000 7fbd8e739640 1 -- start start 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.784+0000 7fbd8e739640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 0x7fbd88102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.784+0000 7fbd8e739640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 0x7fbd88102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.784+0000 7fbd8e739640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd881031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8e739640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88113aa0 con 0x7fbd8810de60 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8e739640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88113920 con 0x7fbd88104a30 2024-09-21T06:10:38.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8e739640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88113c20 con 0x7fbd88105380 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd881031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd881031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53036/0 (socket says 172.21.15.31:53036) 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 -- 172.21.15.31:0/2141326503 learned_addr learned my addr 172.21.15.31:0/2141326503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd7ffff640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 0x7fbd88102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd87fff640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 0x7fbd88102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 -- 172.21.15.31:0/2141326503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 msgr2=0x7fbd88102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 0x7fbd88102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 -- 172.21.15.31:0/2141326503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 msgr2=0x7fbd88102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 0x7fbd88102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 -- 172.21.15.31:0/2141326503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd881a8620 con 0x7fbd8810de60 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd7ffff640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 0x7fbd88102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd87fff640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 0x7fbd88102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:38.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8ccd6640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd881031f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbd800098d0 tx=0x7fbd800317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80003b70 con 0x7fbd8810de60 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd881a88b0 con 0x7fbd8810de60 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80003d10 con 0x7fbd8810de60 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8002fa20 con 0x7fbd8810de60 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.785+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881a8df0 con 0x7fbd8810de60 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.786+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8806c1f0 con 0x7fbd8810de60 2024-09-21T06:10:38.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.788+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbd8002fc20 con 0x7fbd8810de60 2024-09-21T06:10:38.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.789+0000 7fbd85ffb640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbd740b65f0 0x7fbd740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:38.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.789+0000 7fbd87fff640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbd740b65f0 0x7fbd740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:38.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.789+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbd80108b50 con 0x7fbd8810de60 2024-09-21T06:10:38.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.790+0000 7fbd87fff640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbd740b65f0 0x7fbd740b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd881041c0 tx=0x7fbd78005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:38.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:38.794+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd800cf1a0 con 0x7fbd8810de60 2024-09-21T06:10:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.010+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbd88103a70 con 0x7fbd8810de60 2024-09-21T06:10:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.011+0000 7fbd85ffb640 1 -- 172.21.15.31:0/2141326503 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd800d3580 con 0x7fbd8810de60 2024-09-21T06:10:39.013 INFO:teuthology.orchestra.run.smithi031.stdout:81604378694 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbd740b65f0 msgr2=0x7fbd740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbd740b65f0 0x7fbd740b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd881041c0 tx=0x7fbd78005f90 comp rx=0 tx=0).stop 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 msgr2=0x7fbd881031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd881031f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbd800098d0 tx=0x7fbd800317b0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 shutdown_connections 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbd740b65f0 0x7fbd740b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd8810de60 0x7fbd881031f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd88105380 0x7fbd88102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 --2- 172.21.15.31:0/2141326503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd88104a30 0x7fbd88102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 >> 172.21.15.31:0/2141326503 conn(0x7fbd88079760 msgr2=0x7fbd88107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 shutdown_connections 2024-09-21T06:10:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:39.013+0000 7fbd8e739640 1 -- 172.21.15.31:0/2141326503 wait complete. 2024-09-21T06:10:39.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:38 smithi031 ceph-mon[31237]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:38 smithi110 ceph-mon[34550]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:38 smithi107 ceph-mon[34380]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:39.623 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378689 got 81604378694 for osd.2 2024-09-21T06:10:39.624 DEBUG:teuthology.parallel:result is None 2024-09-21T06:10:39.624 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:10:39.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:10:39.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:39 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2141326503' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:10:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:40.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:39 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2141326503' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:10:40.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:39 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2141326503' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:10:40.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 -- 172.21.15.31:0/2703973074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 msgr2=0x7f4784110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 --2- 172.21.15.31:0/2703973074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f4784110640 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f477c009ca0 tx=0x7f477c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 -- 172.21.15.31:0/2703973074 shutdown_connections 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 --2- 172.21.15.31:0/2703973074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f4784110640 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 --2- 172.21.15.31:0/2703973074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 0x7f47841098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 --2- 172.21.15.31:0/2703973074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 0x7f4784103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 -- 172.21.15.31:0/2703973074 >> 172.21.15.31:0/2703973074 conn(0x7f47840ff610 msgr2=0x7f4784101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 -- 172.21.15.31:0/2703973074 shutdown_connections 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.417+0000 7f4789797640 1 -- 172.21.15.31:0/2703973074 wait complete. 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 Processor -- start 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 -- start start 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 0x7f478419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 0x7f478419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f478419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4784113b30 con 0x7f4784109e10 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47841139b0 con 0x7f47841094c0 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f4789797640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4784113cb0 con 0x7f4784103b80 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f47837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f478419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f47837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f478419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50292/0 (socket says 172.21.15.31:50292) 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.418+0000 7f47837fe640 1 -- 172.21.15.31:0/3118972164 learned_addr learned my addr 172.21.15.31:0/3118972164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47827fc640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 0x7f478419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f4782ffd640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 0x7f478419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47837fe640 1 -- 172.21.15.31:0/3118972164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 msgr2=0x7f478419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47837fe640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 0x7f478419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47837fe640 1 -- 172.21.15.31:0/3118972164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 msgr2=0x7f478419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47837fe640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 0x7f478419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47837fe640 1 -- 172.21.15.31:0/3118972164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47841a0330 con 0x7f4784109e10 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47827fc640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 0x7f478419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f4782ffd640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 0x7f478419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f47837fe640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f478419fcc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f477c005cc0 tx=0x7f477c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c003cf0 con 0x7f4784109e10 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47841a05c0 con 0x7f4784109e10 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477c0044f0 con 0x7f4784109e10 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c02fd10 con 0x7f4784109e10 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.419+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841a6e40 con 0x7f4784109e10 2024-09-21T06:10:40.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.420+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784103f90 con 0x7f4784109e10 2024-09-21T06:10:40.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.422+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f477c004040 con 0x7f4784109e10 2024-09-21T06:10:40.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.423+0000 7f476bfff640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f476c0b6620 0x7f476c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:40.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.423+0000 7f4782ffd640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f476c0b6620 0x7f476c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:40.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.423+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f477c108cc0 con 0x7f4784109e10 2024-09-21T06:10:40.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.423+0000 7f4782ffd640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f476c0b6620 0x7f476c0b8b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4774009930 tx=0x7f4774008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:40.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.426+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f477c0cf310 con 0x7f4784109e10 2024-09-21T06:10:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.609+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f478410bc70 con 0x7f476c0b6620 2024-09-21T06:10:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.615+0000 7f476bfff640 1 -- 172.21.15.31:0/3118972164 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258641 (secure 0 0 0) 0x7f4774009140 con 0x7f476c0b6620 2024-09-21T06:10:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:10:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f476c0b6620 msgr2=0x7f476c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f476c0b6620 0x7f476c0b8b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4774009930 tx=0x7f4774008040 comp rx=0 tx=0).stop 2024-09-21T06:10:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 msgr2=0x7f478419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f478419fcc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f477c005cc0 tx=0x7f477c02f470 comp rx=0 tx=0).stop 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 shutdown_connections 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f476c0b6620 0x7f476c0b8b30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4784109e10 0x7f478419fcc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47841094c0 0x7f478419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 --2- 172.21.15.31:0/3118972164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4784103b80 0x7f478419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 >> 172.21.15.31:0/3118972164 conn(0x7f47840ff610 msgr2=0x7f4784108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 shutdown_connections 2024-09-21T06:10:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:40.618+0000 7f4789797640 1 -- 172.21.15.31:0/3118972164 wait complete. 2024-09-21T06:10:40.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:40 smithi031 ceph-mon[31237]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:40.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:40.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:40.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:40.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:40.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:40 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.264 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-21T06:10:39.609601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658357,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":196,"num_read_kb":193,"num_write":208,"num_write_kb":2245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382028,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559796,"statfs":{"total":1151940427776,"available":1151549231104,"internally_reserved":0,"allocated":12177408,"data_stored":6963147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.392039+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:10:07.392039+0000","last_peered":"2024-09-21T06:10:07.392039+0000","last_clean":"2024-09-21T06:10:07.392039+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:10:07.392039+0000","last_undegraded":"2024-09-21T06:10:07.392039+0000","last_fullsized":"2024-09-21T06:10:07.392039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397430+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:10:07.397430+0000","last_peered":"2024-09-21T06:10:07.397430+0000","last_clean":"2024-09-21T06:10:07.397430+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:10:07.397430+0000","last_undegraded":"2024-09-21T06:10:07.397430+0000","last_fullsized":"2024-09-21T06:10:07.397430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397151+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:10:07.397151+0000","last_peered":"2024-09-21T06:10:07.397151+0000","last_clean":"2024-09-21T06:10:07.397151+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:10:07.397151+0000","last_undegraded":"2024-09-21T06:10:07.397151+0000","last_fullsized":"2024-09-21T06:10:07.397151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395249+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:10:07.395249+0000","last_peered":"2024-09-21T06:10:07.395249+0000","last_clean":"2024-09-21T06:10:07.395249+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:10:07.395249+0000","last_undegraded":"2024-09-21T06:10:07.395249+0000","last_fullsized":"2024-09-21T06:10:07.395249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395737+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:10:07.395737+0000","last_peered":"2024-09-21T06:10:07.395737+0000","last_clean":"2024-09-21T06:10:07.395737+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:10:07.395737+0000","last_undegraded":"2024-09-21T06:10:07.395737+0000","last_fullsized":"2024-09-21T06:10:07.395737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.586572+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:10:07.586572+0000","last_peered":"2024-09-21T06:10:07.586572+0000","last_clean":"2024-09-21T06:10:07.586572+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:10:07.586572+0000","last_undegraded":"2024-09-21T06:10:07.586572+0000","last_fullsized":"2024-09-21T06:10:07.586572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394336+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:10:07.394336+0000","last_peered":"2024-09-21T06:10:07.394336+0000","last_clean":"2024-09-21T06:10:07.394336+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:10:07.394336+0000","last_undegraded":"2024-09-21T06:10:07.394336+0000","last_fullsized":"2024-09-21T06:10:07.394336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393528+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:10:07.393528+0000","last_peered":"2024-09-21T06:10:07.393528+0000","last_clean":"2024-09-21T06:10:07.393528+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:10:07.393528+0000","last_undegraded":"2024-09-21T06:10:07.393528+0000","last_fullsized":"2024-09-21T06:10:07.393528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476851+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:10:07.476851+0000","last_peered":"2024-09-21T06:10:07.476851+0000","last_clean":"2024-09-21T06:10:07.476851+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:10:07.476851+0000","last_undegraded":"2024-09-21T06:10:07.476851+0000","last_fullsized":"2024-09-21T06:10:07.476851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587278+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:10:07.587278+0000","last_peered":"2024-09-21T06:10:07.587278+0000","last_clean":"2024-09-21T06:10:07.587278+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:10:07.587278+0000","last_undegraded":"2024-09-21T06:10:07.587278+0000","last_fullsized":"2024-09-21T06:10:07.587278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395156+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:10:07.395156+0000","last_peered":"2024-09-21T06:10:07.395156+0000","last_clean":"2024-09-21T06:10:07.395156+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:10:07.395156+0000","last_undegraded":"2024-09-21T06:10:07.395156+0000","last_fullsized":"2024-09-21T06:10:07.395156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395372+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:10:07.395372+0000","last_peered":"2024-09-21T06:10:07.395372+0000","last_clean":"2024-09-21T06:10:07.395372+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:10:07.395372+0000","last_undegraded":"2024-09-21T06:10:07.395372+0000","last_fullsized":"2024-09-21T06:10:07.395372+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399041+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:10:07.399041+0000","last_peered":"2024-09-21T06:10:07.399041+0000","last_clean":"2024-09-21T06:10:07.399041+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:10:07.399041+0000","last_undegraded":"2024-09-21T06:10:07.399041+0000","last_fullsized":"2024-09-21T06:10:07.399041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396843+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:10:07.396843+0000","last_peered":"2024-09-21T06:10:07.396843+0000","last_clean":"2024-09-21T06:10:07.396843+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:10:07.396843+0000","last_undegraded":"2024-09-21T06:10:07.396843+0000","last_fullsized":"2024-09-21T06:10:07.396843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396575+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:10:07.396575+0000","last_peered":"2024-09-21T06:10:07.396575+0000","last_clean":"2024-09-21T06:10:07.396575+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:10:07.396575+0000","last_undegraded":"2024-09-21T06:10:07.396575+0000","last_fullsized":"2024-09-21T06:10:07.396575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476291+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:10:07.476291+0000","last_peered":"2024-09-21T06:10:07.476291+0000","last_clean":"2024-09-21T06:10:07.476291+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:10:07.476291+0000","last_undegraded":"2024-09-21T06:10:07.476291+0000","last_fullsized":"2024-09-21T06:10:07.476291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.392104+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:10:07.392104+0000","last_peered":"2024-09-21T06:10:07.392104+0000","last_clean":"2024-09-21T06:10:07.392104+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:10:07.392104+0000","last_undegraded":"2024-09-21T06:10:07.392104+0000","last_fullsized":"2024-09-21T06:10:07.392104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399819+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:10:07.399819+0000","last_peered":"2024-09-21T06:10:07.399819+0000","last_clean":"2024-09-21T06:10:07.399819+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:10:07.399819+0000","last_undegraded":"2024-09-21T06:10:07.399819+0000","last_fullsized":"2024-09-21T06:10:07.399819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394421+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:10:07.394421+0000","last_peered":"2024-09-21T06:10:07.394421+0000","last_clean":"2024-09-21T06:10:07.394421+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:10:07.394421+0000","last_undegraded":"2024-09-21T06:10:07.394421+0000","last_fullsized":"2024-09-21T06:10:07.394421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391735+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:10:07.391735+0000","last_peered":"2024-09-21T06:10:07.391735+0000","last_clean":"2024-09-21T06:10:07.391735+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:10:07.391735+0000","last_undegraded":"2024-09-21T06:10:07.391735+0000","last_fullsized":"2024-09-21T06:10:07.391735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399156+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:10:07.399156+0000","last_peered":"2024-09-21T06:10:07.399156+0000","last_clean":"2024-09-21T06:10:07.399156+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:10:07.399156+0000","last_undegraded":"2024-09-21T06:10:07.399156+0000","last_fullsized":"2024-09-21T06:10:07.399156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476468+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:10:07.476468+0000","last_peered":"2024-09-21T06:10:07.476468+0000","last_clean":"2024-09-21T06:10:07.476468+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:10:07.476468+0000","last_undegraded":"2024-09-21T06:10:07.476468+0000","last_fullsized":"2024-09-21T06:10:07.476468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393583+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:10:07.393583+0000","last_peered":"2024-09-21T06:10:07.393583+0000","last_clean":"2024-09-21T06:10:07.393583+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:10:07.393583+0000","last_undegraded":"2024-09-21T06:10:07.393583+0000","last_fullsized":"2024-09-21T06:10:07.393583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398228+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:10:07.398228+0000","last_peered":"2024-09-21T06:10:07.398228+0000","last_clean":"2024-09-21T06:10:07.398228+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:10:07.398228+0000","last_undegraded":"2024-09-21T06:10:07.398228+0000","last_fullsized":"2024-09-21T06:10:07.398228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394350+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:10:07.394350+0000","last_peered":"2024-09-21T06:10:07.394350+0000","last_clean":"2024-09-21T06:10:07.394350+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:10:07.394350+0000","last_undegraded":"2024-09-21T06:10:07.394350+0000","last_fullsized":"2024-09-21T06:10:07.394350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394550+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:10:07.394550+0000","last_peered":"2024-09-21T06:10:07.394550+0000","last_clean":"2024-09-21T06:10:07.394550+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:10:07.394550+0000","last_undegraded":"2024-09-21T06:10:07.394550+0000","last_fullsized":"2024-09-21T06:10:07.394550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391807+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:10:07.391807+0000","last_peered":"2024-09-21T06:10:07.391807+0000","last_clean":"2024-09-21T06:10:07.391807+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:10:07.391807+0000","last_undegraded":"2024-09-21T06:10:07.391807+0000","last_fullsized":"2024-09-21T06:10:07.391807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397343+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:10:07.397343+0000","last_peered":"2024-09-21T06:10:07.397343+0000","last_clean":"2024-09-21T06:10:07.397343+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:10:07.397343+0000","last_undegraded":"2024-09-21T06:10:07.397343+0000","last_fullsized":"2024-09-21T06:10:07.397343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403778+0000","last_change":"2024-09-21T06:10:07.403662+0000","last_active":"2024-09-21T06:10:07.403778+0000","last_peered":"2024-09-21T06:10:07.403778+0000","last_clean":"2024-09-21T06:10:07.403778+0000","last_became_active":"2024-09-21T06:10:07.403220+0000","last_became_peered":"2024-09-21T06:10:07.403220+0000","last_unstale":"2024-09-21T06:10:07.403778+0000","last_undegraded":"2024-09-21T06:10:07.403778+0000","last_fullsized":"2024-09-21T06:10:07.403778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:42.670948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394118+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:10:07.394118+0000","last_peered":"2024-09-21T06:10:07.394118+0000","last_clean":"2024-09-21T06:10:07.394118+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:10:07.394118+0000","last_undegraded":"2024-09-21T06:10:07.394118+0000","last_fullsized":"2024-09-21T06:10:07.394118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394240+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:10:07.394240+0000","last_peered":"2024-09-21T06:10:07.394240+0000","last_clean":"2024-09-21T06:10:07.394240+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:10:07.394240+0000","last_undegraded":"2024-09-21T06:10:07.394240+0000","last_fullsized":"2024-09-21T06:10:07.394240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391629+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:10:07.391629+0000","last_peered":"2024-09-21T06:10:07.391629+0000","last_clean":"2024-09-21T06:10:07.391629+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:10:07.391629+0000","last_undegraded":"2024-09-21T06:10:07.391629+0000","last_fullsized":"2024-09-21T06:10:07.391629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399813+0000","last_change":"2024-09-21T06:10:07.399633+0000","last_active":"2024-09-21T06:10:07.399813+0000","last_peered":"2024-09-21T06:10:07.399813+0000","last_clean":"2024-09-21T06:10:07.399813+0000","last_became_active":"2024-09-21T06:10:07.399200+0000","last_became_peered":"2024-09-21T06:10:07.399200+0000","last_unstale":"2024-09-21T06:10:07.399813+0000","last_undegraded":"2024-09-21T06:10:07.399813+0000","last_fullsized":"2024-09-21T06:10:07.399813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:16.830371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398713+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:10:07.398713+0000","last_peered":"2024-09-21T06:10:07.398713+0000","last_clean":"2024-09-21T06:10:07.398713+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:10:07.398713+0000","last_undegraded":"2024-09-21T06:10:07.398713+0000","last_fullsized":"2024-09-21T06:10:07.398713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398466+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:10:07.398466+0000","last_peered":"2024-09-21T06:10:07.398466+0000","last_clean":"2024-09-21T06:10:07.398466+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:10:07.398466+0000","last_undegraded":"2024-09-21T06:10:07.398466+0000","last_fullsized":"2024-09-21T06:10:07.398466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587183+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:10:07.587183+0000","last_peered":"2024-09-21T06:10:07.587183+0000","last_clean":"2024-09-21T06:10:07.587183+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:10:07.587183+0000","last_undegraded":"2024-09-21T06:10:07.587183+0000","last_fullsized":"2024-09-21T06:10:07.587183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396798+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:10:07.396798+0000","last_peered":"2024-09-21T06:10:07.396798+0000","last_clean":"2024-09-21T06:10:07.396798+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:10:07.396798+0000","last_undegraded":"2024-09-21T06:10:07.396798+0000","last_fullsized":"2024-09-21T06:10:07.396798+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393550+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:10:07.393550+0000","last_peered":"2024-09-21T06:10:07.393550+0000","last_clean":"2024-09-21T06:10:07.393550+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:10:07.393550+0000","last_undegraded":"2024-09-21T06:10:07.393550+0000","last_fullsized":"2024-09-21T06:10:07.393550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398807+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:10:07.398807+0000","last_peered":"2024-09-21T06:10:07.398807+0000","last_clean":"2024-09-21T06:10:07.398807+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:10:07.398807+0000","last_undegraded":"2024-09-21T06:10:07.398807+0000","last_fullsized":"2024-09-21T06:10:07.398807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397914+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:10:07.397914+0000","last_peered":"2024-09-21T06:10:07.397914+0000","last_clean":"2024-09-21T06:10:07.397914+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:10:07.397914+0000","last_undegraded":"2024-09-21T06:10:07.397914+0000","last_fullsized":"2024-09-21T06:10:07.397914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587232+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:10:07.587232+0000","last_peered":"2024-09-21T06:10:07.587232+0000","last_clean":"2024-09-21T06:10:07.587232+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:10:07.587232+0000","last_undegraded":"2024-09-21T06:10:07.587232+0000","last_fullsized":"2024-09-21T06:10:07.587232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476696+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:10:07.476696+0000","last_peered":"2024-09-21T06:10:07.476696+0000","last_clean":"2024-09-21T06:10:07.476696+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:10:07.476696+0000","last_undegraded":"2024-09-21T06:10:07.476696+0000","last_fullsized":"2024-09-21T06:10:07.476696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395089+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:10:07.395089+0000","last_peered":"2024-09-21T06:10:07.395089+0000","last_clean":"2024-09-21T06:10:07.395089+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:10:07.395089+0000","last_undegraded":"2024-09-21T06:10:07.395089+0000","last_fullsized":"2024-09-21T06:10:07.395089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398088+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:10:07.398088+0000","last_peered":"2024-09-21T06:10:07.398088+0000","last_clean":"2024-09-21T06:10:07.398088+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:10:07.398088+0000","last_undegraded":"2024-09-21T06:10:07.398088+0000","last_fullsized":"2024-09-21T06:10:07.398088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399080+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:10:07.399080+0000","last_peered":"2024-09-21T06:10:07.399080+0000","last_clean":"2024-09-21T06:10:07.399080+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:10:07.399080+0000","last_undegraded":"2024-09-21T06:10:07.399080+0000","last_fullsized":"2024-09-21T06:10:07.399080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476411+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:10:07.476411+0000","last_peered":"2024-09-21T06:10:07.476411+0000","last_clean":"2024-09-21T06:10:07.476411+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:10:07.476411+0000","last_undegraded":"2024-09-21T06:10:07.476411+0000","last_fullsized":"2024-09-21T06:10:07.476411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":161,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:34.402710+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:10:34.402710+0000","last_peered":"2024-09-21T06:10:34.402710+0000","last_clean":"2024-09-21T06:10:34.402710+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:10:34.402710+0000","last_undegraded":"2024-09-21T06:10:34.402710+0000","last_fullsized":"2024-09-21T06:10:34.402710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66957,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":111,"num_read_kb":120,"num_write":27,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395465+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:10:07.395465+0000","last_peered":"2024-09-21T06:10:07.395465+0000","last_clean":"2024-09-21T06:10:07.395465+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:10:07.395465+0000","last_undegraded":"2024-09-21T06:10:07.395465+0000","last_fullsized":"2024-09-21T06:10:07.395465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394811+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:10:07.394811+0000","last_peered":"2024-09-21T06:10:07.394811+0000","last_clean":"2024-09-21T06:10:07.394811+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:10:07.394811+0000","last_undegraded":"2024-09-21T06:10:07.394811+0000","last_fullsized":"2024-09-21T06:10:07.394811+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397834+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:10:07.397834+0000","last_peered":"2024-09-21T06:10:07.397834+0000","last_clean":"2024-09-21T06:10:07.397834+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:10:07.397834+0000","last_undegraded":"2024-09-21T06:10:07.397834+0000","last_fullsized":"2024-09-21T06:10:07.397834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398372+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:10:07.398372+0000","last_peered":"2024-09-21T06:10:07.398372+0000","last_clean":"2024-09-21T06:10:07.398372+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:10:07.398372+0000","last_undegraded":"2024-09-21T06:10:07.398372+0000","last_fullsized":"2024-09-21T06:10:07.398372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401281+0000","last_change":"2024-09-21T06:10:07.401185+0000","last_active":"2024-09-21T06:10:07.401281+0000","last_peered":"2024-09-21T06:10:07.401281+0000","last_clean":"2024-09-21T06:10:07.401281+0000","last_became_active":"2024-09-21T06:10:07.400789+0000","last_became_peered":"2024-09-21T06:10:07.400789+0000","last_unstale":"2024-09-21T06:10:07.401281+0000","last_undegraded":"2024-09-21T06:10:07.401281+0000","last_fullsized":"2024-09-21T06:10:07.401281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:03.201239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394895+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:10:07.394895+0000","last_peered":"2024-09-21T06:10:07.394895+0000","last_clean":"2024-09-21T06:10:07.394895+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:10:07.394895+0000","last_undegraded":"2024-09-21T06:10:07.394895+0000","last_fullsized":"2024-09-21T06:10:07.394895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.588324+0000","last_change":"2024-09-21T06:10:07.588201+0000","last_active":"2024-09-21T06:10:07.588324+0000","last_peered":"2024-09-21T06:10:07.588324+0000","last_clean":"2024-09-21T06:10:07.588324+0000","last_became_active":"2024-09-21T06:10:07.587788+0000","last_became_peered":"2024-09-21T06:10:07.587788+0000","last_unstale":"2024-09-21T06:10:07.588324+0000","last_undegraded":"2024-09-21T06:10:07.588324+0000","last_fullsized":"2024-09-21T06:10:07.588324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:16.617435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394307+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:10:07.394307+0000","last_peered":"2024-09-21T06:10:07.394307+0000","last_clean":"2024-09-21T06:10:07.394307+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:10:07.394307+0000","last_undegraded":"2024-09-21T06:10:07.394307+0000","last_fullsized":"2024-09-21T06:10:07.394307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.478379+0000","last_change":"2024-09-21T06:10:07.478289+0000","last_active":"2024-09-21T06:10:07.478379+0000","last_peered":"2024-09-21T06:10:07.478379+0000","last_clean":"2024-09-21T06:10:07.478379+0000","last_became_active":"2024-09-21T06:10:07.477828+0000","last_became_peered":"2024-09-21T06:10:07.477828+0000","last_unstale":"2024-09-21T06:10:07.478379+0000","last_undegraded":"2024-09-21T06:10:07.478379+0000","last_fullsized":"2024-09-21T06:10:07.478379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.721612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395262+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:10:07.395262+0000","last_peered":"2024-09-21T06:10:07.395262+0000","last_clean":"2024-09-21T06:10:07.395262+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:10:07.395262+0000","last_undegraded":"2024-09-21T06:10:07.395262+0000","last_fullsized":"2024-09-21T06:10:07.395262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391874+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:10:07.391874+0000","last_peered":"2024-09-21T06:10:07.391874+0000","last_clean":"2024-09-21T06:10:07.391874+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:10:07.391874+0000","last_undegraded":"2024-09-21T06:10:07.391874+0000","last_fullsized":"2024-09-21T06:10:07.391874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406063+0000","last_change":"2024-09-21T06:10:07.405946+0000","last_active":"2024-09-21T06:10:07.406063+0000","last_peered":"2024-09-21T06:10:07.406063+0000","last_clean":"2024-09-21T06:10:07.406063+0000","last_became_active":"2024-09-21T06:10:07.401620+0000","last_became_peered":"2024-09-21T06:10:07.401620+0000","last_unstale":"2024-09-21T06:10:07.406063+0000","last_undegraded":"2024-09-21T06:10:07.406063+0000","last_fullsized":"2024-09-21T06:10:07.406063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:17.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395217+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:10:07.395217+0000","last_peered":"2024-09-21T06:10:07.395217+0000","last_clean":"2024-09-21T06:10:07.395217+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:10:07.395217+0000","last_undegraded":"2024-09-21T06:10:07.395217+0000","last_fullsized":"2024-09-21T06:10:07.395217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399215+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:10:07.399215+0000","last_peered":"2024-09-21T06:10:07.399215+0000","last_clean":"2024-09-21T06:10:07.399215+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:10:07.399215+0000","last_undegraded":"2024-09-21T06:10:07.399215+0000","last_fullsized":"2024-09-21T06:10:07.399215+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406991+0000","last_change":"2024-09-21T06:10:07.406892+0000","last_active":"2024-09-21T06:10:07.406991+0000","last_peered":"2024-09-21T06:10:07.406991+0000","last_clean":"2024-09-21T06:10:07.406991+0000","last_became_active":"2024-09-21T06:10:07.402546+0000","last_became_peered":"2024-09-21T06:10:07.402546+0000","last_unstale":"2024-09-21T06:10:07.406991+0000","last_undegraded":"2024-09-21T06:10:07.406991+0000","last_fullsized":"2024-09-21T06:10:07.406991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.081510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394662+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:10:07.394662+0000","last_peered":"2024-09-21T06:10:07.394662+0000","last_clean":"2024-09-21T06:10:07.394662+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:10:07.394662+0000","last_undegraded":"2024-09-21T06:10:07.394662+0000","last_fullsized":"2024-09-21T06:10:07.394662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395205+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:10:07.395205+0000","last_peered":"2024-09-21T06:10:07.395205+0000","last_clean":"2024-09-21T06:10:07.395205+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:10:07.395205+0000","last_undegraded":"2024-09-21T06:10:07.395205+0000","last_fullsized":"2024-09-21T06:10:07.395205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395664+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:10:07.395664+0000","last_peered":"2024-09-21T06:10:07.395664+0000","last_clean":"2024-09-21T06:10:07.395664+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:10:07.395664+0000","last_undegraded":"2024-09-21T06:10:07.395664+0000","last_fullsized":"2024-09-21T06:10:07.395664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395073+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:10:07.395073+0000","last_peered":"2024-09-21T06:10:07.395073+0000","last_clean":"2024-09-21T06:10:07.395073+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:10:07.395073+0000","last_undegraded":"2024-09-21T06:10:07.395073+0000","last_fullsized":"2024-09-21T06:10:07.395073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393965+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:10:07.393965+0000","last_peered":"2024-09-21T06:10:07.393965+0000","last_clean":"2024-09-21T06:10:07.393965+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:10:07.393965+0000","last_undegraded":"2024-09-21T06:10:07.393965+0000","last_fullsized":"2024-09-21T06:10:07.393965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394546+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:10:07.394546+0000","last_peered":"2024-09-21T06:10:07.394546+0000","last_clean":"2024-09-21T06:10:07.394546+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:10:07.394546+0000","last_undegraded":"2024-09-21T06:10:07.394546+0000","last_fullsized":"2024-09-21T06:10:07.394546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396702+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:10:07.396702+0000","last_peered":"2024-09-21T06:10:07.396702+0000","last_clean":"2024-09-21T06:10:07.396702+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:10:07.396702+0000","last_undegraded":"2024-09-21T06:10:07.396702+0000","last_fullsized":"2024-09-21T06:10:07.396702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396950+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:10:07.396950+0000","last_peered":"2024-09-21T06:10:07.396950+0000","last_clean":"2024-09-21T06:10:07.396950+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:10:07.396950+0000","last_undegraded":"2024-09-21T06:10:07.396950+0000","last_fullsized":"2024-09-21T06:10:07.396950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399104+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:10:07.399104+0000","last_peered":"2024-09-21T06:10:07.399104+0000","last_clean":"2024-09-21T06:10:07.399104+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:10:07.399104+0000","last_undegraded":"2024-09-21T06:10:07.399104+0000","last_fullsized":"2024-09-21T06:10:07.399104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394998+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:10:07.394998+0000","last_peered":"2024-09-21T06:10:07.394998+0000","last_clean":"2024-09-21T06:10:07.394998+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:10:07.394998+0000","last_undegraded":"2024-09-21T06:10:07.394998+0000","last_fullsized":"2024-09-21T06:10:07.394998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.586879+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:10:07.586879+0000","last_peered":"2024-09-21T06:10:07.586879+0000","last_clean":"2024-09-21T06:10:07.586879+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:10:07.586879+0000","last_undegraded":"2024-09-21T06:10:07.586879+0000","last_fullsized":"2024-09-21T06:10:07.586879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401748+0000","last_change":"2024-09-21T06:10:07.401640+0000","last_active":"2024-09-21T06:10:07.401748+0000","last_peered":"2024-09-21T06:10:07.401748+0000","last_clean":"2024-09-21T06:10:07.401748+0000","last_became_active":"2024-09-21T06:10:07.401242+0000","last_became_peered":"2024-09-21T06:10:07.401242+0000","last_unstale":"2024-09-21T06:10:07.401748+0000","last_undegraded":"2024-09-21T06:10:07.401748+0000","last_fullsized":"2024-09-21T06:10:07.401748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:25.335694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.586794+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:10:07.586794+0000","last_peered":"2024-09-21T06:10:07.586794+0000","last_clean":"2024-09-21T06:10:07.586794+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:10:07.586794+0000","last_undegraded":"2024-09-21T06:10:07.586794+0000","last_fullsized":"2024-09-21T06:10:07.586794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395763+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:10:07.395763+0000","last_peered":"2024-09-21T06:10:07.395763+0000","last_clean":"2024-09-21T06:10:07.395763+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:10:07.395763+0000","last_undegraded":"2024-09-21T06:10:07.395763+0000","last_fullsized":"2024-09-21T06:10:07.395763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397447+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:10:07.397447+0000","last_peered":"2024-09-21T06:10:07.397447+0000","last_clean":"2024-09-21T06:10:07.397447+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:10:07.397447+0000","last_undegraded":"2024-09-21T06:10:07.397447+0000","last_fullsized":"2024-09-21T06:10:07.397447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399941+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:10:07.399941+0000","last_peered":"2024-09-21T06:10:07.399941+0000","last_clean":"2024-09-21T06:10:07.399941+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:10:07.399941+0000","last_undegraded":"2024-09-21T06:10:07.399941+0000","last_fullsized":"2024-09-21T06:10:07.399941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394452+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:10:07.394452+0000","last_peered":"2024-09-21T06:10:07.394452+0000","last_clean":"2024-09-21T06:10:07.394452+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:10:07.394452+0000","last_undegraded":"2024-09-21T06:10:07.394452+0000","last_fullsized":"2024-09-21T06:10:07.394452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395509+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:10:07.395509+0000","last_peered":"2024-09-21T06:10:07.395509+0000","last_clean":"2024-09-21T06:10:07.395509+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:10:07.395509+0000","last_undegraded":"2024-09-21T06:10:07.395509+0000","last_fullsized":"2024-09-21T06:10:07.395509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397556+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:10:07.397556+0000","last_peered":"2024-09-21T06:10:07.397556+0000","last_clean":"2024-09-21T06:10:07.397556+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:10:07.397556+0000","last_undegraded":"2024-09-21T06:10:07.397556+0000","last_fullsized":"2024-09-21T06:10:07.397556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476406+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:10:07.476406+0000","last_peered":"2024-09-21T06:10:07.476406+0000","last_clean":"2024-09-21T06:10:07.476406+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:10:07.476406+0000","last_undegraded":"2024-09-21T06:10:07.476406+0000","last_fullsized":"2024-09-21T06:10:07.476406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394434+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:10:07.394434+0000","last_peered":"2024-09-21T06:10:07.394434+0000","last_clean":"2024-09-21T06:10:07.394434+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:10:07.394434+0000","last_undegraded":"2024-09-21T06:10:07.394434+0000","last_fullsized":"2024-09-21T06:10:07.394434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393700+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:10:07.393700+0000","last_peered":"2024-09-21T06:10:07.393700+0000","last_clean":"2024-09-21T06:10:07.393700+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:10:07.393700+0000","last_undegraded":"2024-09-21T06:10:07.393700+0000","last_fullsized":"2024-09-21T06:10:07.393700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391704+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:10:07.391704+0000","last_peered":"2024-09-21T06:10:07.391704+0000","last_clean":"2024-09-21T06:10:07.391704+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:10:07.391704+0000","last_undegraded":"2024-09-21T06:10:07.391704+0000","last_fullsized":"2024-09-21T06:10:07.391704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394466+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:10:07.394466+0000","last_peered":"2024-09-21T06:10:07.394466+0000","last_clean":"2024-09-21T06:10:07.394466+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:10:07.394466+0000","last_undegraded":"2024-09-21T06:10:07.394466+0000","last_fullsized":"2024-09-21T06:10:07.394466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393751+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:10:07.393751+0000","last_peered":"2024-09-21T06:10:07.393751+0000","last_clean":"2024-09-21T06:10:07.393751+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:10:07.393751+0000","last_undegraded":"2024-09-21T06:10:07.393751+0000","last_fullsized":"2024-09-21T06:10:07.393751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403122+0000","last_change":"2024-09-21T06:10:07.402379+0000","last_active":"2024-09-21T06:10:07.403122+0000","last_peered":"2024-09-21T06:10:07.403122+0000","last_clean":"2024-09-21T06:10:07.403122+0000","last_became_active":"2024-09-21T06:10:07.396828+0000","last_became_peered":"2024-09-21T06:10:07.396828+0000","last_unstale":"2024-09-21T06:10:07.403122+0000","last_undegraded":"2024-09-21T06:10:07.403122+0000","last_fullsized":"2024-09-21T06:10:07.403122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:26.181343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394763+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:10:07.394763+0000","last_peered":"2024-09-21T06:10:07.394763+0000","last_clean":"2024-09-21T06:10:07.394763+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:10:07.394763+0000","last_undegraded":"2024-09-21T06:10:07.394763+0000","last_fullsized":"2024-09-21T06:10:07.394763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398961+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:10:07.398961+0000","last_peered":"2024-09-21T06:10:07.398961+0000","last_clean":"2024-09-21T06:10:07.398961+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:10:07.398961+0000","last_undegraded":"2024-09-21T06:10:07.398961+0000","last_fullsized":"2024-09-21T06:10:07.398961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395391+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:10:07.395391+0000","last_peered":"2024-09-21T06:10:07.395391+0000","last_clean":"2024-09-21T06:10:07.395391+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:10:07.395391+0000","last_undegraded":"2024-09-21T06:10:07.395391+0000","last_fullsized":"2024-09-21T06:10:07.395391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476438+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:10:07.476438+0000","last_peered":"2024-09-21T06:10:07.476438+0000","last_clean":"2024-09-21T06:10:07.476438+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:10:07.476438+0000","last_undegraded":"2024-09-21T06:10:07.476438+0000","last_fullsized":"2024-09-21T06:10:07.476438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395635+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:10:07.395635+0000","last_peered":"2024-09-21T06:10:07.395635+0000","last_clean":"2024-09-21T06:10:07.395635+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:10:07.395635+0000","last_undegraded":"2024-09-21T06:10:07.395635+0000","last_fullsized":"2024-09-21T06:10:07.395635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397975+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:10:07.397975+0000","last_peered":"2024-09-21T06:10:07.397975+0000","last_clean":"2024-09-21T06:10:07.397975+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:10:07.397975+0000","last_undegraded":"2024-09-21T06:10:07.397975+0000","last_fullsized":"2024-09-21T06:10:07.397975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394530+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:10:07.394530+0000","last_peered":"2024-09-21T06:10:07.394530+0000","last_clean":"2024-09-21T06:10:07.394530+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:10:07.394530+0000","last_undegraded":"2024-09-21T06:10:07.394530+0000","last_fullsized":"2024-09-21T06:10:07.394530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394357+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:10:07.394357+0000","last_peered":"2024-09-21T06:10:07.394357+0000","last_clean":"2024-09-21T06:10:07.394357+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:10:07.394357+0000","last_undegraded":"2024-09-21T06:10:07.394357+0000","last_fullsized":"2024-09-21T06:10:07.394357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399889+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:10:07.399889+0000","last_peered":"2024-09-21T06:10:07.399889+0000","last_clean":"2024-09-21T06:10:07.399889+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:10:07.399889+0000","last_undegraded":"2024-09-21T06:10:07.399889+0000","last_fullsized":"2024-09-21T06:10:07.399889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67755,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":115,"num_read_kb":124,"num_write":83,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":203265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":839680,"data_stored":415673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1527808,"data_stored":1073309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":416372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266728,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":847872,"data_stored":416277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1441792,"data_stored":1006740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626482,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":415790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1449984,"data_stored":1006340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":415695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":415894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378694,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40412,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704740,"statfs":{"total":95995035648,"available":95953653760,"internally_reserved":0,"allocated":921600,"data_stored":482664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":917504,"data_stored":482630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":415763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:10:41.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:10:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:40 smithi110 ceph-mon[34550]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:40 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:40 smithi107 ceph-mon[34380]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:10:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:10:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:40 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:10:41.526 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:41.955 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:41 smithi031 ceph-mon[31237]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 -- 172.21.15.31:0/2387932866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 msgr2=0x7f99dc110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2387932866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc110320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009900 tx=0x7f99d402a410 comp rx=0 tx=0).stop 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 -- 172.21.15.31:0/2387932866 shutdown_connections 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2387932866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc110320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2387932866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99dc105290 0x7f99dc10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2387932866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc104860 0x7f99dc104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 -- 172.21.15.31:0/2387932866 >> 172.21.15.31:0/2387932866 conn(0x7f99dc079740 msgr2=0x7f99dc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 -- 172.21.15.31:0/2387932866 shutdown_connections 2024-09-21T06:10:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.978+0000 7f99e08cd640 1 -- 172.21.15.31:0/2387932866 wait complete. 2024-09-21T06:10:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 Processor -- start 2024-09-21T06:10:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 -- start start 2024-09-21T06:10:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99dc104860 0x7f99dc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc105290 0x7f99dc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc113ab0 con 0x7f99dc10de10 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc113930 con 0x7f99dc104860 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99e08cd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc113c30 con 0x7f99dc105290 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99dad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99dad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50308/0 (socket says 172.21.15.31:50308) 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.979+0000 7f99dad9d640 1 -- 172.21.15.31:0/2325356415 learned_addr learned my addr 172.21.15.31:0/2325356415 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99d9d9b640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc105290 0x7f99dc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99da59c640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99dc104860 0x7f99dc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99dad9d640 1 -- 172.21.15.31:0/2325356415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc105290 msgr2=0x7f99dc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99dad9d640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc105290 0x7f99dc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99dad9d640 1 -- 172.21.15.31:0/2325356415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99dc104860 msgr2=0x7f99dc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99dad9d640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99dc104860 0x7f99dc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99dad9d640 1 -- 172.21.15.31:0/2325356415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc1a0410 con 0x7f99dc10de10 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99dad9d640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc19fce0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f99d4004770 tx=0x7f99d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d403f030 con 0x7f99dc10de10 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc1a06a0 con 0x7f99dc10de10 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d4003aa0 con 0x7f99dc10de10 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4035070 con 0x7f99dc10de10 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.980+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc1a6e60 con 0x7f99dc10de10 2024-09-21T06:10:41.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.981+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc06c1e0 con 0x7f99dc10de10 2024-09-21T06:10:41.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.983+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f99d4042030 con 0x7f99dc10de10 2024-09-21T06:10:41.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.984+0000 7f99cb7fe640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99c40b65f0 0x7f99c40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:41.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.984+0000 7f99da59c640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99c40b65f0 0x7f99c40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:41.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.984+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f99d4107e70 con 0x7f99dc10de10 2024-09-21T06:10:41.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.984+0000 7f99da59c640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99c40b65f0 0x7f99c40b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005fd0 tx=0x7f99cc00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:41.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:41.987+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99d40d28a0 con 0x7f99dc10de10 2024-09-21T06:10:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.172+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99dc100c90 con 0x7f99c40b65f0 2024-09-21T06:10:42.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.176+0000 7f99cb7fe640 1 -- 172.21.15.31:0/2325356415 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258641 (secure 0 0 0) 0x7f99dc100c90 con 0x7f99c40b65f0 2024-09-21T06:10:42.180 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:42.181 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.179+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99c40b65f0 msgr2=0x7f99c40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.179+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99c40b65f0 0x7f99c40b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99cc005fd0 tx=0x7f99cc00b390 comp rx=0 tx=0).stop 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.179+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 msgr2=0x7f99dc19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.179+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc19fce0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f99d4004770 tx=0x7f99d4004270 comp rx=0 tx=0).stop 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 shutdown_connections 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99c40b65f0 0x7f99c40b8b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99dc10de10 0x7f99dc19fce0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99dc105290 0x7f99dc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:42.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 --2- 172.21.15.31:0/2325356415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99dc104860 0x7f99dc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:42.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 >> 172.21.15.31:0/2325356415 conn(0x7f99dc079740 msgr2=0x7f99dc0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:42.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 shutdown_connections 2024-09-21T06:10:42.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:42.180+0000 7f99e08cd640 1 -- 172.21.15.31:0/2325356415 wait complete. 2024-09-21T06:10:42.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:41 smithi110 ceph-mon[34550]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:42.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:41 smithi107 ceph-mon[34380]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:42.871 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-21T06:10:41.610147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658357,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":206,"num_read_kb":198,"num_write":208,"num_write_kb":2245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382028,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559796,"statfs":{"total":1151940427776,"available":1151549231104,"internally_reserved":0,"allocated":12177408,"data_stored":6963147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003035"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.392039+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:10:07.392039+0000","last_peered":"2024-09-21T06:10:07.392039+0000","last_clean":"2024-09-21T06:10:07.392039+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:10:07.392039+0000","last_undegraded":"2024-09-21T06:10:07.392039+0000","last_fullsized":"2024-09-21T06:10:07.392039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397430+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:10:07.397430+0000","last_peered":"2024-09-21T06:10:07.397430+0000","last_clean":"2024-09-21T06:10:07.397430+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:10:07.397430+0000","last_undegraded":"2024-09-21T06:10:07.397430+0000","last_fullsized":"2024-09-21T06:10:07.397430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397151+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:10:07.397151+0000","last_peered":"2024-09-21T06:10:07.397151+0000","last_clean":"2024-09-21T06:10:07.397151+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:10:07.397151+0000","last_undegraded":"2024-09-21T06:10:07.397151+0000","last_fullsized":"2024-09-21T06:10:07.397151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395249+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:10:07.395249+0000","last_peered":"2024-09-21T06:10:07.395249+0000","last_clean":"2024-09-21T06:10:07.395249+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:10:07.395249+0000","last_undegraded":"2024-09-21T06:10:07.395249+0000","last_fullsized":"2024-09-21T06:10:07.395249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395737+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:10:07.395737+0000","last_peered":"2024-09-21T06:10:07.395737+0000","last_clean":"2024-09-21T06:10:07.395737+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:10:07.395737+0000","last_undegraded":"2024-09-21T06:10:07.395737+0000","last_fullsized":"2024-09-21T06:10:07.395737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.586572+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:10:07.586572+0000","last_peered":"2024-09-21T06:10:07.586572+0000","last_clean":"2024-09-21T06:10:07.586572+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:10:07.586572+0000","last_undegraded":"2024-09-21T06:10:07.586572+0000","last_fullsized":"2024-09-21T06:10:07.586572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394336+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:10:07.394336+0000","last_peered":"2024-09-21T06:10:07.394336+0000","last_clean":"2024-09-21T06:10:07.394336+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:10:07.394336+0000","last_undegraded":"2024-09-21T06:10:07.394336+0000","last_fullsized":"2024-09-21T06:10:07.394336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393528+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:10:07.393528+0000","last_peered":"2024-09-21T06:10:07.393528+0000","last_clean":"2024-09-21T06:10:07.393528+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:10:07.393528+0000","last_undegraded":"2024-09-21T06:10:07.393528+0000","last_fullsized":"2024-09-21T06:10:07.393528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476851+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:10:07.476851+0000","last_peered":"2024-09-21T06:10:07.476851+0000","last_clean":"2024-09-21T06:10:07.476851+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:10:07.476851+0000","last_undegraded":"2024-09-21T06:10:07.476851+0000","last_fullsized":"2024-09-21T06:10:07.476851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587278+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:10:07.587278+0000","last_peered":"2024-09-21T06:10:07.587278+0000","last_clean":"2024-09-21T06:10:07.587278+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:10:07.587278+0000","last_undegraded":"2024-09-21T06:10:07.587278+0000","last_fullsized":"2024-09-21T06:10:07.587278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395156+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:10:07.395156+0000","last_peered":"2024-09-21T06:10:07.395156+0000","last_clean":"2024-09-21T06:10:07.395156+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:10:07.395156+0000","last_undegraded":"2024-09-21T06:10:07.395156+0000","last_fullsized":"2024-09-21T06:10:07.395156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395372+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:10:07.395372+0000","last_peered":"2024-09-21T06:10:07.395372+0000","last_clean":"2024-09-21T06:10:07.395372+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:10:07.395372+0000","last_undegraded":"2024-09-21T06:10:07.395372+0000","last_fullsized":"2024-09-21T06:10:07.395372+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399041+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:10:07.399041+0000","last_peered":"2024-09-21T06:10:07.399041+0000","last_clean":"2024-09-21T06:10:07.399041+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:10:07.399041+0000","last_undegraded":"2024-09-21T06:10:07.399041+0000","last_fullsized":"2024-09-21T06:10:07.399041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396843+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:10:07.396843+0000","last_peered":"2024-09-21T06:10:07.396843+0000","last_clean":"2024-09-21T06:10:07.396843+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:10:07.396843+0000","last_undegraded":"2024-09-21T06:10:07.396843+0000","last_fullsized":"2024-09-21T06:10:07.396843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396575+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:10:07.396575+0000","last_peered":"2024-09-21T06:10:07.396575+0000","last_clean":"2024-09-21T06:10:07.396575+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:10:07.396575+0000","last_undegraded":"2024-09-21T06:10:07.396575+0000","last_fullsized":"2024-09-21T06:10:07.396575+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476291+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:10:07.476291+0000","last_peered":"2024-09-21T06:10:07.476291+0000","last_clean":"2024-09-21T06:10:07.476291+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:10:07.476291+0000","last_undegraded":"2024-09-21T06:10:07.476291+0000","last_fullsized":"2024-09-21T06:10:07.476291+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.392104+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:10:07.392104+0000","last_peered":"2024-09-21T06:10:07.392104+0000","last_clean":"2024-09-21T06:10:07.392104+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:10:07.392104+0000","last_undegraded":"2024-09-21T06:10:07.392104+0000","last_fullsized":"2024-09-21T06:10:07.392104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399819+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:10:07.399819+0000","last_peered":"2024-09-21T06:10:07.399819+0000","last_clean":"2024-09-21T06:10:07.399819+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:10:07.399819+0000","last_undegraded":"2024-09-21T06:10:07.399819+0000","last_fullsized":"2024-09-21T06:10:07.399819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394421+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:10:07.394421+0000","last_peered":"2024-09-21T06:10:07.394421+0000","last_clean":"2024-09-21T06:10:07.394421+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:10:07.394421+0000","last_undegraded":"2024-09-21T06:10:07.394421+0000","last_fullsized":"2024-09-21T06:10:07.394421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391735+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:10:07.391735+0000","last_peered":"2024-09-21T06:10:07.391735+0000","last_clean":"2024-09-21T06:10:07.391735+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:10:07.391735+0000","last_undegraded":"2024-09-21T06:10:07.391735+0000","last_fullsized":"2024-09-21T06:10:07.391735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399156+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:10:07.399156+0000","last_peered":"2024-09-21T06:10:07.399156+0000","last_clean":"2024-09-21T06:10:07.399156+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:10:07.399156+0000","last_undegraded":"2024-09-21T06:10:07.399156+0000","last_fullsized":"2024-09-21T06:10:07.399156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476468+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:10:07.476468+0000","last_peered":"2024-09-21T06:10:07.476468+0000","last_clean":"2024-09-21T06:10:07.476468+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:10:07.476468+0000","last_undegraded":"2024-09-21T06:10:07.476468+0000","last_fullsized":"2024-09-21T06:10:07.476468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393583+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:10:07.393583+0000","last_peered":"2024-09-21T06:10:07.393583+0000","last_clean":"2024-09-21T06:10:07.393583+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:10:07.393583+0000","last_undegraded":"2024-09-21T06:10:07.393583+0000","last_fullsized":"2024-09-21T06:10:07.393583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398228+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:10:07.398228+0000","last_peered":"2024-09-21T06:10:07.398228+0000","last_clean":"2024-09-21T06:10:07.398228+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:10:07.398228+0000","last_undegraded":"2024-09-21T06:10:07.398228+0000","last_fullsized":"2024-09-21T06:10:07.398228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394350+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:10:07.394350+0000","last_peered":"2024-09-21T06:10:07.394350+0000","last_clean":"2024-09-21T06:10:07.394350+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:10:07.394350+0000","last_undegraded":"2024-09-21T06:10:07.394350+0000","last_fullsized":"2024-09-21T06:10:07.394350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394550+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:10:07.394550+0000","last_peered":"2024-09-21T06:10:07.394550+0000","last_clean":"2024-09-21T06:10:07.394550+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:10:07.394550+0000","last_undegraded":"2024-09-21T06:10:07.394550+0000","last_fullsized":"2024-09-21T06:10:07.394550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391807+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:10:07.391807+0000","last_peered":"2024-09-21T06:10:07.391807+0000","last_clean":"2024-09-21T06:10:07.391807+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:10:07.391807+0000","last_undegraded":"2024-09-21T06:10:07.391807+0000","last_fullsized":"2024-09-21T06:10:07.391807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397343+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:10:07.397343+0000","last_peered":"2024-09-21T06:10:07.397343+0000","last_clean":"2024-09-21T06:10:07.397343+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:10:07.397343+0000","last_undegraded":"2024-09-21T06:10:07.397343+0000","last_fullsized":"2024-09-21T06:10:07.397343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403778+0000","last_change":"2024-09-21T06:10:07.403662+0000","last_active":"2024-09-21T06:10:07.403778+0000","last_peered":"2024-09-21T06:10:07.403778+0000","last_clean":"2024-09-21T06:10:07.403778+0000","last_became_active":"2024-09-21T06:10:07.403220+0000","last_became_peered":"2024-09-21T06:10:07.403220+0000","last_unstale":"2024-09-21T06:10:07.403778+0000","last_undegraded":"2024-09-21T06:10:07.403778+0000","last_fullsized":"2024-09-21T06:10:07.403778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:42.670948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394118+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:10:07.394118+0000","last_peered":"2024-09-21T06:10:07.394118+0000","last_clean":"2024-09-21T06:10:07.394118+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:10:07.394118+0000","last_undegraded":"2024-09-21T06:10:07.394118+0000","last_fullsized":"2024-09-21T06:10:07.394118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394240+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:10:07.394240+0000","last_peered":"2024-09-21T06:10:07.394240+0000","last_clean":"2024-09-21T06:10:07.394240+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:10:07.394240+0000","last_undegraded":"2024-09-21T06:10:07.394240+0000","last_fullsized":"2024-09-21T06:10:07.394240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391629+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:10:07.391629+0000","last_peered":"2024-09-21T06:10:07.391629+0000","last_clean":"2024-09-21T06:10:07.391629+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:10:07.391629+0000","last_undegraded":"2024-09-21T06:10:07.391629+0000","last_fullsized":"2024-09-21T06:10:07.391629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399813+0000","last_change":"2024-09-21T06:10:07.399633+0000","last_active":"2024-09-21T06:10:07.399813+0000","last_peered":"2024-09-21T06:10:07.399813+0000","last_clean":"2024-09-21T06:10:07.399813+0000","last_became_active":"2024-09-21T06:10:07.399200+0000","last_became_peered":"2024-09-21T06:10:07.399200+0000","last_unstale":"2024-09-21T06:10:07.399813+0000","last_undegraded":"2024-09-21T06:10:07.399813+0000","last_fullsized":"2024-09-21T06:10:07.399813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:16.830371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398713+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:10:07.398713+0000","last_peered":"2024-09-21T06:10:07.398713+0000","last_clean":"2024-09-21T06:10:07.398713+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:10:07.398713+0000","last_undegraded":"2024-09-21T06:10:07.398713+0000","last_fullsized":"2024-09-21T06:10:07.398713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398466+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:10:07.398466+0000","last_peered":"2024-09-21T06:10:07.398466+0000","last_clean":"2024-09-21T06:10:07.398466+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:10:07.398466+0000","last_undegraded":"2024-09-21T06:10:07.398466+0000","last_fullsized":"2024-09-21T06:10:07.398466+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587183+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:10:07.587183+0000","last_peered":"2024-09-21T06:10:07.587183+0000","last_clean":"2024-09-21T06:10:07.587183+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:10:07.587183+0000","last_undegraded":"2024-09-21T06:10:07.587183+0000","last_fullsized":"2024-09-21T06:10:07.587183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396798+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:10:07.396798+0000","last_peered":"2024-09-21T06:10:07.396798+0000","last_clean":"2024-09-21T06:10:07.396798+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:10:07.396798+0000","last_undegraded":"2024-09-21T06:10:07.396798+0000","last_fullsized":"2024-09-21T06:10:07.396798+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393550+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:10:07.393550+0000","last_peered":"2024-09-21T06:10:07.393550+0000","last_clean":"2024-09-21T06:10:07.393550+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:10:07.393550+0000","last_undegraded":"2024-09-21T06:10:07.393550+0000","last_fullsized":"2024-09-21T06:10:07.393550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398807+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:10:07.398807+0000","last_peered":"2024-09-21T06:10:07.398807+0000","last_clean":"2024-09-21T06:10:07.398807+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:10:07.398807+0000","last_undegraded":"2024-09-21T06:10:07.398807+0000","last_fullsized":"2024-09-21T06:10:07.398807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397914+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:10:07.397914+0000","last_peered":"2024-09-21T06:10:07.397914+0000","last_clean":"2024-09-21T06:10:07.397914+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:10:07.397914+0000","last_undegraded":"2024-09-21T06:10:07.397914+0000","last_fullsized":"2024-09-21T06:10:07.397914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587232+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:10:07.587232+0000","last_peered":"2024-09-21T06:10:07.587232+0000","last_clean":"2024-09-21T06:10:07.587232+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:10:07.587232+0000","last_undegraded":"2024-09-21T06:10:07.587232+0000","last_fullsized":"2024-09-21T06:10:07.587232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476696+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:10:07.476696+0000","last_peered":"2024-09-21T06:10:07.476696+0000","last_clean":"2024-09-21T06:10:07.476696+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:10:07.476696+0000","last_undegraded":"2024-09-21T06:10:07.476696+0000","last_fullsized":"2024-09-21T06:10:07.476696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395089+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:10:07.395089+0000","last_peered":"2024-09-21T06:10:07.395089+0000","last_clean":"2024-09-21T06:10:07.395089+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:10:07.395089+0000","last_undegraded":"2024-09-21T06:10:07.395089+0000","last_fullsized":"2024-09-21T06:10:07.395089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398088+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:10:07.398088+0000","last_peered":"2024-09-21T06:10:07.398088+0000","last_clean":"2024-09-21T06:10:07.398088+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:10:07.398088+0000","last_undegraded":"2024-09-21T06:10:07.398088+0000","last_fullsized":"2024-09-21T06:10:07.398088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399080+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:10:07.399080+0000","last_peered":"2024-09-21T06:10:07.399080+0000","last_clean":"2024-09-21T06:10:07.399080+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:10:07.399080+0000","last_undegraded":"2024-09-21T06:10:07.399080+0000","last_fullsized":"2024-09-21T06:10:07.399080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476411+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:10:07.476411+0000","last_peered":"2024-09-21T06:10:07.476411+0000","last_clean":"2024-09-21T06:10:07.476411+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:10:07.476411+0000","last_undegraded":"2024-09-21T06:10:07.476411+0000","last_fullsized":"2024-09-21T06:10:07.476411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:39.410504+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:10:39.410504+0000","last_peered":"2024-09-21T06:10:39.410504+0000","last_clean":"2024-09-21T06:10:39.410504+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:10:39.410504+0000","last_undegraded":"2024-09-21T06:10:39.410504+0000","last_fullsized":"2024-09-21T06:10:39.410504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66957,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":121,"num_read_kb":125,"num_write":27,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395465+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:10:07.395465+0000","last_peered":"2024-09-21T06:10:07.395465+0000","last_clean":"2024-09-21T06:10:07.395465+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:10:07.395465+0000","last_undegraded":"2024-09-21T06:10:07.395465+0000","last_fullsized":"2024-09-21T06:10:07.395465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394811+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:10:07.394811+0000","last_peered":"2024-09-21T06:10:07.394811+0000","last_clean":"2024-09-21T06:10:07.394811+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:10:07.394811+0000","last_undegraded":"2024-09-21T06:10:07.394811+0000","last_fullsized":"2024-09-21T06:10:07.394811+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397834+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:10:07.397834+0000","last_peered":"2024-09-21T06:10:07.397834+0000","last_clean":"2024-09-21T06:10:07.397834+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:10:07.397834+0000","last_undegraded":"2024-09-21T06:10:07.397834+0000","last_fullsized":"2024-09-21T06:10:07.397834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398372+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:10:07.398372+0000","last_peered":"2024-09-21T06:10:07.398372+0000","last_clean":"2024-09-21T06:10:07.398372+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:10:07.398372+0000","last_undegraded":"2024-09-21T06:10:07.398372+0000","last_fullsized":"2024-09-21T06:10:07.398372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401281+0000","last_change":"2024-09-21T06:10:07.401185+0000","last_active":"2024-09-21T06:10:07.401281+0000","last_peered":"2024-09-21T06:10:07.401281+0000","last_clean":"2024-09-21T06:10:07.401281+0000","last_became_active":"2024-09-21T06:10:07.400789+0000","last_became_peered":"2024-09-21T06:10:07.400789+0000","last_unstale":"2024-09-21T06:10:07.401281+0000","last_undegraded":"2024-09-21T06:10:07.401281+0000","last_fullsized":"2024-09-21T06:10:07.401281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:03.201239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394895+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:10:07.394895+0000","last_peered":"2024-09-21T06:10:07.394895+0000","last_clean":"2024-09-21T06:10:07.394895+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:10:07.394895+0000","last_undegraded":"2024-09-21T06:10:07.394895+0000","last_fullsized":"2024-09-21T06:10:07.394895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.588324+0000","last_change":"2024-09-21T06:10:07.588201+0000","last_active":"2024-09-21T06:10:07.588324+0000","last_peered":"2024-09-21T06:10:07.588324+0000","last_clean":"2024-09-21T06:10:07.588324+0000","last_became_active":"2024-09-21T06:10:07.587788+0000","last_became_peered":"2024-09-21T06:10:07.587788+0000","last_unstale":"2024-09-21T06:10:07.588324+0000","last_undegraded":"2024-09-21T06:10:07.588324+0000","last_fullsized":"2024-09-21T06:10:07.588324+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:16.617435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394307+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:10:07.394307+0000","last_peered":"2024-09-21T06:10:07.394307+0000","last_clean":"2024-09-21T06:10:07.394307+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:10:07.394307+0000","last_undegraded":"2024-09-21T06:10:07.394307+0000","last_fullsized":"2024-09-21T06:10:07.394307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.478379+0000","last_change":"2024-09-21T06:10:07.478289+0000","last_active":"2024-09-21T06:10:07.478379+0000","last_peered":"2024-09-21T06:10:07.478379+0000","last_clean":"2024-09-21T06:10:07.478379+0000","last_became_active":"2024-09-21T06:10:07.477828+0000","last_became_peered":"2024-09-21T06:10:07.477828+0000","last_unstale":"2024-09-21T06:10:07.478379+0000","last_undegraded":"2024-09-21T06:10:07.478379+0000","last_fullsized":"2024-09-21T06:10:07.478379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.721612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395262+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:10:07.395262+0000","last_peered":"2024-09-21T06:10:07.395262+0000","last_clean":"2024-09-21T06:10:07.395262+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:10:07.395262+0000","last_undegraded":"2024-09-21T06:10:07.395262+0000","last_fullsized":"2024-09-21T06:10:07.395262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391874+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:10:07.391874+0000","last_peered":"2024-09-21T06:10:07.391874+0000","last_clean":"2024-09-21T06:10:07.391874+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:10:07.391874+0000","last_undegraded":"2024-09-21T06:10:07.391874+0000","last_fullsized":"2024-09-21T06:10:07.391874+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406063+0000","last_change":"2024-09-21T06:10:07.405946+0000","last_active":"2024-09-21T06:10:07.406063+0000","last_peered":"2024-09-21T06:10:07.406063+0000","last_clean":"2024-09-21T06:10:07.406063+0000","last_became_active":"2024-09-21T06:10:07.401620+0000","last_became_peered":"2024-09-21T06:10:07.401620+0000","last_unstale":"2024-09-21T06:10:07.406063+0000","last_undegraded":"2024-09-21T06:10:07.406063+0000","last_fullsized":"2024-09-21T06:10:07.406063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:17.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395217+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:10:07.395217+0000","last_peered":"2024-09-21T06:10:07.395217+0000","last_clean":"2024-09-21T06:10:07.395217+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:10:07.395217+0000","last_undegraded":"2024-09-21T06:10:07.395217+0000","last_fullsized":"2024-09-21T06:10:07.395217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399215+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:10:07.399215+0000","last_peered":"2024-09-21T06:10:07.399215+0000","last_clean":"2024-09-21T06:10:07.399215+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:10:07.399215+0000","last_undegraded":"2024-09-21T06:10:07.399215+0000","last_fullsized":"2024-09-21T06:10:07.399215+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406991+0000","last_change":"2024-09-21T06:10:07.406892+0000","last_active":"2024-09-21T06:10:07.406991+0000","last_peered":"2024-09-21T06:10:07.406991+0000","last_clean":"2024-09-21T06:10:07.406991+0000","last_became_active":"2024-09-21T06:10:07.402546+0000","last_became_peered":"2024-09-21T06:10:07.402546+0000","last_unstale":"2024-09-21T06:10:07.406991+0000","last_undegraded":"2024-09-21T06:10:07.406991+0000","last_fullsized":"2024-09-21T06:10:07.406991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.081510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394662+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:10:07.394662+0000","last_peered":"2024-09-21T06:10:07.394662+0000","last_clean":"2024-09-21T06:10:07.394662+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:10:07.394662+0000","last_undegraded":"2024-09-21T06:10:07.394662+0000","last_fullsized":"2024-09-21T06:10:07.394662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395205+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:10:07.395205+0000","last_peered":"2024-09-21T06:10:07.395205+0000","last_clean":"2024-09-21T06:10:07.395205+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:10:07.395205+0000","last_undegraded":"2024-09-21T06:10:07.395205+0000","last_fullsized":"2024-09-21T06:10:07.395205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395664+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:10:07.395664+0000","last_peered":"2024-09-21T06:10:07.395664+0000","last_clean":"2024-09-21T06:10:07.395664+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:10:07.395664+0000","last_undegraded":"2024-09-21T06:10:07.395664+0000","last_fullsized":"2024-09-21T06:10:07.395664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395073+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:10:07.395073+0000","last_peered":"2024-09-21T06:10:07.395073+0000","last_clean":"2024-09-21T06:10:07.395073+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:10:07.395073+0000","last_undegraded":"2024-09-21T06:10:07.395073+0000","last_fullsized":"2024-09-21T06:10:07.395073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393965+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:10:07.393965+0000","last_peered":"2024-09-21T06:10:07.393965+0000","last_clean":"2024-09-21T06:10:07.393965+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:10:07.393965+0000","last_undegraded":"2024-09-21T06:10:07.393965+0000","last_fullsized":"2024-09-21T06:10:07.393965+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394546+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:10:07.394546+0000","last_peered":"2024-09-21T06:10:07.394546+0000","last_clean":"2024-09-21T06:10:07.394546+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:10:07.394546+0000","last_undegraded":"2024-09-21T06:10:07.394546+0000","last_fullsized":"2024-09-21T06:10:07.394546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396702+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:10:07.396702+0000","last_peered":"2024-09-21T06:10:07.396702+0000","last_clean":"2024-09-21T06:10:07.396702+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:10:07.396702+0000","last_undegraded":"2024-09-21T06:10:07.396702+0000","last_fullsized":"2024-09-21T06:10:07.396702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396950+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:10:07.396950+0000","last_peered":"2024-09-21T06:10:07.396950+0000","last_clean":"2024-09-21T06:10:07.396950+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:10:07.396950+0000","last_undegraded":"2024-09-21T06:10:07.396950+0000","last_fullsized":"2024-09-21T06:10:07.396950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399104+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:10:07.399104+0000","last_peered":"2024-09-21T06:10:07.399104+0000","last_clean":"2024-09-21T06:10:07.399104+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:10:07.399104+0000","last_undegraded":"2024-09-21T06:10:07.399104+0000","last_fullsized":"2024-09-21T06:10:07.399104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394998+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:10:07.394998+0000","last_peered":"2024-09-21T06:10:07.394998+0000","last_clean":"2024-09-21T06:10:07.394998+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:10:07.394998+0000","last_undegraded":"2024-09-21T06:10:07.394998+0000","last_fullsized":"2024-09-21T06:10:07.394998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.586879+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:10:07.586879+0000","last_peered":"2024-09-21T06:10:07.586879+0000","last_clean":"2024-09-21T06:10:07.586879+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:10:07.586879+0000","last_undegraded":"2024-09-21T06:10:07.586879+0000","last_fullsized":"2024-09-21T06:10:07.586879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401748+0000","last_change":"2024-09-21T06:10:07.401640+0000","last_active":"2024-09-21T06:10:07.401748+0000","last_peered":"2024-09-21T06:10:07.401748+0000","last_clean":"2024-09-21T06:10:07.401748+0000","last_became_active":"2024-09-21T06:10:07.401242+0000","last_became_peered":"2024-09-21T06:10:07.401242+0000","last_unstale":"2024-09-21T06:10:07.401748+0000","last_undegraded":"2024-09-21T06:10:07.401748+0000","last_fullsized":"2024-09-21T06:10:07.401748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:25.335694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.586794+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:10:07.586794+0000","last_peered":"2024-09-21T06:10:07.586794+0000","last_clean":"2024-09-21T06:10:07.586794+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:10:07.586794+0000","last_undegraded":"2024-09-21T06:10:07.586794+0000","last_fullsized":"2024-09-21T06:10:07.586794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395763+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:10:07.395763+0000","last_peered":"2024-09-21T06:10:07.395763+0000","last_clean":"2024-09-21T06:10:07.395763+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:10:07.395763+0000","last_undegraded":"2024-09-21T06:10:07.395763+0000","last_fullsized":"2024-09-21T06:10:07.395763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397447+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:10:07.397447+0000","last_peered":"2024-09-21T06:10:07.397447+0000","last_clean":"2024-09-21T06:10:07.397447+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:10:07.397447+0000","last_undegraded":"2024-09-21T06:10:07.397447+0000","last_fullsized":"2024-09-21T06:10:07.397447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399941+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:10:07.399941+0000","last_peered":"2024-09-21T06:10:07.399941+0000","last_clean":"2024-09-21T06:10:07.399941+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:10:07.399941+0000","last_undegraded":"2024-09-21T06:10:07.399941+0000","last_fullsized":"2024-09-21T06:10:07.399941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394452+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:10:07.394452+0000","last_peered":"2024-09-21T06:10:07.394452+0000","last_clean":"2024-09-21T06:10:07.394452+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:10:07.394452+0000","last_undegraded":"2024-09-21T06:10:07.394452+0000","last_fullsized":"2024-09-21T06:10:07.394452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395509+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:10:07.395509+0000","last_peered":"2024-09-21T06:10:07.395509+0000","last_clean":"2024-09-21T06:10:07.395509+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:10:07.395509+0000","last_undegraded":"2024-09-21T06:10:07.395509+0000","last_fullsized":"2024-09-21T06:10:07.395509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397556+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:10:07.397556+0000","last_peered":"2024-09-21T06:10:07.397556+0000","last_clean":"2024-09-21T06:10:07.397556+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:10:07.397556+0000","last_undegraded":"2024-09-21T06:10:07.397556+0000","last_fullsized":"2024-09-21T06:10:07.397556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476406+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:10:07.476406+0000","last_peered":"2024-09-21T06:10:07.476406+0000","last_clean":"2024-09-21T06:10:07.476406+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:10:07.476406+0000","last_undegraded":"2024-09-21T06:10:07.476406+0000","last_fullsized":"2024-09-21T06:10:07.476406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394434+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:10:07.394434+0000","last_peered":"2024-09-21T06:10:07.394434+0000","last_clean":"2024-09-21T06:10:07.394434+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:10:07.394434+0000","last_undegraded":"2024-09-21T06:10:07.394434+0000","last_fullsized":"2024-09-21T06:10:07.394434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393700+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:10:07.393700+0000","last_peered":"2024-09-21T06:10:07.393700+0000","last_clean":"2024-09-21T06:10:07.393700+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:10:07.393700+0000","last_undegraded":"2024-09-21T06:10:07.393700+0000","last_fullsized":"2024-09-21T06:10:07.393700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.391704+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:10:07.391704+0000","last_peered":"2024-09-21T06:10:07.391704+0000","last_clean":"2024-09-21T06:10:07.391704+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:10:07.391704+0000","last_undegraded":"2024-09-21T06:10:07.391704+0000","last_fullsized":"2024-09-21T06:10:07.391704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394466+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:10:07.394466+0000","last_peered":"2024-09-21T06:10:07.394466+0000","last_clean":"2024-09-21T06:10:07.394466+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:10:07.394466+0000","last_undegraded":"2024-09-21T06:10:07.394466+0000","last_fullsized":"2024-09-21T06:10:07.394466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393751+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:10:07.393751+0000","last_peered":"2024-09-21T06:10:07.393751+0000","last_clean":"2024-09-21T06:10:07.393751+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:10:07.393751+0000","last_undegraded":"2024-09-21T06:10:07.393751+0000","last_fullsized":"2024-09-21T06:10:07.393751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403122+0000","last_change":"2024-09-21T06:10:07.402379+0000","last_active":"2024-09-21T06:10:07.403122+0000","last_peered":"2024-09-21T06:10:07.403122+0000","last_clean":"2024-09-21T06:10:07.403122+0000","last_became_active":"2024-09-21T06:10:07.396828+0000","last_became_peered":"2024-09-21T06:10:07.396828+0000","last_unstale":"2024-09-21T06:10:07.403122+0000","last_undegraded":"2024-09-21T06:10:07.403122+0000","last_fullsized":"2024-09-21T06:10:07.403122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:26.181343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394763+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:10:07.394763+0000","last_peered":"2024-09-21T06:10:07.394763+0000","last_clean":"2024-09-21T06:10:07.394763+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:10:07.394763+0000","last_undegraded":"2024-09-21T06:10:07.394763+0000","last_fullsized":"2024-09-21T06:10:07.394763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398961+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:10:07.398961+0000","last_peered":"2024-09-21T06:10:07.398961+0000","last_clean":"2024-09-21T06:10:07.398961+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:10:07.398961+0000","last_undegraded":"2024-09-21T06:10:07.398961+0000","last_fullsized":"2024-09-21T06:10:07.398961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395391+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:10:07.395391+0000","last_peered":"2024-09-21T06:10:07.395391+0000","last_clean":"2024-09-21T06:10:07.395391+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:10:07.395391+0000","last_undegraded":"2024-09-21T06:10:07.395391+0000","last_fullsized":"2024-09-21T06:10:07.395391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476438+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:10:07.476438+0000","last_peered":"2024-09-21T06:10:07.476438+0000","last_clean":"2024-09-21T06:10:07.476438+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:10:07.476438+0000","last_undegraded":"2024-09-21T06:10:07.476438+0000","last_fullsized":"2024-09-21T06:10:07.476438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395635+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:10:07.395635+0000","last_peered":"2024-09-21T06:10:07.395635+0000","last_clean":"2024-09-21T06:10:07.395635+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:10:07.395635+0000","last_undegraded":"2024-09-21T06:10:07.395635+0000","last_fullsized":"2024-09-21T06:10:07.395635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.397975+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:10:07.397975+0000","last_peered":"2024-09-21T06:10:07.397975+0000","last_clean":"2024-09-21T06:10:07.397975+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:10:07.397975+0000","last_undegraded":"2024-09-21T06:10:07.397975+0000","last_fullsized":"2024-09-21T06:10:07.397975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394530+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:10:07.394530+0000","last_peered":"2024-09-21T06:10:07.394530+0000","last_clean":"2024-09-21T06:10:07.394530+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:10:07.394530+0000","last_undegraded":"2024-09-21T06:10:07.394530+0000","last_fullsized":"2024-09-21T06:10:07.394530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394357+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:10:07.394357+0000","last_peered":"2024-09-21T06:10:07.394357+0000","last_clean":"2024-09-21T06:10:07.394357+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:10:07.394357+0000","last_undegraded":"2024-09-21T06:10:07.394357+0000","last_fullsized":"2024-09-21T06:10:07.394357+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399889+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:10:07.399889+0000","last_peered":"2024-09-21T06:10:07.399889+0000","last_clean":"2024-09-21T06:10:07.399889+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:10:07.399889+0000","last_undegraded":"2024-09-21T06:10:07.399889+0000","last_fullsized":"2024-09-21T06:10:07.399889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67755,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":125,"num_read_kb":129,"num_write":83,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":203265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":839680,"data_stored":415673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32684,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712468,"statfs":{"total":95995035648,"available":95961567232,"internally_reserved":0,"allocated":1527808,"data_stored":1073309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":416372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266729,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":847872,"data_stored":416277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712556,"statfs":{"total":95995035648,"available":95961657344,"internally_reserved":0,"allocated":1441792,"data_stored":1006740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":415790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1449984,"data_stored":1006340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":415695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":415894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378695,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40412,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704740,"statfs":{"total":95995035648,"available":95953653760,"internally_reserved":0,"allocated":921600,"data_stored":482664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":917504,"data_stored":482630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":843776,"data_stored":415763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:10:42.874 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:10:42.875 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T06:10:42.875 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T06:10:42.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T06:10:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:43.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:42 smithi031 ceph-mon[31237]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:43.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:42 smithi031 ceph-mon[31237]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:43.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:42 smithi110 ceph-mon[34550]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:43.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:42 smithi110 ceph-mon[34550]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:43.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:42 smithi107 ceph-mon[34380]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:43.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:42 smithi107 ceph-mon[34380]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:10:43.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 -- 172.21.15.31:0/175955331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834104a30 msgr2=0x7ff834104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 --2- 172.21.15.31:0/175955331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834104a30 0x7ff834104e40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff824009900 tx=0x7ff82402a410 comp rx=0 tx=0).stop 2024-09-21T06:10:43.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 -- 172.21.15.31:0/175955331 shutdown_connections 2024-09-21T06:10:43.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 --2- 172.21.15.31:0/175955331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 0x7ff834110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 --2- 172.21.15.31:0/175955331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834105380 0x7ff83410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 --2- 172.21.15.31:0/175955331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834104a30 0x7ff834104e40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 -- 172.21.15.31:0/175955331 >> 172.21.15.31:0/175955331 conn(0x7ff8340797b0 msgr2=0x7ff834079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 -- 172.21.15.31:0/175955331 shutdown_connections 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 -- 172.21.15.31:0/175955331 wait complete. 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.610+0000 7ff839bb5640 1 Processor -- start 2024-09-21T06:10:43.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 -- start start 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834104a30 0x7ff83419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 0x7ff83419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 0x7ff83419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff834113ac0 con 0x7ff834105380 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff834113940 con 0x7ff834104a30 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff839bb5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff834113c40 con 0x7ff83410de60 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff832ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 0x7ff83419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff832ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 0x7ff83419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50324/0 (socket says 172.21.15.31:50324) 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff832ffd640 1 -- 172.21.15.31:0/34650715 learned_addr learned my addr 172.21.15.31:0/34650715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff8337fe640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834104a30 0x7ff83419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff833fff640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 0x7ff83419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.611+0000 7ff832ffd640 1 -- 172.21.15.31:0/34650715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 msgr2=0x7ff83419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff832ffd640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 0x7ff83419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff832ffd640 1 -- 172.21.15.31:0/34650715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834104a30 msgr2=0x7ff83419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff832ffd640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834104a30 0x7ff83419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff832ffd640 1 -- 172.21.15.31:0/34650715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8341a03e0 con 0x7ff834105380 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff8337fe640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834104a30 0x7ff83419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff833fff640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 0x7ff83419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff832ffd640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 0x7ff83419aa10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff82800d950 tx=0x7ff82800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828017070 con 0x7ff834105380 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff828009c30 con 0x7ff834105380 2024-09-21T06:10:43.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8341a06d0 con 0x7ff834105380 2024-09-21T06:10:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8280047a0 con 0x7ff834105380 2024-09-21T06:10:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.612+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8341a6e30 con 0x7ff834105380 2024-09-21T06:10:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.613+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff834104e40 con 0x7ff834105380 2024-09-21T06:10:43.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.615+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff828004020 con 0x7ff834105380 2024-09-21T06:10:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.616+0000 7ff830ff9640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff81c0b65f0 0x7ff81c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.616+0000 7ff8337fe640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff81c0b65f0 0x7ff81c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.616+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff8280e7790 con 0x7ff834105380 2024-09-21T06:10:43.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.617+0000 7ff8337fe640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff81c0b65f0 0x7ff81c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8240096c0 tx=0x7ff824031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:43.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.620+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff828020020 con 0x7ff834105380 2024-09-21T06:10:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.846+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff8341a7120 con 0x7ff834105380 2024-09-21T06:10:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.846+0000 7ff830ff9640 1 -- 172.21.15.31:0/34650715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff8280adde0 con 0x7ff834105380 2024-09-21T06:10:43.848 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:43.848 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T06:10:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff81c0b65f0 msgr2=0x7ff81c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff81c0b65f0 0x7ff81c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff8240096c0 tx=0x7ff824031400 comp rx=0 tx=0).stop 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 msgr2=0x7ff83419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 0x7ff83419aa10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff82800d950 tx=0x7ff82800de10 comp rx=0 tx=0).stop 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 shutdown_connections 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff83410de60 0x7ff83419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff81c0b65f0 0x7ff81c0b8b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff834105380 0x7ff83419aa10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 --2- 172.21.15.31:0/34650715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff834104a30 0x7ff83419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 >> 172.21.15.31:0/34650715 conn(0x7ff8340797b0 msgr2=0x7ff8341007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 shutdown_connections 2024-09-21T06:10:43.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:43.849+0000 7ff839bb5640 1 -- 172.21.15.31:0/34650715 wait complete. 2024-09-21T06:10:44.480 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T06:10:44.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:10:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:45.093 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:44 smithi031 ceph-mon[31237]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:45.093 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:44 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/34650715' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:10:45.111 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:44 smithi107 ceph-mon[34380]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:45.111 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:44 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/34650715' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 -- 172.21.15.31:0/879541520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc1094a0 msgr2=0x7fe0dc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 --2- 172.21.15.31:0/879541520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc1094a0 0x7fe0dc1098b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009900 tx=0x7fe0d002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 -- 172.21.15.31:0/879541520 shutdown_connections 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 --2- 172.21.15.31:0/879541520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0dc109df0 0x7fe0dc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 --2- 172.21.15.31:0/879541520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc1094a0 0x7fe0dc1098b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 --2- 172.21.15.31:0/879541520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc103b60 0x7fe0dc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 -- 172.21.15.31:0/879541520 >> 172.21.15.31:0/879541520 conn(0x7fe0dc07b850 msgr2=0x7fe0dc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 -- 172.21.15.31:0/879541520 shutdown_connections 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.183+0000 7fe0e1079640 1 -- 172.21.15.31:0/879541520 wait complete. 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.184+0000 7fe0e1079640 1 Processor -- start 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.184+0000 7fe0e1079640 1 -- start start 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0e1079640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc103b60 0x7fe0dc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0e1079640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0dc1094a0 0x7fe0dc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0e1079640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 0x7fe0dc19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0e1079640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc113b50 con 0x7fe0dc109df0 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0e1079640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1139d0 con 0x7fe0dc103b60 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0e1079640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc113cd0 con 0x7fe0dc1094a0 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 0x7fe0dc19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 0x7fe0dc19fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50336/0 (socket says 172.21.15.31:50336) 2024-09-21T06:10:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 -- 172.21.15.31:0/2201245379 learned_addr learned my addr 172.21.15.31:0/2201245379 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0dad9d640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc103b60 0x7fe0dc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0da59c640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0dc1094a0 0x7fe0dc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 -- 172.21.15.31:0/2201245379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0dc1094a0 msgr2=0x7fe0dc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0dc1094a0 0x7fe0dc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 -- 172.21.15.31:0/2201245379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc103b60 msgr2=0x7fe0dc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc103b60 0x7fe0dc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 -- 172.21.15.31:0/2201245379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1a05b0 con 0x7fe0dc109df0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0db59e640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 0x7fe0dc19fe80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400b970 tx=0x7fe0d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.185+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4019070 con 0x7fe0dc109df0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.186+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d4014440 con 0x7fe0dc109df0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.186+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1a08a0 con 0x7fe0dc109df0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.186+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40075d0 con 0x7fe0dc109df0 2024-09-21T06:10:45.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.186+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a7000 con 0x7fe0dc109df0 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.187+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc103ff0 con 0x7fe0dc109df0 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.188+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe0d401c070 con 0x7fe0dc109df0 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.189+0000 7fe0c3fff640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0c40b65f0 0x7fe0c40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.190+0000 7fe0dad9d640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0c40b65f0 0x7fe0c40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.190+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe0d40e6eb0 con 0x7fe0dc109df0 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.190+0000 7fe0dad9d640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0c40b65f0 0x7fe0c40b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009990 tx=0x7fe0cc009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.193+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0d40b18e0 con 0x7fe0dc109df0 2024-09-21T06:10:45.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:44 smithi110 ceph-mon[34550]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:45.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:44 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/34650715' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:10:45.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.418+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0dc19b770 con 0x7fe0dc109df0 2024-09-21T06:10:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.419+0000 7fe0c3fff640 1 -- 172.21.15.31:0/2201245379 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fe0dc19b770 con 0x7fe0dc109df0 2024-09-21T06:10:45.421 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:45.421 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:10:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:10:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0c40b65f0 msgr2=0x7fe0c40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0c40b65f0 0x7fe0c40b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009990 tx=0x7fe0cc009110 comp rx=0 tx=0).stop 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 msgr2=0x7fe0dc19fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 0x7fe0dc19fe80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400b970 tx=0x7fe0d400bd70 comp rx=0 tx=0).stop 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 shutdown_connections 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0c40b65f0 0x7fe0c40b8b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0dc109df0 0x7fe0dc19fe80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0dc1094a0 0x7fe0dc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 --2- 172.21.15.31:0/2201245379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0dc103b60 0x7fe0dc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 >> 172.21.15.31:0/2201245379 conn(0x7fe0dc07b850 msgr2=0x7fe0dc1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 shutdown_connections 2024-09-21T06:10:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:45.422+0000 7fe0e1079640 1 -- 172.21.15.31:0/2201245379 wait complete. 2024-09-21T06:10:46.105 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:45 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2201245379' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:46.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:10:46.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:45 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2201245379' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:46.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:45 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2201245379' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:46.403 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 -- 172.21.15.31:0/295934260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338103b60 msgr2=0x7fd338103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 --2- 172.21.15.31:0/295934260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338103b60 0x7fd338103f70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd328009900 tx=0x7fd32802a410 comp rx=0 tx=0).stop 2024-09-21T06:10:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 -- 172.21.15.31:0/295934260 shutdown_connections 2024-09-21T06:10:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 --2- 172.21.15.31:0/295934260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd338109df0 0x7fd338110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 --2- 172.21.15.31:0/295934260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3381094a0 0x7fd3381098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 --2- 172.21.15.31:0/295934260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338103b60 0x7fd338103f70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 -- 172.21.15.31:0/295934260 >> 172.21.15.31:0/295934260 conn(0x7fd33807b850 msgr2=0x7fd33807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 -- 172.21.15.31:0/295934260 shutdown_connections 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 -- 172.21.15.31:0/295934260 wait complete. 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.893+0000 7fd33db25640 1 Processor -- start 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 -- start start 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd338103b60 0x7fd33819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3381094a0 0x7fd33819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 0x7fd33819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd338113ab0 con 0x7fd338109df0 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd338113930 con 0x7fd338103b60 2024-09-21T06:10:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd33db25640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd338113c30 con 0x7fd3381094a0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 0x7fd33819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 0x7fd33819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50348/0 (socket says 172.21.15.31:50348) 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 -- 172.21.15.31:0/2882388640 learned_addr learned my addr 172.21.15.31:0/2882388640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd336ffd640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3381094a0 0x7fd33819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd3377fe640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd338103b60 0x7fd33819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 -- 172.21.15.31:0/2882388640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3381094a0 msgr2=0x7fd33819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3381094a0 0x7fd33819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 -- 172.21.15.31:0/2882388640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd338103b60 msgr2=0x7fd33819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd338103b60 0x7fd33819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.894+0000 7fd337fff640 1 -- 172.21.15.31:0/2882388640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3381a0410 con 0x7fd338109df0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.895+0000 7fd337fff640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 0x7fd33819fce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd33000cad0 tx=0x7fd33000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.895+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33000ae20 con 0x7fd338109df0 2024-09-21T06:10:46.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.895+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd330005020 con 0x7fd338109df0 2024-09-21T06:10:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.895+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33000f6b0 con 0x7fd338109df0 2024-09-21T06:10:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.895+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3381a0700 con 0x7fd338109df0 2024-09-21T06:10:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.895+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3381a6e60 con 0x7fd338109df0 2024-09-21T06:10:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.896+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd338103f70 con 0x7fd338109df0 2024-09-21T06:10:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.897+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd3300051d0 con 0x7fd338109df0 2024-09-21T06:10:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.898+0000 7fd334ff9640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3200b65f0 0x7fd3200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.898+0000 7fd3377fe640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3200b65f0 0x7fd3200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.898+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd3300e7520 con 0x7fd338109df0 2024-09-21T06:10:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.899+0000 7fd3377fe640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3200b65f0 0x7fd3200b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd328031440 tx=0x7fd328009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:46.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:46.903+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd330014030 con 0x7fd338109df0 2024-09-21T06:10:47.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.131+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd33810b630 con 0x7fd338109df0 2024-09-21T06:10:47.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:46 smithi031 ceph-mon[31237]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:47.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.132+0000 7fd334ff9640 1 -- 172.21.15.31:0/2882388640 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fd3300b1ef0 con 0x7fd338109df0 2024-09-21T06:10:47.134 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:47.134 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:10:47.134 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:10:47.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3200b65f0 msgr2=0x7fd3200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:47.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3200b65f0 0x7fd3200b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd328031440 tx=0x7fd328009560 comp rx=0 tx=0).stop 2024-09-21T06:10:47.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 msgr2=0x7fd33819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:47.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 0x7fd33819fce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd33000cad0 tx=0x7fd33000cf90 comp rx=0 tx=0).stop 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 shutdown_connections 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3200b65f0 0x7fd3200b8b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd338109df0 0x7fd33819fce0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3381094a0 0x7fd33819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 --2- 172.21.15.31:0/2882388640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd338103b60 0x7fd33819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 >> 172.21.15.31:0/2882388640 conn(0x7fd33807b850 msgr2=0x7fd338107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 shutdown_connections 2024-09-21T06:10:47.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:47.134+0000 7fd33db25640 1 -- 172.21.15.31:0/2882388640 wait complete. 2024-09-21T06:10:47.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:46 smithi110 ceph-mon[34550]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:47.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:46 smithi107 ceph-mon[34380]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:47.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:10:48.093 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:48.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:47 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2882388640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:48.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:47 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2882388640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:48.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:47 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2882388640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 -- 172.21.15.31:0/1355870661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f78108cf0 msgr2=0x7f6f7810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 --2- 172.21.15.31:0/1355870661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f78108cf0 0x7f6f7810f520 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009ca0 tx=0x7f6f7002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 -- 172.21.15.31:0/1355870661 shutdown_connections 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 --2- 172.21.15.31:0/1355870661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f78108cf0 0x7f6f7810f520 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 --2- 172.21.15.31:0/1355870661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f781083a0 0x7f6f781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 --2- 172.21.15.31:0/1355870661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f78102a60 0x7f6f78102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 -- 172.21.15.31:0/1355870661 >> 172.21.15.31:0/1355870661 conn(0x7f6f78079730 msgr2=0x7f6f78079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 -- 172.21.15.31:0/1355870661 shutdown_connections 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.555+0000 7f6f8011b640 1 -- 172.21.15.31:0/1355870661 wait complete. 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 Processor -- start 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 -- start start 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f78102a60 0x7f6f78199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 0x7f6f781998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f78108cf0 0x7f6f7819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78112960 con 0x7f6f781083a0 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781127e0 con 0x7f6f78108cf0 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f8011b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78112ae0 con 0x7f6f78102a60 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f7d6b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 0x7f6f781998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.556+0000 7f6f7d6b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 0x7f6f781998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50356/0 (socket says 172.21.15.31:50356) 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 -- 172.21.15.31:0/2894891995 learned_addr learned my addr 172.21.15.31:0/2894891995 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7e6b8640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f78108cf0 0x7f6f7819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7deb7640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f78102a60 0x7f6f78199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 -- 172.21.15.31:0/2894891995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f78102a60 msgr2=0x7f6f78199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f78102a60 0x7f6f78199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 -- 172.21.15.31:0/2894891995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f78108cf0 msgr2=0x7f6f7819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f78108cf0 0x7f6f7819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 -- 172.21.15.31:0/2894891995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7819f2c0 con 0x7f6f781083a0 2024-09-21T06:10:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7deb7640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f78102a60 0x7f6f78199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f7d6b6640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 0x7f6f781998f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800d740 tx=0x7f6f6800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68017070 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f6800de50 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7819f5b0 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004ce0 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.557+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f781a5d10 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.558+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f78102e70 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.560+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6f68004e80 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.561+0000 7f6f6effd640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f640b65f0 0x7f6f640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.561+0000 7f6f7deb7640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f640b65f0 0x7f6f640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.561+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6f680e74a0 con 0x7f6f781083a0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.561+0000 7f6f7deb7640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f640b65f0 0x7f6f640b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009b40 tx=0x7f6f74005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.563+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6f680ada90 con 0x7f6f781083a0 2024-09-21T06:10:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.751+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f7819a480 con 0x7f6f781083a0 2024-09-21T06:10:48.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.752+0000 7f6f6effd640 1 -- 172.21.15.31:0/2894891995 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23089 (secure 0 0 0) 0x7f6f7819a480 con 0x7f6f781083a0 2024-09-21T06:10:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:48.755 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:10:07.387838+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:10:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f640b65f0 msgr2=0x7f6f640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f640b65f0 0x7f6f640b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009b40 tx=0x7f6f74005e50 comp rx=0 tx=0).stop 2024-09-21T06:10:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 msgr2=0x7f6f781998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 0x7f6f781998f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800d740 tx=0x7f6f6800dc00 comp rx=0 tx=0).stop 2024-09-21T06:10:48.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 shutdown_connections 2024-09-21T06:10:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f78108cf0 0x7f6f7819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f640b65f0 0x7f6f640b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f781083a0 0x7f6f781998f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 --2- 172.21.15.31:0/2894891995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f78102a60 0x7f6f78199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.754+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 >> 172.21.15.31:0/2894891995 conn(0x7f6f78079730 msgr2=0x7f6f781043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:48.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.755+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 shutdown_connections 2024-09-21T06:10:48.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:48.755+0000 7f6f8011b640 1 -- 172.21.15.31:0/2894891995 wait complete. 2024-09-21T06:10:49.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:48 smithi031 ceph-mon[31237]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:49.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:48 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2894891995' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:48 smithi110 ceph-mon[34550]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:48 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2894891995' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:48 smithi107 ceph-mon[34380]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:48 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2894891995' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:10:49.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:10:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 -- 172.21.15.31:0/1078987969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 msgr2=0x7f6cdc10fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/1078987969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc10fcc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009900 tx=0x7f6ccc02a480 comp rx=0 tx=0).stop 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 -- 172.21.15.31:0/1078987969 shutdown_connections 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/1078987969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cdc110200 0x7f6cdc112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/1078987969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc10fcc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/1078987969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 0x7f6cdc102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.129+0000 7f6ce3edd640 1 -- 172.21.15.31:0/1078987969 >> 172.21.15.31:0/1078987969 conn(0x7f6cdc079760 msgr2=0x7f6cdc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.130+0000 7f6ce3edd640 1 -- 172.21.15.31:0/1078987969 shutdown_connections 2024-09-21T06:10:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.130+0000 7f6ce3edd640 1 -- 172.21.15.31:0/1078987969 wait complete. 2024-09-21T06:10:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.130+0000 7f6ce3edd640 1 Processor -- start 2024-09-21T06:10:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 -- start start 2024-09-21T06:10:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 0x7f6cdc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cdc110200 0x7f6cdc1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc115c70 con 0x7f6cdc102fa0 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc115af0 con 0x7f6cdc110200 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce3edd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc115df0 con 0x7f6cdc102650 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46996/0 (socket says 172.21.15.31:46996) 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 -- 172.21.15.31:0/3684494865 learned_addr learned my addr 172.21.15.31:0/3684494865 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1c79640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 0x7f6cdc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce247a640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cdc110200 0x7f6cdc1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 -- 172.21.15.31:0/3684494865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 msgr2=0x7f6cdc19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 0x7f6cdc19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 -- 172.21.15.31:0/3684494865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cdc110200 msgr2=0x7f6cdc1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cdc110200 0x7f6cdc1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1478640 1 -- 172.21.15.31:0/3684494865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc1a25b0 con 0x7f6cdc102fa0 2024-09-21T06:10:50.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.131+0000 7f6ce1c79640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 0x7f6cdc19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.132+0000 7f6ce1478640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc19cbe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005940 tx=0x7f6ccc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.132+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc03f030 con 0x7f6cdc102fa0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.132+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc003cb0 con 0x7f6cdc102fa0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.132+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1a2840 con 0x7f6cdc102fa0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.132+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc035070 con 0x7f6cdc102fa0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.132+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a9000 con 0x7f6cdc102fa0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.133+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc06c1f0 con 0x7f6cdc102fa0 2024-09-21T06:10:50.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.134+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6ccc042030 con 0x7f6cdc102fa0 2024-09-21T06:10:50.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.135+0000 7f6cd2ffd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6cc80b65f0 0x7f6cc80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:50.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.135+0000 7f6ce1c79640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6cc80b65f0 0x7f6cc80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:50.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.135+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6ccc107cb0 con 0x7f6cdc102fa0 2024-09-21T06:10:50.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.136+0000 7f6ce1c79640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6cc80b65f0 0x7f6cc80b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8005d00 tx=0x7f6cd8005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.140+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ccc0ce2a0 con 0x7f6cdc102fa0 2024-09-21T06:10:50.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.369+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cdc19d770 con 0x7f6cdc102fa0 2024-09-21T06:10:50.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.369+0000 7f6cd2ffd640 1 -- 172.21.15.31:0/3684494865 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f6ccc0d2680 con 0x7f6cdc102fa0 2024-09-21T06:10:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:10:50.372 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:10:50.372 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6cc80b65f0 msgr2=0x7f6cc80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6cc80b65f0 0x7f6cc80b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8005d00 tx=0x7f6cd8005c70 comp rx=0 tx=0).stop 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 msgr2=0x7f6cdc19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc19cbe0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005940 tx=0x7f6ccc0042f0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 shutdown_connections 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6cdc110200 0x7f6cdc1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6cc80b65f0 0x7f6cc80b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cdc102fa0 0x7f6cdc19cbe0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 --2- 172.21.15.31:0/3684494865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cdc102650 0x7f6cdc19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 >> 172.21.15.31:0/3684494865 conn(0x7f6cdc079760 msgr2=0x7f6cdc109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 shutdown_connections 2024-09-21T06:10:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:50.372+0000 7f6ce3edd640 1 -- 172.21.15.31:0/3684494865 wait complete. 2024-09-21T06:10:50.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:50 smithi031 ceph-mon[31237]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:50.959 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:50 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3684494865' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:50.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:09:36.903399+0000', 'modified': '2024-09-21T06:09:45.967301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24686': {'gid': 24686, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6837/2795950298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 2795950298}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 2795950298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24719': {'gid': 24719, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.107:6839/648775801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 648775801}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 648775801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24719, 'qdb_cluster': [24719]} 2024-09-21T06:10:50.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:10:50.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:10:51.246 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:51.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:50 smithi110 ceph-mon[34550]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:51.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:50 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3684494865' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:50 smithi107 ceph-mon[34380]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:50 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3684494865' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:10:51.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 -- 172.21.15.31:0/2056664196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190102a60 msgr2=0x7f9190102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 --2- 172.21.15.31:0/2056664196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190102a60 0x7f9190102e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002a410 comp rx=0 tx=0).stop 2024-09-21T06:10:51.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 -- 172.21.15.31:0/2056664196 shutdown_connections 2024-09-21T06:10:51.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 --2- 172.21.15.31:0/2056664196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9190108cf0 0x7f919010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 --2- 172.21.15.31:0/2056664196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91901083a0 0x7f91901087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 --2- 172.21.15.31:0/2056664196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190102a60 0x7f9190102e70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 -- 172.21.15.31:0/2056664196 >> 172.21.15.31:0/2056664196 conn(0x7f9190079730 msgr2=0x7f9190079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 -- 172.21.15.31:0/2056664196 shutdown_connections 2024-09-21T06:10:51.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.721+0000 7f9195d58640 1 -- 172.21.15.31:0/2056664196 wait complete. 2024-09-21T06:10:51.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 Processor -- start 2024-09-21T06:10:51.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 -- start start 2024-09-21T06:10:51.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9190102a60 0x7f919010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91901083a0 0x7f91901104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 0x7f9190110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9190113a80 con 0x7f9190108cf0 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9190113900 con 0x7f9190102a60 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f9195d58640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9190113c00 con 0x7f91901083a0 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.722+0000 7f918ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 0x7f9190110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 0x7f9190110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47008/0 (socket says 172.21.15.31:47008) 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 -- 172.21.15.31:0/4148803192 learned_addr learned my addr 172.21.15.31:0/4148803192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918effd640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91901083a0 0x7f91901104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918f7fe640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9190102a60 0x7f919010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 -- 172.21.15.31:0/4148803192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91901083a0 msgr2=0x7f91901104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91901083a0 0x7f91901104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 -- 172.21.15.31:0/4148803192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9190102a60 msgr2=0x7f919010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9190102a60 0x7f919010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 -- 172.21.15.31:0/4148803192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91901a87b0 con 0x7f9190108cf0 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918ffff640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 0x7f9190110a30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f918800cad0 tx=0x7f918800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918cff9640 1 -- 172.21.15.31:0/4148803192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918800ae20 con 0x7f9190108cf0 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918cff9640 1 -- 172.21.15.31:0/4148803192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188005020 con 0x7f9190108cf0 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91901a8aa0 con 0x7f9190108cf0 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f918cff9640 1 -- 172.21.15.31:0/4148803192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918800f6b0 con 0x7f9190108cf0 2024-09-21T06:10:51.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.723+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a8f90 con 0x7f9190108cf0 2024-09-21T06:10:51.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.726+0000 7f919c2cc740 1 Processor -- start 2024-09-21T06:10:51.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.726+0000 7f918cff9640 1 -- 172.21.15.31:0/4148803192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f91880051d0 con 0x7f9190108cf0 2024-09-21T06:10:51.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.726+0000 7f919c2cc740 1 -- start start 2024-09-21T06:10:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.726+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.726+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253da980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f919c2cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f125351c10 con 0x55f125033f60 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f919c2cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f125341040 con 0x55f12500b0a0 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f919c2cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f125344e50 con 0x55f125013280 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253d9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47024/0 (socket says 172.21.15.31:47024) 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 -- 172.21.15.31:0/2546507263 learned_addr learned my addr 172.21.15.31:0/2546507263 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918f7fe640 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253da980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 -- 172.21.15.31:0/2546507263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 msgr2=0x55f1253da980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253da980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 -- 172.21.15.31:0/2546507263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 msgr2=0x55f1253da440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253da440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 -- 172.21.15.31:0/2546507263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f12509f540 con 0x55f125033f60 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918cff9640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91780b65f0 0x7f91780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918effd640 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253d9f00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91840058a0 tx=0x7f918400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d39ddedaf4c676f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918f7fe640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91780b65f0 0x7f91780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.727+0000 7f918cff9640 1 -- 172.21.15.31:0/4148803192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f91880e7540 con 0x7f9190108cf0 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.728+0000 7f9175ffb640 1 -- 172.21.15.31:0/2546507263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184004030 con 0x55f125033f60 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.728+0000 7f9175ffb640 1 -- 172.21.15.31:0/2546507263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91840054d0 con 0x55f125033f60 2024-09-21T06:10:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.728+0000 7f9175ffb640 1 -- 172.21.15.31:0/2546507263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184004c00 con 0x55f125033f60 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.728+0000 7f918f7fe640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91780b65f0 0x7f91780b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f9180031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.728+0000 7f919c2cc740 1 -- 172.21.15.31:0/2546507263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 msgr2=0x55f1253d9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.728+0000 7f919c2cc740 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253d9f00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91840058a0 tx=0x7f918400b820 comp rx=0 tx=0).stop 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 -- 172.21.15.31:0/2546507263 shutdown_connections 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253da980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 --2- 172.21.15.31:0/2546507263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253d9f00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 -- 172.21.15.31:0/2546507263 >> 172.21.15.31:0/2546507263 conn(0x55f125028350 msgr2=0x55f1253d7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 -- 172.21.15.31:0/2546507263 shutdown_connections 2024-09-21T06:10:51.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 -- 172.21.15.31:0/2546507263 wait complete. 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 -- start start 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.729+0000 7f919c2cc740 1 Processor -- start 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f1251a25f0 con 0x55f125033f60 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1252dc010 con 0x55f12500b0a0 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1253e86e0 con 0x55f125013280 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f919c2cc740 10 client.0 _ll_register_callbacks cb 0x55f12533b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47038/0 (socket says 172.21.15.31:47038) 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 -- 172.21.15.31:0/3574038252 learned_addr learned my addr 172.21.15.31:0/3574038252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918f7fe640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918effd640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 -- 172.21.15.31:0/3574038252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 msgr2=0x55f1253e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 -- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 msgr2=0x55f1253e57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253e57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 -- 172.21.15.31:0/3574038252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f125309a00 con 0x55f125033f60 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918effd640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f918ffff640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f19d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f918800caa0 tx=0x7f91880e5270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c9dfa757ea1e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.730+0000 7f916effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f916effd640 1 -- 172.21.15.31:0/3574038252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918801d040 con 0x55f125033f60 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f916effd640 1 -- 172.21.15.31:0/3574038252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f91880a8e90 con 0x55f125033f60 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91840054d0 con 0x55f125033f60 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f916effd640 1 -- 172.21.15.31:0/3574038252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f91880b6eb0 con 0x55f125033f60 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 10 client.15192 fetch_fsmap learned FSMap version 19 2024-09-21T06:10:51.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f1253f4600 con 0x55f125033f60 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f916effd640 1 -- 172.21.15.31:0/3574038252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91840054d0 con 0x55f125033f60 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f916effd640 1 -- 172.21.15.31:0/3574038252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f9188004a40 con 0x55f125033f60 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 10 client.15192 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 20 client.15192 populate_metadata read hostname 'smithi031' 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124ff28e0 0x55f1253f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 4 client.15192 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.731+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f125353b60 con 0x55f124ff28e0 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.732+0000 7f918f7fe640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124ff28e0 0x55f1253f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.732+0000 7f918f7fe640 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124ff28e0 0x55f1253f8050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ff133ea498468b server_cookie=73e0658ad2d53a49 in_seq=0 out_seq=0 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.732+0000 7f916effd640 10 client.15192 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f916effd640 1 -- 172.21.15.31:0/3574038252 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f125353b60 con 0x55f124ff28e0 2024-09-21T06:10:51.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f916effd640 10 client.15192 handle_command_reply: tid=0 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 client.15192 shutdown 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 msgr2=0x55f1253f19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f19d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f918800caa0 tx=0x7f91880e5270 comp rx=0 tx=0).stop 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 shutdown_connections 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124ff28e0 0x55f1253f8050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f19d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f125013280 0x55f1253e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 --2- 172.21.15.31:0/3574038252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f12500b0a0 0x55f1253e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 >> 172.21.15.31:0/3574038252 conn(0x55f125028350 msgr2=0x55f1253dd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f916effd640 0 client.15192 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 shutdown_connections 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.733+0000 7f919c2cc740 1 -- 172.21.15.31:0/3574038252 wait complete. 2024-09-21T06:10:51.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.734+0000 7f919c2cc740 20 client.15192 trim_cache size 0 max 16384 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 Processor -- start 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 -- start start 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125013280 0x55f1253d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f1253f6390 con 0x55f125033f60 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1253f6a60 con 0x55f125013280 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f919c2cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1253f7130 con 0x55f12500b0a0 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f918ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f918ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47046/0 (socket says 172.21.15.31:47046) 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f918ffff640 1 -- 172.21.15.31:0/1348995147 learned_addr learned my addr 172.21.15.31:0/1348995147 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f918effd640 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125013280 0x55f1253d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.752+0000 7f918f7fe640 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918ffff640 1 -- 172.21.15.31:0/1348995147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 msgr2=0x55f1253e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918ffff640 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253e0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918ffff640 1 -- 172.21.15.31:0/1348995147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125013280 msgr2=0x55f1253d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918ffff640 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125013280 0x55f1253d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918ffff640 1 -- 172.21.15.31:0/1348995147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1253d2080 con 0x55f125033f60 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918effd640 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125013280 0x55f1253d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f918ffff640 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f1940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f918800a6e0 tx=0x7f91880b20f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf0fd733b5dd8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f91757fa640 1 -- 172.21.15.31:0/1348995147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918801d070 con 0x55f125033f60 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f91757fa640 1 -- 172.21.15.31:0/1348995147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880b66a0 con 0x55f125033f60 2024-09-21T06:10:51.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.753+0000 7f91757fa640 1 -- 172.21.15.31:0/1348995147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880ac450 con 0x55f125033f60 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 -- 172.21.15.31:0/1348995147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 msgr2=0x55f1253f1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f1940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f918800a6e0 tx=0x7f91880b20f0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 -- 172.21.15.31:0/1348995147 shutdown_connections 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125033f60 0x55f1253f1940 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125013280 0x55f1253d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 --2- 172.21.15.31:0/1348995147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 -- 172.21.15.31:0/1348995147 >> 172.21.15.31:0/1348995147 conn(0x55f125028350 msgr2=0x55f1253e5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 -- 172.21.15.31:0/1348995147 shutdown_connections 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 -- 172.21.15.31:0/1348995147 wait complete. 2024-09-21T06:10:51.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 -- start start 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.754+0000 7f919c2cc740 1 Processor -- start 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 0x55f1253fa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125033f60 0x55f1253fae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f1253f7ed0 con 0x55f125013280 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1253f7800 con 0x55f125033f60 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1253f85a0 con 0x55f12500b0a0 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f919c2cc740 10 client.0 _ll_register_callbacks cb 0x7f91880a9c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 0x55f1253fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 0x55f1253fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47054/0 (socket says 172.21.15.31:47054) 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 -- 172.21.15.31:0/332514830 learned_addr learned my addr 172.21.15.31:0/332514830 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918f7fe640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918ffff640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125033f60 0x55f1253fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 -- 172.21.15.31:0/332514830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 msgr2=0x55f1253fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 -- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125033f60 msgr2=0x55f1253fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125033f60 0x55f1253fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 -- 172.21.15.31:0/332514830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1253ffb00 con 0x55f125013280 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918ffff640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125033f60 0x55f1253fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f918effd640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 0x55f1253fa900 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f91840058a0 tx=0x7f9184012380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27be45735a8a984e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.755+0000 7f916effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f916effd640 1 -- 172.21.15.31:0/332514830 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918402e060 con 0x55f125013280 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f916effd640 1 -- 172.21.15.31:0/332514830 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f91840054d0 con 0x55f125013280 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91880b66a0 con 0x55f125013280 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f916effd640 1 -- 172.21.15.31:0/332514830 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f91840035f0 con 0x55f125013280 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 10 client.15198 fetch_fsmap learned FSMap version 19 2024-09-21T06:10:51.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f1254637e0 con 0x55f125013280 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f916effd640 1 -- 172.21.15.31:0/332514830 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91880b66a0 con 0x55f125013280 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f916effd640 1 -- 172.21.15.31:0/332514830 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f918401c020 con 0x55f125013280 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 10 client.15198 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 20 client.15198 populate_metadata read hostname 'smithi031' 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124fc7c90 0x55f125466e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 4 client.15198 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.756+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "status"}) -- 0x55f12534f610 con 0x55f124fc7c90 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f918f7fe640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124fc7c90 0x55f125466e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f918f7fe640 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124fc7c90 0x55f125466e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb0107af75415af server_cookie=445d7053937e7f5c in_seq=0 out_seq=0 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f916effd640 10 client.15198 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f916effd640 1 -- 172.21.15.31:0/332514830 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f12534f610 con 0x55f124fc7c90 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f916effd640 10 client.15198 handle_command_reply: tid=0 2024-09-21T06:10:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f919c2cc740 1 client.15198 shutdown 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 msgr2=0x55f1253fa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.757+0000 7f919c2cc740 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 0x55f1253fa900 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f91840058a0 tx=0x7f9184012380 comp rx=0 tx=0).stop 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 shutdown_connections 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f124fc7c90 0x55f125466e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f125033f60 0x55f1253fae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f125013280 0x55f1253fa900 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 --2- 172.21.15.31:0/332514830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f12500b0a0 0x55f1253fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 >> 172.21.15.31:0/332514830 conn(0x55f125028350 msgr2=0x55f1253d63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f916effd640 0 client.15198 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 shutdown_connections 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 1 -- 172.21.15.31:0/332514830 wait complete. 2024-09-21T06:10:51.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f919c2cc740 20 client.15198 trim_cache size 0 max 16384 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "c04f7042-77de-11ef-baf5-efdc52797490", 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24719, 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 74.475985527999995, 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 19, 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 77, 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 84.733145945000004 2024-09-21T06:10:51.765 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91780b65f0 msgr2=0x7f91780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91780b65f0 0x7f91780b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f9180031400 comp rx=0 tx=0).stop 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 msgr2=0x7f9190110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 0x7f9190110a30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f918800cad0 tx=0x7f918800cf90 comp rx=0 tx=0).stop 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 shutdown_connections 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91780b65f0 0x7f91780b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9190108cf0 0x7f9190110a30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91901083a0 0x7f91901104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 --2- 172.21.15.31:0/4148803192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9190102a60 0x7f919010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:51.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.758+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 >> 172.21.15.31:0/4148803192 conn(0x7f9190079730 msgr2=0x7f9190101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:51.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.759+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 shutdown_connections 2024-09-21T06:10:51.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:51.759+0000 7f9195d58640 1 -- 172.21.15.31:0/4148803192 wait complete. 2024-09-21T06:10:52.420 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T06:10:52.420 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:10:52.435 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:10:52.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T06:10:52.692 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:53.128 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:52 smithi031 ceph-mon[31237]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 -- 172.21.15.31:0/3027879459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b90108cf0 msgr2=0x7f8b9010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 --2- 172.21.15.31:0/3027879459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b90108cf0 0x7f8b9010f520 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009ca0 tx=0x7f8b8802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 -- 172.21.15.31:0/3027879459 shutdown_connections 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 --2- 172.21.15.31:0/3027879459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b90108cf0 0x7f8b9010f520 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 --2- 172.21.15.31:0/3027879459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b901083a0 0x7f8b901087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 --2- 172.21.15.31:0/3027879459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b90102a60 0x7f8b90102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 -- 172.21.15.31:0/3027879459 >> 172.21.15.31:0/3027879459 conn(0x7f8b90079730 msgr2=0x7f8b90079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 -- 172.21.15.31:0/3027879459 shutdown_connections 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.126+0000 7f8b96478640 1 -- 172.21.15.31:0/3027879459 wait complete. 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 Processor -- start 2024-09-21T06:10:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 -- start start 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b90102a60 0x7f8b9019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 0x7f8b9019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b90108cf0 0x7f8b901a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90112960 con 0x7f8b901083a0 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901127e0 con 0x7f8b90102a60 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.127+0000 7f8b96478640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90112ae0 con 0x7f8b90108cf0 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 0x7f8b9019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 0x7f8b9019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47082/0 (socket says 172.21.15.31:47082) 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 -- 172.21.15.31:0/4072016935 learned_addr learned my addr 172.21.15.31:0/4072016935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b94a15640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b90108cf0 0x7f8b901a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8ffff640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b90102a60 0x7f8b9019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 -- 172.21.15.31:0/4072016935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b90108cf0 msgr2=0x7f8b901a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b90108cf0 0x7f8b901a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 -- 172.21.15.31:0/4072016935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b90102a60 msgr2=0x7f8b9019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b90102a60 0x7f8b9019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 -- 172.21.15.31:0/4072016935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b901a24f0 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8f7fe640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 0x7f8b9019cbe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d740 tx=0x7f8b8400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84017070 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b84009820 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b901a27e0 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.128+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84004ce0 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.129+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b901a9000 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.129+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b90103ce0 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.131+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8b84005220 con 0x7f8b901083a0 2024-09-21T06:10:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.132+0000 7f8b8d7fa640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8b780b6620 0x7f8b780b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.132+0000 7f8b8ffff640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8b780b6620 0x7f8b780b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.132+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8b840e72b0 con 0x7f8b901083a0 2024-09-21T06:10:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.133+0000 7f8b8ffff640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8b780b6620 0x7f8b780b8b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000abe0 tx=0x7f8b8000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.136+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b840ad900 con 0x7f8b901083a0 2024-09-21T06:10:53.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:52 smithi110 ceph-mon[34550]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:53.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:52 smithi107 ceph-mon[34380]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:53.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.342+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8b9006b570 con 0x7f8b780b6620 2024-09-21T06:10:53.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.343+0000 7f8b8d7fa640 1 -- 172.21.15.31:0/4072016935 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8b9006b570 con 0x7f8b780b6620 2024-09-21T06:10:53.344 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-09-21T06:10:53.345 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-09-21T06:10:53.345 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8b780b6620 msgr2=0x7f8b780b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8b780b6620 0x7f8b780b8b30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000abe0 tx=0x7f8b8000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 msgr2=0x7f8b9019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 0x7f8b9019cbe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400d740 tx=0x7f8b8400db40 comp rx=0 tx=0).stop 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 shutdown_connections 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b90108cf0 0x7f8b901a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8b780b6620 0x7f8b780b8b30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b901083a0 0x7f8b9019cbe0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 --2- 172.21.15.31:0/4072016935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b90102a60 0x7f8b9019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:53.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 >> 172.21.15.31:0/4072016935 conn(0x7f8b90079730 msgr2=0x7f8b9006ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:53.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 shutdown_connections 2024-09-21T06:10:53.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:53.345+0000 7f8b96478640 1 -- 172.21.15.31:0/4072016935 wait complete. 2024-09-21T06:10:53.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T06:10:54.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:53 smithi031 ceph-mon[31237]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:54.148 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:54.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:53 smithi110 ceph-mon[34550]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:54.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:53 smithi107 ceph-mon[34380]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 -- 172.21.15.31:0/1566007643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8381062b0 msgr2=0x7ff8381066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/1566007643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8381062b0 0x7ff8381066c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009900 tx=0x7ff82c02a410 comp rx=0 tx=0).stop 2024-09-21T06:10:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 -- 172.21.15.31:0/1566007643 shutdown_connections 2024-09-21T06:10:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/1566007643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff838106c00 0x7ff838110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/1566007643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8381062b0 0x7ff8381066c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/1566007643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8381020c0 0x7ff8381024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 -- 172.21.15.31:0/1566007643 >> 172.21.15.31:0/1566007643 conn(0x7ff838079e40 msgr2=0x7ff83807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 -- 172.21.15.31:0/1566007643 shutdown_connections 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.583+0000 7ff83e8e2640 1 -- 172.21.15.31:0/1566007643 wait complete. 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.584+0000 7ff83e8e2640 1 Processor -- start 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.584+0000 7ff83e8e2640 1 -- start start 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8381020c0 0x7ff83819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8381062b0 0x7ff83819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 0x7ff83819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113b20 con 0x7ff838106c00 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8381139a0 con 0x7ff8381020c0 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff838113ca0 con 0x7ff8381062b0 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 0x7ff83819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 0x7ff83819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47096/0 (socket says 172.21.15.31:47096) 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 -- 172.21.15.31:0/2968001945 learned_addr learned my addr 172.21.15.31:0/2968001945 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff8377fe640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8381062b0 0x7ff83819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff837fff640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8381020c0 0x7ff83819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 -- 172.21.15.31:0/2968001945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8381062b0 msgr2=0x7ff83819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8381062b0 0x7ff83819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 -- 172.21.15.31:0/2968001945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8381020c0 msgr2=0x7ff83819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8381020c0 0x7ff83819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 -- 172.21.15.31:0/2968001945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8381a04a0 con 0x7ff838106c00 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83ce7f640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 0x7ff83819fd70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff83000eaa0 tx=0x7ff83000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830004d20 con 0x7ff838106c00 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff830005020 con 0x7ff838106c00 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8381a0790 con 0x7ff838106c00 2024-09-21T06:10:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.585+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8300116b0 con 0x7ff838106c00 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.586+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381a6ef0 con 0x7ff838106c00 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.587+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83806b860 con 0x7ff838106c00 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.588+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff8300051d0 con 0x7ff838106c00 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.589+0000 7ff8357fa640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8200b65f0 0x7ff8200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.589+0000 7ff837fff640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8200b65f0 0x7ff8200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.589+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff8300e7b40 con 0x7ff838106c00 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.590+0000 7ff837fff640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8200b65f0 0x7ff8200b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff828009990 tx=0x7ff828009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:54.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.592+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff830016030 con 0x7ff838106c00 2024-09-21T06:10:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.790+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff838063800 con 0x7ff8200b65f0 2024-09-21T06:10:54.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.803+0000 7ff8357fa640 1 -- 172.21.15.31:0/2968001945 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7ff838063800 con 0x7ff8200b65f0 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (90s) 14s ago 90s 82.4M - 19.3.0-5120-g18432218 ddce307e1f32 22ee396dc0fc 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi107 running (88s) 21s ago 87s 93.6M - 19.3.0-5120-g18432218 ddce307e1f32 824fe551eecd 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi110 running (101s) 21s ago 100s 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 5a0a47c244de 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (109s) 14s ago 108s 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 241ef4fdd1f8 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi107 running (106s) 21s ago 105s 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 d00d1338772c 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi110 running (103s) 21s ago 103s 81.7M - 19.3.0-5120-g18432218 ddce307e1f32 4c4fe3b56612 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (93s) 14s ago 92s 81.6M - 19.3.0-5120-g18432218 ddce307e1f32 ee5a195a6923 2024-09-21T06:10:54.805 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi107 running (99s) 21s ago 98s 86.5M - 19.3.0-5120-g18432218 ddce307e1f32 c57195ac8e99 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi110 running (96s) 21s ago 96s 81.4M - 19.3.0-5120-g18432218 ddce307e1f32 4da8b98930b7 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (112s) 14s ago 111s 82.6M - 19.3.0-5120-g18432218 ddce307e1f32 0e0af7b12f8c 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi107 running (115s) 21s ago 115s 84.3M - 19.3.0-5120-g18432218 ddce307e1f32 f75eddb83e30 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi110 running (117s) 21s ago 117s 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 9ed8dca21fd2 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (9m) 14s ago 9m 584M - 19.3.0-5120-g18432218 ddce307e1f32 283f39cb603a 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi107 *:8443,8765 running (7m) 21s ago 7m 525M - 19.3.0-5120-g18432218 ddce307e1f32 c8af4cd14750 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi110 *:8443,8765 running (7m) 21s ago 7m 523M - 19.3.0-5120-g18432218 ddce307e1f32 7b6497934dcb 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (9m) 14s ago 9m 120M 2048M 19.3.0-5120-g18432218 ddce307e1f32 8fe9989c0afa 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi107 running (8m) 21s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 651178a5eb5a 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi110 running (8m) 21s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 f53d66b96a12 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 14s ago 6m 115M 4273M 19.3.0-5120-g18432218 ddce307e1f32 64904715df81 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi107 running (6m) 21s ago 6m 125M 4271M 19.3.0-5120-g18432218 ddce307e1f32 30d955ba62db 2024-09-21T06:10:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi110 running (5m) 21s ago 5m 125M 4272M 19.3.0-5120-g18432218 ddce307e1f32 81e0bc7f91d6 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (5m) 14s ago 5m 113M 4273M 19.3.0-5120-g18432218 ddce307e1f32 bdc64bd44a6b 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi107 running (5m) 21s ago 5m 112M 4271M 19.3.0-5120-g18432218 ddce307e1f32 69cff832c6be 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi110 running (4m) 21s ago 4m 115M 4272M 19.3.0-5120-g18432218 ddce307e1f32 d4e3c37b794d 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (4m) 14s ago 4m 113M 4273M 19.3.0-5120-g18432218 ddce307e1f32 899043428def 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi107 running (3m) 21s ago 3m 121M 4271M 19.3.0-5120-g18432218 ddce307e1f32 9c2a437c7c2e 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi110 running (3m) 21s ago 3m 118M 4272M 19.3.0-5120-g18432218 ddce307e1f32 00657abe235b 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (3m) 14s ago 3m 112M 4273M 19.3.0-5120-g18432218 ddce307e1f32 fcb70cc1450e 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi107 running (2m) 21s ago 2m 120M 4271M 19.3.0-5120-g18432218 ddce307e1f32 a7b08048005b 2024-09-21T06:10:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi110 running (2m) 21s ago 2m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 4c1c853bc3ec 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8200b65f0 msgr2=0x7ff8200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8200b65f0 0x7ff8200b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff828009990 tx=0x7ff828009110 comp rx=0 tx=0).stop 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 msgr2=0x7ff83819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 0x7ff83819fd70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff83000eaa0 tx=0x7ff83000eea0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 shutdown_connections 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff8200b65f0 0x7ff8200b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff838106c00 0x7ff83819fd70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8381062b0 0x7ff83819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 --2- 172.21.15.31:0/2968001945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8381020c0 0x7ff83819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 >> 172.21.15.31:0/2968001945 conn(0x7ff838079e40 msgr2=0x7ff8381075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 shutdown_connections 2024-09-21T06:10:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:54.805+0000 7ff83e8e2640 1 -- 172.21.15.31:0/2968001945 wait complete. 2024-09-21T06:10:55.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:54 smithi031 ceph-mon[31237]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:55.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:54 smithi110 ceph-mon[34550]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:55.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:54 smithi107 ceph-mon[34380]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:55.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T06:10:55.741 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:56.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:55 smithi031 ceph-mon[31237]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 -- 172.21.15.31:0/881479855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae44105300 msgr2=0x7fae4410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/881479855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae44105300 0x7fae4410d850 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae38009900 tx=0x7fae3802a410 comp rx=0 tx=0).stop 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 -- 172.21.15.31:0/881479855 shutdown_connections 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/881479855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae4410de10 0x7fae44110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/881479855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae44105300 0x7fae4410d850 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/881479855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae441049b0 0x7fae44104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 -- 172.21.15.31:0/881479855 >> 172.21.15.31:0/881479855 conn(0x7fae44079730 msgr2=0x7fae44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 -- 172.21.15.31:0/881479855 shutdown_connections 2024-09-21T06:10:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.225+0000 7fae4b8f8640 1 -- 172.21.15.31:0/881479855 wait complete. 2024-09-21T06:10:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 Processor -- start 2024-09-21T06:10:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 -- start start 2024-09-21T06:10:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 0x7fae4419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:56.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae44105300 0x7fae4419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae4410de10 0x7fae4419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fae44113ab0 con 0x7fae441049b0 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae44113930 con 0x7fae4410de10 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae4b8f8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae44113c30 con 0x7fae44105300 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 0x7fae4419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 0x7fae4419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47118/0 (socket says 172.21.15.31:47118) 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 -- 172.21.15.31:0/97191411 learned_addr learned my addr 172.21.15.31:0/97191411 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae48e93640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae44105300 0x7fae4419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49e95640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae4410de10 0x7fae4419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 -- 172.21.15.31:0/97191411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae44105300 msgr2=0x7fae4419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae44105300 0x7fae4419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 -- 172.21.15.31:0/97191411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae4410de10 msgr2=0x7fae4419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae4410de10 0x7fae4419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.226+0000 7fae49694640 1 -- 172.21.15.31:0/97191411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441a03d0 con 0x7fae441049b0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.227+0000 7fae49694640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 0x7fae4419a520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae3400ccb0 tx=0x7fae3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.227+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34003b70 con 0x7fae441049b0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.227+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3400ae50 con 0x7fae441049b0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.227+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae441a06c0 con 0x7fae441049b0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.227+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3400f6f0 con 0x7fae441049b0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.227+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a6ee0 con 0x7fae441049b0 2024-09-21T06:10:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.228+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae44104dc0 con 0x7fae441049b0 2024-09-21T06:10:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.230+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fae34004000 con 0x7fae441049b0 2024-09-21T06:10:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.230+0000 7fae427fc640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fae2c0b66a0 0x7fae2c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.230+0000 7fae48e93640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fae2c0b66a0 0x7fae2c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.231+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fae340e7730 con 0x7fae441049b0 2024-09-21T06:10:56.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.231+0000 7fae48e93640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fae2c0b66a0 0x7fae2c0b8bb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae38009230 tx=0x7fae38031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:56.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.235+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fae34014070 con 0x7fae441049b0 2024-09-21T06:10:56.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:55 smithi110 ceph-mon[34550]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:56.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:55 smithi107 ceph-mon[34380]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:56.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.435+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fae44063800 con 0x7fae2c0b66a0 2024-09-21T06:10:56.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.440+0000 7fae427fc640 1 -- 172.21.15.31:0/97191411 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fae44063800 con 0x7fae2c0b66a0 2024-09-21T06:10:56.442 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T06:10:56.442 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 22s ago 2m smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi107=b;smithi107=e;smithi107=h;smithi107=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-09-21T06:10:56.442 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 22s ago 7m smithi031=x;smithi107=y;smithi110=z;count:3 2024-09-21T06:10:56.443 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 22s ago 8m smithi031:172.21.15.31=a;smithi107:172.21.15.107=b;smithi110:172.21.15.110=c;count:3 2024-09-21T06:10:56.443 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 22s ago - 2024-09-21T06:10:56.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.442+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fae2c0b66a0 msgr2=0x7fae2c0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.442+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fae2c0b66a0 0x7fae2c0b8bb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae38009230 tx=0x7fae38031400 comp rx=0 tx=0).stop 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.442+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 msgr2=0x7fae4419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.442+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 0x7fae4419a520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae3400ccb0 tx=0x7fae3400a620 comp rx=0 tx=0).stop 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 shutdown_connections 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae4410de10 0x7fae4419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae44105300 0x7fae4419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fae2c0b66a0 0x7fae2c0b8bb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 --2- 172.21.15.31:0/97191411 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae441049b0 0x7fae4419a520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 >> 172.21.15.31:0/97191411 conn(0x7fae44079730 msgr2=0x7fae44100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 shutdown_connections 2024-09-21T06:10:56.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:56.443+0000 7fae4b8f8640 1 -- 172.21.15.31:0/97191411 wait complete. 2024-09-21T06:10:57.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:56 smithi031 ceph-mon[31237]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:57.087 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:56 smithi031 ceph-mon[31237]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:57.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T06:10:57.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:56 smithi110 ceph-mon[34550]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:57.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:56 smithi110 ceph-mon[34550]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:57.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:56 smithi107 ceph-mon[34380]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:10:57.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:56 smithi107 ceph-mon[34380]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:57.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 -- 172.21.15.31:0/1106091605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94110200 msgr2=0x7fba94112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:57.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 --2- 172.21.15.31:0/1106091605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94110200 0x7fba94112620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009ca0 tx=0x7fba8c02a950 comp rx=0 tx=0).stop 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 -- 172.21.15.31:0/1106091605 shutdown_connections 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 --2- 172.21.15.31:0/1106091605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94110200 0x7fba94112620 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 --2- 172.21.15.31:0/1106091605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba94102fa0 0x7fba9410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 --2- 172.21.15.31:0/1106091605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba94102650 0x7fba94102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 -- 172.21.15.31:0/1106091605 >> 172.21.15.31:0/1106091605 conn(0x7fba94079760 msgr2=0x7fba94079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 -- 172.21.15.31:0/1106091605 shutdown_connections 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.807+0000 7fba9b84f640 1 -- 172.21.15.31:0/1106091605 wait complete. 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 Processor -- start 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 -- start start 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 0x7fba9419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba94102fa0 0x7fba9419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:57.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba94110200 0x7fba941a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba94115ca0 con 0x7fba94102650 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba94115b20 con 0x7fba94110200 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba9b84f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fba94115e20 con 0x7fba94102fa0 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba995eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 0x7fba9419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba995eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 0x7fba9419c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47140/0 (socket says 172.21.15.31:47140) 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.808+0000 7fba995eb640 1 -- 172.21.15.31:0/609905050 learned_addr learned my addr 172.21.15.31:0/609905050 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba98dea640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba94102fa0 0x7fba9419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba99dec640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba94110200 0x7fba941a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba995eb640 1 -- 172.21.15.31:0/609905050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba94102fa0 msgr2=0x7fba9419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba995eb640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba94102fa0 0x7fba9419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba995eb640 1 -- 172.21.15.31:0/609905050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba94110200 msgr2=0x7fba941a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba995eb640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba94110200 0x7fba941a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba995eb640 1 -- 172.21.15.31:0/609905050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba941a2580 con 0x7fba94102650 2024-09-21T06:10:57.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba995eb640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 0x7fba9419c610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fba9000e9f0 tx=0x7fba9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9000cd80 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba941a2870 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba900054d0 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90010690 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.809+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba941a8fd0 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.811+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9406c1f0 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.812+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fba90004e90 con 0x7fba94102650 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.813+0000 7fba8a7fc640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba800b65f0 0x7fba800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:57.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.813+0000 7fba98dea640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba800b65f0 0x7fba800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:57.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.813+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fba900e7b70 con 0x7fba94102650 2024-09-21T06:10:57.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.813+0000 7fba98dea640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba800b65f0 0x7fba800b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fba84005fd0 tx=0x7fba84005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:57.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:57.817+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba900ae1c0 con 0x7fba94102650 2024-09-21T06:10:58.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.012+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fba94109380 con 0x7fba800b65f0 2024-09-21T06:10:58.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.014+0000 7fba8a7fc640 1 -- 172.21.15.31:0/609905050 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fba94109380 con 0x7fba800b65f0 2024-09-21T06:10:58.015 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-09-21T06:10:58.015 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-09-21T06:10:58.015 INFO:teuthology.orchestra.run.smithi031.stdout:smithi107 172.21.15.107 2024-09-21T06:10:58.016 INFO:teuthology.orchestra.run.smithi031.stdout:smithi110 172.21.15.110 2024-09-21T06:10:58.016 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2024-09-21T06:10:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba800b65f0 msgr2=0x7fba800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba800b65f0 0x7fba800b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fba84005fd0 tx=0x7fba84005850 comp rx=0 tx=0).stop 2024-09-21T06:10:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 msgr2=0x7fba9419c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 0x7fba9419c610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fba9000e9f0 tx=0x7fba9000edf0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 shutdown_connections 2024-09-21T06:10:58.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba94110200 0x7fba941a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba94102fa0 0x7fba9419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fba800b65f0 0x7fba800b8b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 --2- 172.21.15.31:0/609905050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba94102650 0x7fba9419c610 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 >> 172.21.15.31:0/609905050 conn(0x7fba94079760 msgr2=0x7fba94108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 shutdown_connections 2024-09-21T06:10:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:58.016+0000 7fba9b84f640 1 -- 172.21.15.31:0/609905050 wait complete. 2024-09-21T06:10:58.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T06:10:58.908 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:10:59.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:58 smithi031 ceph-mon[31237]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:59.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:10:58 smithi031 ceph-mon[31237]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:58 smithi110 ceph-mon[34550]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:10:58 smithi110 ceph-mon[34550]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:58 smithi107 ceph-mon[34380]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:10:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:10:58 smithi107 ceph-mon[34380]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 -- 172.21.15.31:0/4072252420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8109e40 msgr2=0x7fc3f8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/4072252420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8109e40 0x7fc3f8110670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009ca0 tx=0x7fc3f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 -- 172.21.15.31:0/4072252420 shutdown_connections 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/4072252420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8109e40 0x7fc3f8110670 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/4072252420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f81094f0 0x7fc3f8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/4072252420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f8103bb0 0x7fc3f8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 -- 172.21.15.31:0/4072252420 >> 172.21.15.31:0/4072252420 conn(0x7fc3f80ff600 msgr2=0x7fc3f8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 -- 172.21.15.31:0/4072252420 shutdown_connections 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.372+0000 7fc3fcc12640 1 -- 172.21.15.31:0/4072252420 wait complete. 2024-09-21T06:10:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 Processor -- start 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 -- start start 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 0x7fc3f819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f81094f0 0x7fc3f819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f8109e40 0x7fc3f819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8113b00 con 0x7fc3f8103bb0 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8113980 con 0x7fc3f8109e40 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3fcc12640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8113c80 con 0x7fc3f81094f0 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 0x7fc3f819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 0x7fc3f819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53820/0 (socket says 172.21.15.31:53820) 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.373+0000 7fc3f659c640 1 -- 172.21.15.31:0/1731601771 learned_addr learned my addr 172.21.15.31:0/1731601771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f5d9b640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f81094f0 0x7fc3f819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f6d9d640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f8109e40 0x7fc3f819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f659c640 1 -- 172.21.15.31:0/1731601771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f81094f0 msgr2=0x7fc3f819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f659c640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f81094f0 0x7fc3f819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f659c640 1 -- 172.21.15.31:0/1731601771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f8109e40 msgr2=0x7fc3f819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f659c640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f8109e40 0x7fc3f819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f659c640 1 -- 172.21.15.31:0/1731601771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f81a0300 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3f659c640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 0x7fc3f819a450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800e9f0 tx=0x7fc3e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800ccc0 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e800ce60 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8010430 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f81a05f0 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.374+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f81a6e10 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.375+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f8103fc0 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.377+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc3e80046c0 con 0x7fc3f8103bb0 2024-09-21T06:10:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.378+0000 7fc3e77fe640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3e00b66a0 0x7fc3e00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:10:59.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.378+0000 7fc3f5d9b640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3e00b66a0 0x7fc3e00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:10:59.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.378+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc3e80e7bf0 con 0x7fc3f8103bb0 2024-09-21T06:10:59.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.378+0000 7fc3f5d9b640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3e00b66a0 0x7fc3e00b8bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8106f00 tx=0x7fc3ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:10:59.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.381+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3e8056710 con 0x7fc3f8103bb0 2024-09-21T06:10:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.578+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc3f8063800 con 0x7fc3e00b66a0 2024-09-21T06:10:59.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.580+0000 7fc3e77fe640 1 -- 172.21.15.31:0/1731601771 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc3f8063800 con 0x7fc3e00b66a0 2024-09-21T06:10:59.582 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T06:10:59.582 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T06:10:59.582 INFO:teuthology.orchestra.run.smithi031.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T06:10:59.582 INFO:teuthology.orchestra.run.smithi031.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3e00b66a0 msgr2=0x7fc3e00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3e00b66a0 0x7fc3e00b8bb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8106f00 tx=0x7fc3ec00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 msgr2=0x7fc3f819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 0x7fc3f819a450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800e9f0 tx=0x7fc3e800edf0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 shutdown_connections 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f8109e40 0x7fc3f819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3f81094f0 0x7fc3f819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3e00b66a0 0x7fc3e00b8bb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 --2- 172.21.15.31:0/1731601771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3f8103bb0 0x7fc3f819a450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 >> 172.21.15.31:0/1731601771 conn(0x7fc3f80ff600 msgr2=0x7fc3f8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 shutdown_connections 2024-09-21T06:10:59.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:10:59.582+0000 7fc3fcc12640 1 -- 172.21.15.31:0/1731601771 wait complete. 2024-09-21T06:11:00.199 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:11:00.216 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:11:00.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T06:11:00.477 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.919+0000 7fad75ec6640 1 -- 172.21.15.31:0/3726970293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 msgr2=0x7fad70104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.919+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3726970293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad70104e70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fad60009900 tx=0x7fad6002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 -- 172.21.15.31:0/3726970293 shutdown_connections 2024-09-21T06:11:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3726970293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 0x7fad701103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3726970293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 0x7fad7010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3726970293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad70104e70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 -- 172.21.15.31:0/3726970293 >> 172.21.15.31:0/3726970293 conn(0x7fad700796d0 msgr2=0x7fad70079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 -- 172.21.15.31:0/3726970293 shutdown_connections 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 -- 172.21.15.31:0/3726970293 wait complete. 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 Processor -- start 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.920+0000 7fad75ec6640 1 -- start start 2024-09-21T06:11:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad75ec6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad7019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad75ec6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 0x7fad7019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad75ec6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 0x7fad7019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad75ec6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fad70113b40 con 0x7fad70104a60 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad75ec6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad701139c0 con 0x7fad701053b0 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad75ec6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad70113cc0 con 0x7fad7010dec0 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad7019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad7019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53842/0 (socket says 172.21.15.31:53842) 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 -- 172.21.15.31:0/3837051686 learned_addr learned my addr 172.21.15.31:0/3837051686 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6ffff640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 0x7fad7019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6effd640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 0x7fad7019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 -- 172.21.15.31:0/3837051686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 msgr2=0x7fad7019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 0x7fad7019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 -- 172.21.15.31:0/3837051686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 msgr2=0x7fad7019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 0x7fad7019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 -- 172.21.15.31:0/3837051686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad701a04c0 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6ffff640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 0x7fad7019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6effd640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 0x7fad7019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.921+0000 7fad6f7fe640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad7019a550 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad60005940 tx=0x7fad600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.922+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6003f030 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.922+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701a0750 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.922+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad60003cb0 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.922+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60035070 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.922+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad701a6f10 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.923+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7006c260 con 0x7fad70104a60 2024-09-21T06:11:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.924+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fad60042030 con 0x7fad70104a60 2024-09-21T06:11:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.925+0000 7fad6cff9640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fad580b65f0 0x7fad580b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.925+0000 7fad6effd640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fad580b65f0 0x7fad580b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.925+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fad60108200 con 0x7fad70104a60 2024-09-21T06:11:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.926+0000 7fad6effd640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fad580b65f0 0x7fad580b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad7010fb90 tx=0x7fad64005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:00.928+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad600ce7f0 con 0x7fad70104a60 2024-09-21T06:11:01.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:00 smithi031 ceph-mon[31237]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:01.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:00 smithi031 ceph-mon[31237]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:11:01.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.157+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fad701a70e0 con 0x7fad70104a60 2024-09-21T06:11:01.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.157+0000 7fad6cff9640 1 -- 172.21.15.31:0/3837051686 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+3046 (secure 0 0 0) 0x7fad600d2bd0 con 0x7fad70104a60 2024-09-21T06:11:01.159 INFO:teuthology.orchestra.run.smithi031.stdout:e19 2024-09-21T06:11:01.160 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-21T06:09:46:301719+0000 2024-09-21T06:11:01.160 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:11:01.160 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:11:01.161 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-21T06:11:01.161 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:01.161 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-21T06:11:01.161 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 19 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-21T06:09:36.903399+0000 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-21T06:09:45.967301+0000 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-21T06:11:01.162 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24719} 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-09-21T06:11:01.163 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24719 members: 24719 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{0:24719} state up:active seq 4 addr [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{0:24686} state up:standby-replay seq 1 addr [v2:172.21.15.107:6836/2795950298,v1:172.21.15.107:6837/2795950298] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14766} state up:standby seq 1 addr [v2:172.21.15.31:6834/3018651170,v1:172.21.15.31:6835/3018651170] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{-1:14778} state up:standby seq 1 addr [v2:172.21.15.31:6836/594088756,v1:172.21.15.31:6837/594088756] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{-1:14829} state up:standby seq 1 addr [v2:172.21.15.31:6838/3970721911,v1:172.21.15.31:6839/3970721911] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{-1:14835} state up:standby seq 1 addr [v2:172.21.15.31:6840/4177463092,v1:172.21.15.31:6841/4177463092] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{-1:24613} state up:standby seq 1 addr [v2:172.21.15.110:6832/128096132,v1:172.21.15.110:6833/128096132] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{-1:24626} state up:standby seq 1 addr [v2:172.21.15.107:6832/4010348472,v1:172.21.15.107:6833/4010348472] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{-1:24646} state up:standby seq 1 addr [v2:172.21.15.110:6834/1733260219,v1:172.21.15.110:6835/1733260219] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{-1:24658} state up:standby seq 1 addr [v2:172.21.15.110:6836/969332587,v1:172.21.15.110:6837/969332587] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{-1:24662} state up:standby seq 1 addr [v2:172.21.15.107:6834/306946935,v1:172.21.15.107:6835/306946935] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{-1:24676} state up:standby seq 1 addr [v2:172.21.15.110:6838/1863570857,v1:172.21.15.110:6839/1863570857] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:11:01.165 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:01.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fad580b65f0 msgr2=0x7fad580b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fad580b65f0 0x7fad580b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad7010fb90 tx=0x7fad64005e50 comp rx=0 tx=0).stop 2024-09-21T06:11:01.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 msgr2=0x7fad7019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:01.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad7019a550 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad60005940 tx=0x7fad600042f0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 shutdown_connections 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad7010dec0 0x7fad7019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad701053b0 0x7fad7019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fad580b65f0 0x7fad580b8b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 --2- 172.21.15.31:0/3837051686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad70104a60 0x7fad7019a550 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 >> 172.21.15.31:0/3837051686 conn(0x7fad700796d0 msgr2=0x7fad701079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 shutdown_connections 2024-09-21T06:11:01.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:01.160+0000 7fad75ec6640 1 -- 172.21.15.31:0/3837051686 wait complete. 2024-09-21T06:11:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:00 smithi110 ceph-mon[34550]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:00 smithi110 ceph-mon[34550]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:11:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:00 smithi107 ceph-mon[34380]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:00 smithi107 ceph-mon[34380]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:11:01.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T06:11:02.121 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:02.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:01 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3837051686' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:11:02.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:01 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3837051686' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:11:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:01 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3837051686' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 -- 172.21.15.31:0/3070915028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c109df0 msgr2=0x7faf4c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3070915028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c109df0 0x7faf4c110620 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faf44009ca0 tx=0x7faf4402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 -- 172.21.15.31:0/3070915028 shutdown_connections 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3070915028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c109df0 0x7faf4c110620 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3070915028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c1094a0 0x7faf4c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3070915028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf4c103b60 0x7faf4c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 -- 172.21.15.31:0/3070915028 >> 172.21.15.31:0/3070915028 conn(0x7faf4c07b850 msgr2=0x7faf4c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 -- 172.21.15.31:0/3070915028 shutdown_connections 2024-09-21T06:11:02.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 -- 172.21.15.31:0/3070915028 wait complete. 2024-09-21T06:11:02.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.602+0000 7faf51d3a640 1 Processor -- start 2024-09-21T06:11:02.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 -- start start 2024-09-21T06:11:02.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c103b60 0x7faf4c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 0x7faf4c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf4c109df0 0x7faf4c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c113ab0 con 0x7faf4c1094a0 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c113930 con 0x7faf4c109df0 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf51d3a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c113c30 con 0x7faf4c103b60 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf4affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 0x7faf4c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf4affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 0x7faf4c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53858/0 (socket says 172.21.15.31:53858) 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf4affd640 1 -- 172.21.15.31:0/3932965243 learned_addr learned my addr 172.21.15.31:0/3932965243 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf4b7fe640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c103b60 0x7faf4c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.603+0000 7faf4bfff640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf4c109df0 0x7faf4c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf4affd640 1 -- 172.21.15.31:0/3932965243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c103b60 msgr2=0x7faf4c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf4affd640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c103b60 0x7faf4c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf4affd640 1 -- 172.21.15.31:0/3932965243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf4c109df0 msgr2=0x7faf4c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf4affd640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf4c109df0 0x7faf4c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf4affd640 1 -- 172.21.15.31:0/3932965243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c1a03b0 con 0x7faf4c1094a0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf4affd640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 0x7faf4c19a9e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d740 tx=0x7faf4000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40017070 con 0x7faf4c1094a0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4000de50 con 0x7faf4c1094a0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c1a06a0 con 0x7faf4c1094a0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4000f620 con 0x7faf4c1094a0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.604+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c1a6e00 con 0x7faf4c1094a0 2024-09-21T06:11:02.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.605+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c103f70 con 0x7faf4c1094a0 2024-09-21T06:11:02.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.607+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7faf4001a070 con 0x7faf4c1094a0 2024-09-21T06:11:02.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.607+0000 7faf48ff9640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faf340b65f0 0x7faf340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:02.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.607+0000 7faf4b7fe640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faf340b65f0 0x7faf340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:02.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.608+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7faf400e6f50 con 0x7faf4c1094a0 2024-09-21T06:11:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.608+0000 7faf4b7fe640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faf340b65f0 0x7faf340b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bbe0 tx=0x7faf3c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:02.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.612+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf400b1980 con 0x7faf4c1094a0 2024-09-21T06:11:02.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.800+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7faf4c10bc40 con 0x7faf4c1094a0 2024-09-21T06:11:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.801+0000 7faf48ff9640 1 -- 172.21.15.31:0/3932965243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5689 (secure 0 0 0) 0x7faf400b68c0 con 0x7faf4c1094a0 2024-09-21T06:11:02.803 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 77 2024-09-21T06:11:02.803 INFO:teuthology.orchestra.run.smithi031.stdout:fsid c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:11:02.803 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-21T06:01:12.820557+0000 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-21T06:10:07.387838+0000 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 26 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-21T06:11:02.804 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-21T06:11:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2024-09-21T06:11:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-21T06:11:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-21T06:11:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 9 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/215542336,v1:172.21.15.31:6803/215542336] [v2:172.21.15.31:6804/215542336,v1:172.21.15.31:6805/215542336] exists,up 80675501-db5a-4326-bf65-f570e2ef7dcf 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 14 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6800/2850742213,v1:172.21.15.107:6801/2850742213] [v2:172.21.15.107:6802/2850742213,v1:172.21.15.107:6803/2850742213] exists,up b0e6db75-1424-4686-b6d1-f889a7cf426c 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 19 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/3771685673,v1:172.21.15.110:6801/3771685673] [v2:172.21.15.110:6802/3771685673,v1:172.21.15.110:6803/3771685673] exists,up c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 26 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/1116793462,v1:172.21.15.31:6811/1116793462] [v2:172.21.15.31:6812/1116793462,v1:172.21.15.31:6813/1116793462] exists,up 24b23ec1-a44d-4d63-9a6e-fab54498c7db 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 31 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6808/888650447,v1:172.21.15.107:6809/888650447] [v2:172.21.15.107:6810/888650447,v1:172.21.15.107:6811/888650447] exists,up a5c6b1c9-2f76-4407-a87a-abcddb48fba3 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/3518155164,v1:172.21.15.110:6809/3518155164] [v2:172.21.15.110:6810/3518155164,v1:172.21.15.110:6811/3518155164] exists,up 44342172-be5e-4e39-ad8e-0aed23c0efb1 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 42 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/3870004119,v1:172.21.15.31:6819/3870004119] [v2:172.21.15.31:6820/3870004119,v1:172.21.15.31:6821/3870004119] exists,up f9f50053-37df-4668-b5a6-c186af35a2b8 2024-09-21T06:11:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 47 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6816/1767013823,v1:172.21.15.107:6817/1767013823] [v2:172.21.15.107:6818/1767013823,v1:172.21.15.107:6819/1767013823] exists,up 26b1487e-7c8b-4705-8418-c1687458e181 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 53 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/4244140994,v1:172.21.15.110:6817/4244140994] [v2:172.21.15.110:6818/4244140994,v1:172.21.15.110:6819/4244140994] exists,up ee40c435-fe2e-4026-bfbc-2dfed559f5d9 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 58 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/1883621951,v1:172.21.15.31:6827/1883621951] [v2:172.21.15.31:6828/1883621951,v1:172.21.15.31:6829/1883621951] exists,up 0d5a01fe-27f8-4e91-86b1-031461e41633 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 64 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6824/3727307895,v1:172.21.15.107:6825/3727307895] [v2:172.21.15.107:6826/3727307895,v1:172.21.15.107:6827/3727307895] exists,up 0c49045f-5d86-4a85-9c2b-22e12fb42351 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/2652663870,v1:172.21.15.110:6825/2652663870] [v2:172.21.15.110:6826/2652663870,v1:172.21.15.110:6827/2652663870] exists,up 3922d4ae-a015-4a0e-92e8-2eb5daa7afe4 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.4 [2,11] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.b [8,11] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.e [2,4] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.c [3,9] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.10 [5,9] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.11 [4,8] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.13 [4,9] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.1a [5,6] 2024-09-21T06:11:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.23 [4,7] 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/4284062429 expires 2024-09-22T06:03:05.499388+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/10892239 expires 2024-09-22T06:03:05.499388+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/3238988869 expires 2024-09-22T06:01:35.161130+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2647070256 expires 2024-09-22T06:01:35.161130+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1169957326 expires 2024-09-22T06:03:05.499388+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/733625975 expires 2024-09-22T06:01:35.161130+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1111343468 expires 2024-09-22T06:02:01.461252+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2445761929 expires 2024-09-22T06:02:01.461252+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/465044057 expires 2024-09-22T06:01:35.161130+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/3238988869 expires 2024-09-22T06:01:35.161130+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/65624344 expires 2024-09-22T06:03:05.499388+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/979439078 expires 2024-09-22T06:02:01.461252+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/836492518 expires 2024-09-22T06:02:01.461252+0000 2024-09-21T06:11:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/979439078 expires 2024-09-22T06:02:01.461252+0000 2024-09-21T06:11:02.809 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/65624344 expires 2024-09-22T06:03:05.499388+0000 2024-09-21T06:11:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.803+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faf340b65f0 msgr2=0x7faf340b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.803+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faf340b65f0 0x7faf340b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00bbe0 tx=0x7faf3c00b390 comp rx=0 tx=0).stop 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.803+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 msgr2=0x7faf4c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.803+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 0x7faf4c19a9e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf4000d740 tx=0x7faf4000dc00 comp rx=0 tx=0).stop 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 shutdown_connections 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faf4c109df0 0x7faf4c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7faf340b65f0 0x7faf340b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1094a0 0x7faf4c19a9e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 --2- 172.21.15.31:0/3932965243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c103b60 0x7faf4c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 >> 172.21.15.31:0/3932965243 conn(0x7faf4c07b850 msgr2=0x7faf4c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 shutdown_connections 2024-09-21T06:11:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:02.804+0000 7faf51d3a640 1 -- 172.21.15.31:0/3932965243 wait complete. 2024-09-21T06:11:03.113 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:02 smithi031 ceph-mon[31237]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:11:03.113 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:02 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3932965243' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:11:03.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:02 smithi110 ceph-mon[34550]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:11:03.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:02 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3932965243' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:11:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:02 smithi107 ceph-mon[34380]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:11:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:02 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3932965243' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:11:03.418 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T06:11:03.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:11:03.697 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:04.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 -- 172.21.15.31:0/4138415179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d81052b0 msgr2=0x7f78d810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 --2- 172.21.15.31:0/4138415179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d81052b0 0x7f78d810d8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009900 tx=0x7f78cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:04.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 -- 172.21.15.31:0/4138415179 shutdown_connections 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 --2- 172.21.15.31:0/4138415179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78d810de30 0x7f78d8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 --2- 172.21.15.31:0/4138415179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d81052b0 0x7f78d810d8f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 --2- 172.21.15.31:0/4138415179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d8104880 0x7f78d8104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 -- 172.21.15.31:0/4138415179 >> 172.21.15.31:0/4138415179 conn(0x7f78d8079730 msgr2=0x7f78d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 -- 172.21.15.31:0/4138415179 shutdown_connections 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.155+0000 7f78dcb76640 1 -- 172.21.15.31:0/4138415179 wait complete. 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.156+0000 7f78dcb76640 1 Processor -- start 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.156+0000 7f78dcb76640 1 -- start start 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 0x7f78d8102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78d81052b0 0x7f78d8102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d810de30 0x7f78d8103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8113a70 con 0x7f78d8104880 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f78d81138f0 con 0x7f78d810de30 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f78d8113bf0 con 0x7f78d81052b0 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 0x7f78d8102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 0x7f78d8102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53870/0 (socket says 172.21.15.31:53870) 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 -- 172.21.15.31:0/1690522607 learned_addr learned my addr 172.21.15.31:0/1690522607 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d5d9b640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78d81052b0 0x7f78d8102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d6d9d640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d810de30 0x7f78d8103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 -- 172.21.15.31:0/1690522607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78d81052b0 msgr2=0x7f78d8102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78d81052b0 0x7f78d8102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 -- 172.21.15.31:0/1690522607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d810de30 msgr2=0x7f78d8103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d810de30 0x7f78d8103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 -- 172.21.15.31:0/1690522607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d81a8740 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d6d9d640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d810de30 0x7f78d8103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78d659c640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 0x7f78d8102790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f78c800ccb0 tx=0x7f78c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8003990 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c8004020 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81a8a30 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.157+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c800f6b0 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.158+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a8f40 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.159+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d8104c90 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.160+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f78c80044b0 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.161+0000 7f78c77fe640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78c00b65f0 0x7f78c00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.161+0000 7f78d5d9b640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78c00b65f0 0x7f78c00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.161+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f78c80e79b0 con 0x7f78d8104880 2024-09-21T06:11:04.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.162+0000 7f78d5d9b640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78c00b65f0 0x7f78c00b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000c00 tx=0x7f78cc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:04.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.164+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78c8014030 con 0x7f78d8104880 2024-09-21T06:11:04.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.394+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d8103a90 con 0x7f78d8104880 2024-09-21T06:11:04.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.394+0000 7f78c77fe640 1 -- 172.21.15.31:0/1690522607 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f78c80ae000 con 0x7f78d8104880 2024-09-21T06:11:04.397 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:04.397 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:11:04.397 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78c00b65f0 msgr2=0x7f78c00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78c00b65f0 0x7f78c00b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000c00 tx=0x7f78cc031400 comp rx=0 tx=0).stop 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 msgr2=0x7f78d8102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 0x7f78d8102790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f78c800ccb0 tx=0x7f78c800a620 comp rx=0 tx=0).stop 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 shutdown_connections 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f78d810de30 0x7f78d8103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78d81052b0 0x7f78d8102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f78c00b65f0 0x7f78c00b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 --2- 172.21.15.31:0/1690522607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d8104880 0x7f78d8102790 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 >> 172.21.15.31:0/1690522607 conn(0x7f78d8079730 msgr2=0x7f78d8100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 shutdown_connections 2024-09-21T06:11:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:04.397+0000 7f78dcb76640 1 -- 172.21.15.31:0/1690522607 wait complete. 2024-09-21T06:11:05.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:11:05.107 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:04 smithi031 ceph-mon[31237]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:11:05.107 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:04 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1690522607' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:04 smithi110 ceph-mon[34550]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:11:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:04 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1690522607' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:04 smithi107 ceph-mon[34380]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T06:11:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:04 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1690522607' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:05.346 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 -- 172.21.15.31:0/688167092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 msgr2=0x7f34b010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 --2- 172.21.15.31:0/688167092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b010f550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009ca0 tx=0x7f34a802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 -- 172.21.15.31:0/688167092 shutdown_connections 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 --2- 172.21.15.31:0/688167092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b010f550 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 --2- 172.21.15.31:0/688167092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34b01083d0 0x7f34b01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 --2- 172.21.15.31:0/688167092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0102ae0 0x7f34b0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 -- 172.21.15.31:0/688167092 >> 172.21.15.31:0/688167092 conn(0x7f34b0079760 msgr2=0x7f34b0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 -- 172.21.15.31:0/688167092 shutdown_connections 2024-09-21T06:11:05.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 -- 172.21.15.31:0/688167092 wait complete. 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 Processor -- start 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.855+0000 7f34b59d8640 1 -- start start 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34b59d8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34b0102ae0 0x7f34b019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34b59d8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b01083d0 0x7f34b019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34b59d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b019bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34b59d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112990 con 0x7f34b0108d20 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34b59d8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112810 con 0x7f34b01083d0 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34b59d8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112b10 con 0x7f34b0102ae0 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b019bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b019bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53878/0 (socket says 172.21.15.31:53878) 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 -- 172.21.15.31:0/3787019615 learned_addr learned my addr 172.21.15.31:0/3787019615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34aeffd640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34b0102ae0 0x7f34b019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34a67fc640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b01083d0 0x7f34b019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 -- 172.21.15.31:0/3787019615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34b0102ae0 msgr2=0x7f34b019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34b0102ae0 0x7f34b019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 -- 172.21.15.31:0/3787019615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b01083d0 msgr2=0x7f34b019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b01083d0 0x7f34b019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 -- 172.21.15.31:0/3787019615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b01a6400 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.856+0000 7f34af7fe640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b019bc80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005cc0 tx=0x7f34a802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.857+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8003bf0 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.857+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a8003d90 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.857+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b01a6690 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.857+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a802fbc0 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.857+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a6bd0 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.858+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b0103d10 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.860+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f34a802fd60 con 0x7f34b0108d20 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.860+0000 7f34b49d6640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f349c0b6620 0x7f349c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.861+0000 7f34aeffd640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f349c0b6620 0x7f349c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:05.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.861+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f34a8108b30 con 0x7f34b0108d20 2024-09-21T06:11:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.861+0000 7f34aeffd640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f349c0b6620 0x7f349c0b8b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005d00 tx=0x7f34a0005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:05.864+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34a80cf180 con 0x7f34b0108d20 2024-09-21T06:11:06.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.121+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34b0102ef0 con 0x7f34b0108d20 2024-09-21T06:11:06.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.122+0000 7f34b49d6640 1 -- 172.21.15.31:0/3787019615 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f34a80098b0 con 0x7f34b0108d20 2024-09-21T06:11:06.124 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:06.124 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:11:06.124 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:06.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f349c0b6620 msgr2=0x7f349c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f349c0b6620 0x7f349c0b8b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005d00 tx=0x7f34a0005c70 comp rx=0 tx=0).stop 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 msgr2=0x7f34b019bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b019bc80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005cc0 tx=0x7f34a802f380 comp rx=0 tx=0).stop 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 shutdown_connections 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f349c0b6620 0x7f349c0b8b30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34b0108d20 0x7f34b019bc80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b01083d0 0x7f34b019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 --2- 172.21.15.31:0/3787019615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34b0102ae0 0x7f34b019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 >> 172.21.15.31:0/3787019615 conn(0x7f34b0079760 msgr2=0x7f34b01043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 shutdown_connections 2024-09-21T06:11:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:06.125+0000 7f34b59d8640 1 -- 172.21.15.31:0/3787019615 wait complete. 2024-09-21T06:11:06.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:11:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:07.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:06 smithi031 ceph-mon[31237]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:11:07.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:06 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3787019615' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:07.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:06 smithi110 ceph-mon[34550]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:11:07.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:06 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3787019615' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:07.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:06 smithi107 ceph-mon[34380]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T06:11:07.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:06 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3787019615' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 -- 172.21.15.31:0/660598261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c1049b0 msgr2=0x7fd88c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 --2- 172.21.15.31:0/660598261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c1049b0 0x7fd88c104dc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009ca0 tx=0x7fd87c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 -- 172.21.15.31:0/660598261 shutdown_connections 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 --2- 172.21.15.31:0/660598261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c10de10 0x7fd88c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 --2- 172.21.15.31:0/660598261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 0x7fd88c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 --2- 172.21.15.31:0/660598261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c1049b0 0x7fd88c104dc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 -- 172.21.15.31:0/660598261 >> 172.21.15.31:0/660598261 conn(0x7fd88c079730 msgr2=0x7fd88c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 -- 172.21.15.31:0/660598261 shutdown_connections 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.561+0000 7fd89107f640 1 -- 172.21.15.31:0/660598261 wait complete. 2024-09-21T06:11:07.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 Processor -- start 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 -- start start 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c1049b0 0x7fd88c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 0x7fd88c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 0x7fd88c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113a90 con 0x7fd88c10de10 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113910 con 0x7fd88c105300 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd89107f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c113c10 con 0x7fd88c1049b0 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd88b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 0x7fd88c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd88b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 0x7fd88c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53906/0 (socket says 172.21.15.31:53906) 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd88b59e640 1 -- 172.21.15.31:0/2814817898 learned_addr learned my addr 172.21.15.31:0/2814817898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd88ad9d640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c1049b0 0x7fd88c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.562+0000 7fd88a59c640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 0x7fd88c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88b59e640 1 -- 172.21.15.31:0/2814817898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c1049b0 msgr2=0x7fd88c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88b59e640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c1049b0 0x7fd88c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88b59e640 1 -- 172.21.15.31:0/2814817898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 msgr2=0x7fd88c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88b59e640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 0x7fd88c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88b59e640 1 -- 172.21.15.31:0/2814817898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c1a0470 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88a59c640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 0x7fd88c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd88b59e640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 0x7fd88c19fd40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c970 tx=0x7fd88400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884007bb0 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1a0760 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd884007d50 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884004e60 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.563+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a6ec0 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.564+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88c104dc0 con 0x7fd88c10de10 2024-09-21T06:11:07.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.566+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd884005000 con 0x7fd88c10de10 2024-09-21T06:11:07.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.566+0000 7fd883fff640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8780b65f0 0x7fd8780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:07.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.566+0000 7fd88ad9d640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8780b65f0 0x7fd8780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:07.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.567+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd8840e7590 con 0x7fd88c10de10 2024-09-21T06:11:07.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.567+0000 7fd88ad9d640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8780b65f0 0x7fd8780b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098d0 tx=0x7fd87c0093e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:07.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.569+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd884014030 con 0x7fd88c10de10 2024-09-21T06:11:07.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.763+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd88c19b630 con 0x7fd88c10de10 2024-09-21T06:11:07.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.765+0000 7fd883fff640 1 -- 172.21.15.31:0/2814817898 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23089 (secure 0 0 0) 0x7fd88c19b630 con 0x7fd88c10de10 2024-09-21T06:11:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:07.768 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:10:07.387838+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:07.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8780b65f0 msgr2=0x7fd8780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8780b65f0 0x7fd8780b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098d0 tx=0x7fd87c0093e0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 msgr2=0x7fd88c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:07.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 0x7fd88c19fd40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c970 tx=0x7fd88400ce30 comp rx=0 tx=0).stop 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 shutdown_connections 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8780b65f0 0x7fd8780b8b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c10de10 0x7fd88c19fd40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88c105300 0x7fd88c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 --2- 172.21.15.31:0/2814817898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd88c1049b0 0x7fd88c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 >> 172.21.15.31:0/2814817898 conn(0x7fd88c079730 msgr2=0x7fd88c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 shutdown_connections 2024-09-21T06:11:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:07.768+0000 7fd89107f640 1 -- 172.21.15.31:0/2814817898 wait complete. 2024-09-21T06:11:08.116 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:07 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2814817898' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:08.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:07 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2814817898' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:08.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:07 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2814817898' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:08.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:09:36.903399+0000', 'modified': '2024-09-21T06:09:45.967301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24719}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24686': {'gid': 24686, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6837/2795950298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6836', 'nonce': 2795950298}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 2795950298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24719': {'gid': 24719, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.107:6839/648775801', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 648775801}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 648775801}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24719, 'qdb_cluster': [24719]} 2024-09-21T06:11:08.421 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:11:08.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:11:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:09.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:08 smithi031 ceph-mon[31237]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-09-21T06:11:09.045 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:08 smithi031 ceph-mon[31237]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 -- 172.21.15.31:0/3681383863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c1094a0 msgr2=0x7f873c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 --2- 172.21.15.31:0/3681383863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c1094a0 0x7f873c1098b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8730009900 tx=0x7f873002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 -- 172.21.15.31:0/3681383863 shutdown_connections 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 --2- 172.21.15.31:0/3681383863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c109df0 0x7f873c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 --2- 172.21.15.31:0/3681383863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c1094a0 0x7f873c1098b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 --2- 172.21.15.31:0/3681383863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f873c103b60 0x7f873c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 -- 172.21.15.31:0/3681383863 >> 172.21.15.31:0/3681383863 conn(0x7f873c07b850 msgr2=0x7f873c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 -- 172.21.15.31:0/3681383863 shutdown_connections 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.136+0000 7f8741258640 1 -- 172.21.15.31:0/3681383863 wait complete. 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 Processor -- start 2024-09-21T06:11:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 -- start start 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f873c103b60 0x7f873c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1094a0 0x7f873c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 0x7f873c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f873c113ad0 con 0x7f873c109df0 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f873c113950 con 0x7f873c1094a0 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f8741258640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f873c113c50 con 0x7f873c103b60 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f873b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 0x7f873c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f873b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 0x7f873c19fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39412/0 (socket says 172.21.15.31:39412) 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f873b59e640 1 -- 172.21.15.31:0/3562450096 learned_addr learned my addr 172.21.15.31:0/3562450096 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f873ad9d640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f873c103b60 0x7f873c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.137+0000 7f873a59c640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1094a0 0x7f873c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f873b59e640 1 -- 172.21.15.31:0/3562450096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f873c103b60 msgr2=0x7f873c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f873b59e640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f873c103b60 0x7f873c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f873b59e640 1 -- 172.21.15.31:0/3562450096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1094a0 msgr2=0x7f873c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f873b59e640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1094a0 0x7f873c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f873b59e640 1 -- 172.21.15.31:0/3562450096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c1a0470 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f873b59e640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 0x7f873c19fd40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f873400b970 tx=0x7f873400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f8723fff640 1 -- 172.21.15.31:0/3562450096 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734019070 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c1a0760 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f8723fff640 1 -- 172.21.15.31:0/3562450096 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8734014440 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f8723fff640 1 -- 172.21.15.31:0/3562450096 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340075d0 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.138+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c1a6ec0 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 Processor -- start 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f8723fff640 1 -- 172.21.15.31:0/3562450096 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f873401c070 con 0x7f873c109df0 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 -- start start 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc12f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc13440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b84c280 0x56078bc13980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56078bb8ac10 con 0x56078b8440a0 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56078bb7a040 con 0x56078b84c280 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f87477cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56078bb7de50 con 0x56078b86cf60 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f873ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc13440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f873ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc13440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39426/0 (socket says 172.21.15.31:39426) 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.141+0000 7f873ad9d640 1 -- 172.21.15.31:0/1042059300 learned_addr learned my addr 172.21.15.31:0/1042059300 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873a59c640 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b84c280 0x56078bc13980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873a59c640 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc12f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873ad9d640 1 -- 172.21.15.31:0/1042059300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 msgr2=0x56078bc12f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f8723fff640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87240b6560 0x7f87240b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc12f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873ad9d640 1 -- 172.21.15.31:0/1042059300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b84c280 msgr2=0x56078bc13980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b84c280 0x56078bc13980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873ad9d640 1 -- 172.21.15.31:0/1042059300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078b8d8540 con 0x56078b8440a0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873a59c640 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b84c280 0x56078bc13980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873b59e640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87240b6560 0x7f87240b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc13440 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f872c00ac50 tx=0x7f872c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6a107a8919284f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f8723fff640 1 -- 172.21.15.31:0/3562450096 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f87340e6950 con 0x7f873c109df0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f87217fa640 1 -- 172.21.15.31:0/1042059300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c01d070 con 0x56078b8440a0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f87217fa640 1 -- 172.21.15.31:0/1042059300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f872c018af0 con 0x56078b8440a0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.142+0000 7f87217fa640 1 -- 172.21.15.31:0/1042059300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c010b30 con 0x56078b8440a0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 -- 172.21.15.31:0/1042059300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 msgr2=0x56078bc13440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f873b59e640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87240b6560 0x7f87240b8a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87340ac7b0 tx=0x7f8734071040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc13440 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f872c00ac50 tx=0x7f872c00f790 comp rx=0 tx=0).stop 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 -- 172.21.15.31:0/1042059300 shutdown_connections 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b84c280 0x56078bc13980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc13440 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 --2- 172.21.15.31:0/1042059300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc12f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 -- 172.21.15.31:0/1042059300 >> 172.21.15.31:0/1042059300 conn(0x56078b861350 msgr2=0x56078bc10df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 -- 172.21.15.31:0/1042059300 shutdown_connections 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.143+0000 7f87477cc740 1 -- 172.21.15.31:0/1042059300 wait complete. 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 -- start start 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 Processor -- start 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b8440a0 0x56078bc22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 0x56078bc232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56078b9db5f0 con 0x56078b84c280 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56078bb15010 con 0x56078b8440a0 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56078bc25a10 con 0x56078b86cf60 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f87477cc740 10 client.0 _ll_register_callbacks cb 0x56078bb74340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:09.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f873ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 0x56078bc232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f873ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 0x56078bc232d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39428/0 (socket says 172.21.15.31:39428) 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.144+0000 7f873ad9d640 1 -- 172.21.15.31:0/1790128490 learned_addr learned my addr 172.21.15.31:0/1790128490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873a59c640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b8440a0 0x56078bc22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873a59c640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc2ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873ad9d640 1 -- 172.21.15.31:0/1790128490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 msgr2=0x56078bc2ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc2ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873ad9d640 1 -- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b8440a0 msgr2=0x56078bc22d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b8440a0 0x56078bc22d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873ad9d640 1 -- 172.21.15.31:0/1790128490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078bb42a00 con 0x56078b84c280 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873a59c640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc2ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 0x56078bc232d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f872c010810 tx=0x7f872c0186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db223a185ba2c5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f871eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f871eff5640 1 -- 172.21.15.31:0/1790128490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c01d040 con 0x56078b84c280 2024-09-21T06:11:09.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f871eff5640 1 -- 172.21.15.31:0/1790128490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f872c0188e0 con 0x56078b84c280 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872c018af0 con 0x56078b84c280 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f871eff5640 1 -- 172.21.15.31:0/1790128490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f872c0108b0 con 0x56078b84c280 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f87477cc740 10 client.15270 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.145+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56078bc31950 con 0x56078b84c280 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f871eff5640 1 -- 172.21.15.31:0/1790128490 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f872c018af0 con 0x56078b84c280 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f871eff5640 1 -- 172.21.15.31:0/1790128490 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f872c020030 con 0x56078b84c280 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f87477cc740 10 client.15270 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f87477cc740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f87477cc740 20 client.15270 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f87477cc740 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b82b8e0 0x56078bc34f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f87477cc740 4 client.15270 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56078bb8cb60 con 0x56078b82b8e0 2024-09-21T06:11:09.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.146+0000 7f873a59c640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b82b8e0 0x56078bc34f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.147+0000 7f873a59c640 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b82b8e0 0x56078bc34f70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9462985d5c5cc0 server_cookie=f19ddb5c1ee54691 in_seq=0 out_seq=0 2024-09-21T06:11:09.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.147+0000 7f871eff5640 10 client.15270 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.147+0000 7f871eff5640 1 -- 172.21.15.31:0/1790128490 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56078bb8cb60 con 0x56078b82b8e0 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.147+0000 7f871eff5640 10 client.15270 handle_command_reply: tid=0 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.147+0000 7f87477cc740 1 client.15270 shutdown 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 msgr2=0x56078bc232d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 0x56078bc232d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f872c010810 tx=0x7f872c0186f0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 shutdown_connections 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b82b8e0 0x56078bc34f70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b86cf60 0x56078bc2ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b84c280 0x56078bc232d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f871eff5640 0 client.15270 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 --2- 172.21.15.31:0/1790128490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b8440a0 0x56078bc22d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 >> 172.21.15.31:0/1790128490 conn(0x56078b861350 msgr2=0x56078bc16090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 shutdown_connections 2024-09-21T06:11:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 1 -- 172.21.15.31:0/1790128490 wait complete. 2024-09-21T06:11:09.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.148+0000 7f87477cc740 20 client.15270 trim_cache size 0 max 16384 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 Processor -- start 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 -- start start 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc22d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56078bc332b0 con 0x56078b8440a0 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56078bc33980 con 0x56078b86cf60 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f87477cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56078bc34050 con 0x56078b84c280 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f873a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f873a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc22080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39444/0 (socket says 172.21.15.31:39444) 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.166+0000 7f873a59c640 1 -- 172.21.15.31:0/1937987218 learned_addr learned my addr 172.21.15.31:0/1937987218 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:09.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc22d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873a59c640 1 -- 172.21.15.31:0/1937987218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 msgr2=0x56078bc22d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873a59c640 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc22d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873a59c640 1 -- 172.21.15.31:0/1937987218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 msgr2=0x56078bc232b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873a59c640 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873a59c640 1 -- 172.21.15.31:0/1937987218 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078bc21090 con 0x56078b8440a0 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f873a59c640 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc22080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8730005ab0 tx=0x7f873002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc237a0929b7437c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f8720ff9640 1 -- 172.21.15.31:0/1937987218 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873004b070 con 0x56078b8440a0 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f8720ff9640 1 -- 172.21.15.31:0/1937987218 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8730044070 con 0x56078b8440a0 2024-09-21T06:11:09.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.167+0000 7f8720ff9640 1 -- 172.21.15.31:0/1937987218 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730035040 con 0x56078b8440a0 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 -- 172.21.15.31:0/1937987218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 msgr2=0x56078bc22080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc22080 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8730005ab0 tx=0x7f873002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 -- 172.21.15.31:0/1937987218 shutdown_connections 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc232b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc22d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 --2- 172.21.15.31:0/1937987218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc22080 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 -- 172.21.15.31:0/1937987218 >> 172.21.15.31:0/1937987218 conn(0x56078b861350 msgr2=0x56078bc1dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 -- 172.21.15.31:0/1937987218 shutdown_connections 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 -- 172.21.15.31:0/1937987218 wait complete. 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 -- start start 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.168+0000 7f87477cc740 1 Processor -- start 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc3c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc3ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc3d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56078bc34df0 con 0x56078b8440a0 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56078bc34720 con 0x56078b86cf60 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56078bc354c0 con 0x56078b84c280 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f87477cc740 10 client.0 _ll_register_callbacks cb 0x7f872c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc3c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc3c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39456/0 (socket says 172.21.15.31:39456) 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 -- 172.21.15.31:0/1574350689 learned_addr learned my addr 172.21.15.31:0/1574350689 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc3ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc3d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 -- 172.21.15.31:0/1574350689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 msgr2=0x56078bc3ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc3ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 -- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 msgr2=0x56078bc3d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc3d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 -- 172.21.15.31:0/1574350689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078bc3b790 con 0x56078b8440a0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f873a59c640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc3c8c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8730006010 tx=0x7f8730031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a7dffe4a2d4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.169+0000 7f871eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f871eff5640 1 -- 172.21.15.31:0/1574350689 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730035070 con 0x56078b8440a0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f871eff5640 1 -- 172.21.15.31:0/1574350689 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8730032450 con 0x56078b8440a0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8730044070 con 0x56078b8440a0 2024-09-21T06:11:09.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f871eff5640 1 -- 172.21.15.31:0/1574350689 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f873004b050 con 0x56078b8440a0 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 10 client.15276 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56078bca0630 con 0x56078b8440a0 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f871eff5640 1 -- 172.21.15.31:0/1574350689 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8730044070 con 0x56078b8440a0 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f871eff5640 1 -- 172.21.15.31:0/1574350689 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8730031ae0 con 0x56078b8440a0 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 10 client.15276 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 20 client.15276 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b800c90 0x56078bca3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 4 client.15276 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.170+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "status"}) -- 0x56078bb88610 con 0x56078b800c90 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b800c90 0x56078bca3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:09.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f873ad9d640 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b800c90 0x56078bca3b00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb9b36df280f30e server_cookie=77781221f9fe669e in_seq=0 out_seq=0 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f871eff5640 10 client.15276 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f871eff5640 1 -- 172.21.15.31:0/1574350689 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56078bb88610 con 0x56078b800c90 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f871eff5640 10 client.15276 handle_command_reply: tid=0 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 client.15276 shutdown 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 msgr2=0x56078bc3c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc3c8c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8730006010 tx=0x7f8730031400 comp rx=0 tx=0).stop 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 shutdown_connections 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56078b800c90 0x56078bca3b00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56078b86cf60 0x56078bc3d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56078b84c280 0x56078bc3ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f871eff5640 0 client.15276 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 --2- 172.21.15.31:0/1574350689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56078b8440a0 0x56078bc3c8c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.171+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 >> 172.21.15.31:0/1574350689 conn(0x56078b861350 msgr2=0x56078bc22d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 shutdown_connections 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f87477cc740 1 -- 172.21.15.31:0/1574350689 wait complete. 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f87477cc740 20 client.15276 trim_cache size 0 max 16384 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "c04f7042-77de-11ef-baf5-efdc52797490", 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24719, 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 91.889858926000002, 2024-09-21T06:11:09.179 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 19, 2024-09-21T06:11:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 77, 2024-09-21T06:11:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T06:11:09.180 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 102.14701927199999 2024-09-21T06:11:09.180 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87240b6560 msgr2=0x7f87240b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87240b6560 0x7f87240b8a70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f87340ac7b0 tx=0x7f8734071040 comp rx=0 tx=0).stop 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 msgr2=0x7f873c19fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 0x7f873c19fd40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f873400b970 tx=0x7f873400be30 comp rx=0 tx=0).stop 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 shutdown_connections 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87240b6560 0x7f87240b8a70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f873c109df0 0x7f873c19fd40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1094a0 0x7f873c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 --2- 172.21.15.31:0/3562450096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f873c103b60 0x7f873c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 >> 172.21.15.31:0/3562450096 conn(0x7f873c07b850 msgr2=0x7f873c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 shutdown_connections 2024-09-21T06:11:09.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:09.172+0000 7f8741258640 1 -- 172.21.15.31:0/3562450096 wait complete. 2024-09-21T06:11:09.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:08 smithi110 ceph-mon[34550]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-09-21T06:11:09.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:08 smithi110 ceph-mon[34550]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T06:11:09.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:08 smithi107 ceph-mon[34380]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-09-21T06:11:09.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:08 smithi107 ceph-mon[34380]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T06:11:09.905 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T06:11:09.929 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T06:11:09.929 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T06:11:09.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:09.929 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-21T06:11:09.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:09.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-21T06:11:10.018 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-09-21T06:11:10.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:11:10.020 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-21T06:11:10.020 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:11:10.020 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-21T06:11:10.020 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-21T06:11:10.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T06:11:10.320 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 -- 172.21.15.31:0/3059442594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc102a60 msgr2=0x7f36cc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3059442594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc102a60 0x7f36cc102e70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009900 tx=0x7f36bc02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 -- 172.21.15.31:0/3059442594 shutdown_connections 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3059442594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36cc108cf0 0x7f36cc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3059442594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc1083a0 0x7f36cc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3059442594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc102a60 0x7f36cc102e70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 -- 172.21.15.31:0/3059442594 >> 172.21.15.31:0/3059442594 conn(0x7f36cc079730 msgr2=0x7f36cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 -- 172.21.15.31:0/3059442594 shutdown_connections 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.800+0000 7f36d2c97640 1 -- 172.21.15.31:0/3059442594 wait complete. 2024-09-21T06:11:10.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 Processor -- start 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 -- start start 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc102a60 0x7f36cc10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 0x7f36cc1104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36cc108cf0 0x7f36cc110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc113ac0 con 0x7f36cc1083a0 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc113940 con 0x7f36cc102a60 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d2c97640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc113c40 con 0x7f36cc108cf0 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 0x7f36cc1104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 0x7f36cc1104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39474/0 (socket says 172.21.15.31:39474) 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36cbfff640 1 -- 172.21.15.31:0/3623537153 learned_addr learned my addr 172.21.15.31:0/3623537153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.801+0000 7f36d0a33640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc102a60 0x7f36cc10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36d1234640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36cc108cf0 0x7f36cc110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36cbfff640 1 -- 172.21.15.31:0/3623537153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36cc108cf0 msgr2=0x7f36cc110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36cbfff640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36cc108cf0 0x7f36cc110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36cbfff640 1 -- 172.21.15.31:0/3623537153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc102a60 msgr2=0x7f36cc10ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36cbfff640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc102a60 0x7f36cc10ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36cbfff640 1 -- 172.21.15.31:0/3623537153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc1a87f0 con 0x7f36cc1083a0 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36cbfff640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 0x7f36cc1104f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d770 tx=0x7f36c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0017070 con 0x7f36cc1083a0 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc1a8ae0 con 0x7f36cc1083a0 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c0009c30 con 0x7f36cc1083a0 2024-09-21T06:11:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c000f620 con 0x7f36cc1083a0 2024-09-21T06:11:10.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.802+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc1a9020 con 0x7f36cc1083a0 2024-09-21T06:11:10.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.803+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36cc102e70 con 0x7f36cc1083a0 2024-09-21T06:11:10.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.805+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f36c001a070 con 0x7f36cc1083a0 2024-09-21T06:11:10.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.806+0000 7f36c9ffb640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f36b40b6560 0x7f36b40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:10.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.806+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f36c00e6e80 con 0x7f36cc1083a0 2024-09-21T06:11:10.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.806+0000 7f36d0a33640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f36b40b6560 0x7f36b40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:10.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.806+0000 7f36d0a33640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f36b40b6560 0x7f36b40b8a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f36bc000c00 tx=0x7f36bc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:10.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:10.809+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36c0009de0 con 0x7f36cc1083a0 2024-09-21T06:11:11.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.035+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36cc1112b0 con 0x7f36cc1083a0 2024-09-21T06:11:11.037 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:10 smithi031 ceph-mon[31237]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-09-21T06:11:11.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.036+0000 7f36c9ffb640 1 -- 172.21.15.31:0/3623537153 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7f36cc1112b0 con 0x7f36cc1083a0 2024-09-21T06:11:11.038 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T06:11:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f36b40b6560 msgr2=0x7f36b40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f36b40b6560 0x7f36b40b8a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f36bc000c00 tx=0x7f36bc031400 comp rx=0 tx=0).stop 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 msgr2=0x7f36cc1104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 0x7f36cc1104f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d770 tx=0x7f36c000dc30 comp rx=0 tx=0).stop 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 shutdown_connections 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36cc108cf0 0x7f36cc110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f36b40b6560 0x7f36b40b8a70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36cc1083a0 0x7f36cc1104f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.038+0000 7f36d2c97640 1 --2- 172.21.15.31:0/3623537153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36cc102a60 0x7f36cc10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.039+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 >> 172.21.15.31:0/3623537153 conn(0x7f36cc079730 msgr2=0x7f36cc101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.039+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 shutdown_connections 2024-09-21T06:11:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11.039+0000 7f36d2c97640 1 -- 172.21.15.31:0/3623537153 wait complete. 2024-09-21T06:11:11.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:10 smithi110 ceph-mon[34550]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-09-21T06:11:11.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:10 smithi107 ceph-mon[34380]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-09-21T06:11:11.671 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:11:11.672 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-21T06:11:11.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:11.673 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-09-21T06:11:11.690 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:11:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:11:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-09-21T06:11:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T06:11:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42177sec preferred_lft 42177sec 2024-09-21T06:11:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-21T06:11:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:11:11.692 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:11:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-09-21T06:11:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-09-21T06:11:11.692 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:11:11.692 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:11:11.693 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T06:11:11.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:11:11.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-09-21T06:11:11.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-09-21T06:11:11.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:11:11.693 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T06:11:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:11.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:11.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:11.917 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:11:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-21T06:11:11.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:11.946 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-09-21T06:11:12.010 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-09-21T06:11:12.011 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:11:12.011 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T06:11:12.011 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T06:11:12.011 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:11:12.011 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:11:12.011 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T06:11:12.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T06:11:12.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T06:11:12.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T06:11:12.012 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T06:11:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:12.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:11 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3623537153' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:11:12.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:11 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3623537153' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:11:12.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:11 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3623537153' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:11:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:12.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:12.338 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-21T06:11:12.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:12.355 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-09-21T06:11:12.411 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:11:12.411 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T06:11:12.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-21T06:11:12.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-21T06:11:12.411 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T06:11:12.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:12.530 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-21T06:11:12.530 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:11:12.530 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T06:11:12.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-21T06:11:12.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:11:12.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-21T06:11:12.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-21T06:11:12.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-21T06:11:12.531 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T06:11:12.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:12.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:12.657 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:11:12.657 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-21T06:11:12.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-09-21T06:11:12.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:11:12.657 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-21T06:11:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:12.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:12.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:11:12.736 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /tmp/cephfs 2024-09-21T06:11:12.752 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-21T06:11:12.752 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:11:12.752 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /tmp/cephfs 2024-09-21T06:11:12.813 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T06:11:12.813 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T06:11:12.813 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/11ff3a3e-12f9-435b-bdf0-22b9c15d836e 2024-09-21T06:11:12.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:12.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/11ff3a3e-12f9-435b-bdf0-22b9c15d836e /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-21T06:11:12.953 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-21T06:11:12.954 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T06:11:12.954 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-21T06:11:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-21T06:11:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T06:11:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-21T06:11:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.107:3300,172.21.15.110:3300" 2024-09-21T06:11:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:3300,172.21.15.107:3300,172.21.15.110:3300:/volumes/qa/sv_1/11ff3a3e-12f9-435b-bdf0-22b9c15d836e 2024-09-21T06:11:13.088 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=c04f7042-77de-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T06:11:13.090 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:11:13.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:11:13.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /tmp/cephfs 2024-09-21T06:11:13.118 DEBUG:teuthology.orchestra.run.smithi031:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:11:13.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:12 smithi031 ceph-mon[31237]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:11:13.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:11:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:13.193 INFO:teuthology.orchestra.run.smithi031.stdout:client25024 2024-09-21T06:11:13.193 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:11:13.193 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:11:13.210 DEBUG:teuthology.orchestra.run.smithi031:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:11:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:12 smithi110 ceph-mon[34550]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:11:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:12 smithi107 ceph-mon[34380]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:11:13.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:11:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:13.308 INFO:teuthology.orchestra.run.smithi031.stdout:client25024 2024-09-21T06:11:13.308 DEBUG:teuthology.orchestra.run.smithi031:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:11:13.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:11:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:13.365 INFO:teuthology.orchestra.run.smithi031.stdout:client25024 2024-09-21T06:11:13.365 DEBUG:teuthology.orchestra.run.smithi031:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c04f7042-77de-11ef-baf5-efdc52797490.client25024/status') 2024-09-21T06:11:13.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:11:13.425 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.25024 (3)172.21.15.31:0/4182434461 2024-09-21T06:11:13.426 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-09-21T06:11:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-09-21T06:11:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-09-21T06:11:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:71 bytes copied, 2.3754e-05 s, 3.0 MB/s 2024-09-21T06:11:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:11:13.433 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T06:11:13.444 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T06:11:13.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:11:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 -- 172.21.15.31:0/2709861733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1029b0 msgr2=0x7f7bcc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/2709861733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1029b0 0x7f7bcc102dc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009900 tx=0x7f7bbc02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 -- 172.21.15.31:0/2709861733 shutdown_connections 2024-09-21T06:11:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/2709861733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 0x7f7bcc10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/2709861733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/2709861733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1029b0 0x7f7bcc102dc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 -- 172.21.15.31:0/2709861733 >> 172.21.15.31:0/2709861733 conn(0x7f7bcc0fe3b0 msgr2=0x7f7bcc100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:13.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 -- 172.21.15.31:0/2709861733 shutdown_connections 2024-09-21T06:11:13.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.591+0000 7f7bd2f52640 1 -- 172.21.15.31:0/2709861733 wait complete. 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 Processor -- start 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 -- start start 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bcc1029b0 0x7f7bcc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 0x7f7bcc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1128f0 con 0x7f7bcc1082e0 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc112770 con 0x7f7bcc108c30 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bd2f52640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc112a70 con 0x7f7bcc1029b0 2024-09-21T06:11:13.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39500/0 (socket says 172.21.15.31:39500) 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.592+0000 7f7bcbfff640 1 -- 172.21.15.31:0/1578588432 learned_addr learned my addr 172.21.15.31:0/1578588432 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bd0cee640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bcc1029b0 0x7f7bcc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bd14ef640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 0x7f7bcc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bcbfff640 1 -- 172.21.15.31:0/1578588432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bcc1029b0 msgr2=0x7f7bcc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bcbfff640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bcc1029b0 0x7f7bcc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bcbfff640 1 -- 172.21.15.31:0/1578588432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 msgr2=0x7f7bcc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bcbfff640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 0x7f7bcc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bcbfff640 1 -- 172.21.15.31:0/1578588432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc19f280 con 0x7f7bcc1082e0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bd14ef640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 0x7f7bcc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bcbfff640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1998f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400d750 tx=0x7f7bb400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4017070 con 0x7f7bcc1082e0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bb40054d0 con 0x7f7bcc1082e0 2024-09-21T06:11:13.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4004c30 con 0x7f7bcc1082e0 2024-09-21T06:11:13.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc19f570 con 0x7f7bcc1082e0 2024-09-21T06:11:13.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.593+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc1a5d00 con 0x7f7bcc1082e0 2024-09-21T06:11:13.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.594+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bcc102dc0 con 0x7f7bcc1082e0 2024-09-21T06:11:13.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.596+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f7bb4004dd0 con 0x7f7bcc1082e0 2024-09-21T06:11:13.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.597+0000 7f7bc9ffb640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7bb80b65e0 0x7f7bb80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.597+0000 7f7bd0cee640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7bb80b65e0 0x7f7bb80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.597+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7bb40e72d0 con 0x7f7bcc1082e0 2024-09-21T06:11:13.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.598+0000 7f7bd0cee640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7bb80b65e0 0x7f7bb80b8af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc031440 tx=0x7f7bbc005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:13.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.600+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7bb40adb00 con 0x7f7bcc1082e0 2024-09-21T06:11:13.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.790+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bcc02e640 con 0x7f7bcc1082e0 2024-09-21T06:11:13.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.791+0000 7f7bc9ffb640 1 -- 172.21.15.31:0/1578588432 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23089 (secure 0 0 0) 0x7f7bb40e6020 con 0x7f7bcc1082e0 2024-09-21T06:11:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:13.793 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:10:07.387838+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:13.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7bb80b65e0 msgr2=0x7f7bb80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7bb80b65e0 0x7f7bb80b8af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc031440 tx=0x7f7bbc005d50 comp rx=0 tx=0).stop 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 msgr2=0x7f7bcc1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1998f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400d750 tx=0x7f7bb400dc10 comp rx=0 tx=0).stop 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 shutdown_connections 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bcc108c30 0x7f7bcc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7bb80b65e0 0x7f7bb80b8af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bcc1082e0 0x7f7bcc1998f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 --2- 172.21.15.31:0/1578588432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bcc1029b0 0x7f7bcc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 >> 172.21.15.31:0/1578588432 conn(0x7f7bcc0fe3b0 msgr2=0x7f7bcc107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 shutdown_connections 2024-09-21T06:11:13.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.794+0000 7f7bd2f52640 1 -- 172.21.15.31:0/1578588432 wait complete. 2024-09-21T06:11:13.807 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:05:05.978676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:09:33.023556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:09:34.842253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:11:13.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 -- 172.21.15.31:0/938904232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2190108c50 msgr2=0x7f219010f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 --2- 172.21.15.31:0/938904232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2190108c50 0x7f219010f480 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2188009ca0 tx=0x7f218802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 -- 172.21.15.31:0/938904232 shutdown_connections 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 --2- 172.21.15.31:0/938904232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2190108c50 0x7f219010f480 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 --2- 172.21.15.31:0/938904232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2190108300 0x7f2190108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 --2- 172.21.15.31:0/938904232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21901029d0 0x7f2190102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 -- 172.21.15.31:0/938904232 >> 172.21.15.31:0/938904232 conn(0x7f21900fe430 msgr2=0x7f2190100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:13.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 -- 172.21.15.31:0/938904232 shutdown_connections 2024-09-21T06:11:13.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 -- 172.21.15.31:0/938904232 wait complete. 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.924+0000 7f2197fc6640 1 Processor -- start 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 -- start start 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 0x7f2190199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2190108300 0x7f21901998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2190108c50 0x7f219019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2190112960 con 0x7f21901029d0 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21901127e0 con 0x7f2190108300 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2197fc6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2190112ae0 con 0x7f2190108c50 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 0x7f2190199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 0x7f2190199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39524/0 (socket says 172.21.15.31:39524) 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 -- 172.21.15.31:0/4190299650 learned_addr learned my addr 172.21.15.31:0/4190299650 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195561640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2190108300 0x7f21901998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2196563640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2190108c50 0x7f219019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 -- 172.21.15.31:0/4190299650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2190108c50 msgr2=0x7f219019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2190108c50 0x7f219019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 -- 172.21.15.31:0/4190299650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2190108300 msgr2=0x7f21901998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2190108300 0x7f21901998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 -- 172.21.15.31:0/4190299650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219019f250 con 0x7f21901029d0 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.925+0000 7f2195d62640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 0x7f2190199320 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f218c00d950 tx=0x7f218c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.926+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c017030 con 0x7f21901029d0 2024-09-21T06:11:13.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.926+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f218c004020 con 0x7f21901029d0 2024-09-21T06:11:13.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.926+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0047a0 con 0x7f21901029d0 2024-09-21T06:11:13.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.926+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219019f540 con 0x7f21901029d0 2024-09-21T06:11:13.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.926+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21901a5cd0 con 0x7f21901029d0 2024-09-21T06:11:13.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.927+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2190102de0 con 0x7f21901029d0 2024-09-21T06:11:13.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.928+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f218c009a30 con 0x7f21901029d0 2024-09-21T06:11:13.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.929+0000 7f2186ffd640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f217c0b65e0 0x7f217c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:13.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.929+0000 7f2195561640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f217c0b65e0 0x7f217c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:13.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.929+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f218c0e7490 con 0x7f21901029d0 2024-09-21T06:11:13.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.930+0000 7f2195561640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f217c0b65e0 0x7f217c0b8af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2190101ce0 tx=0x7f2180005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:13.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:13.932+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f218c0adcc0 con 0x7f21901029d0 2024-09-21T06:11:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.121+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f219002e640 con 0x7f21901029d0 2024-09-21T06:11:14.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.121+0000 7f2186ffd640 1 -- 172.21.15.31:0/4190299650 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f218c0b20a0 con 0x7f21901029d0 2024-09-21T06:11:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-21T06:11:14.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f217c0b65e0 msgr2=0x7f217c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f217c0b65e0 0x7f217c0b8af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2190101ce0 tx=0x7f2180005f90 comp rx=0 tx=0).stop 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 msgr2=0x7f2190199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 0x7f2190199320 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f218c00d950 tx=0x7f218c00de10 comp rx=0 tx=0).stop 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 shutdown_connections 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2190108c50 0x7f219019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2190108300 0x7f21901998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f217c0b65e0 0x7f217c0b8af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 --2- 172.21.15.31:0/4190299650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21901029d0 0x7f2190199320 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.123+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 >> 172.21.15.31:0/4190299650 conn(0x7f21900fe430 msgr2=0x7f2190107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.124+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 shutdown_connections 2024-09-21T06:11:14.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.124+0000 7f2197fc6640 1 -- 172.21.15.31:0/4190299650 wait complete. 2024-09-21T06:11:14.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:13 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1578588432' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:14.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:11:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 -- 172.21.15.31:0/1566551982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01061c0 msgr2=0x7f66d01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 --2- 172.21.15.31:0/1566551982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01061c0 0x7f66d01065d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009900 tx=0x7f66c002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 -- 172.21.15.31:0/1566551982 shutdown_connections 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 --2- 172.21.15.31:0/1566551982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d0106b10 0x7f66d01105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 --2- 172.21.15.31:0/1566551982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01061c0 0x7f66d01065d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 --2- 172.21.15.31:0/1566551982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66d0101fe0 0x7f66d01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 -- 172.21.15.31:0/1566551982 >> 172.21.15.31:0/1566551982 conn(0x7f66d0079e10 msgr2=0x7f66d007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 -- 172.21.15.31:0/1566551982 shutdown_connections 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 -- 172.21.15.31:0/1566551982 wait complete. 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 Processor -- start 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.252+0000 7f66d73c3640 1 -- start start 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d73c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 0x7f66d019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d73c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66d01061c0 0x7f66d019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d73c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d0106b10 0x7f66d019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d73c3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01139e0 con 0x7f66d0101fe0 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d73c3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0113860 con 0x7f66d01061c0 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d73c3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0113b60 con 0x7f66d0106b10 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 0x7f66d019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 0x7f66d019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39546/0 (socket says 172.21.15.31:39546) 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 -- 172.21.15.31:0/3303527077 learned_addr learned my addr 172.21.15.31:0/3303527077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d5960640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d0106b10 0x7f66d019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d495e640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66d01061c0 0x7f66d019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 -- 172.21.15.31:0/3303527077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d0106b10 msgr2=0x7f66d019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d0106b10 0x7f66d019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 -- 172.21.15.31:0/3303527077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66d01061c0 msgr2=0x7f66d019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66d01061c0 0x7f66d019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 -- 172.21.15.31:0/3303527077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01a0310 con 0x7f66d0101fe0 2024-09-21T06:11:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66d515f640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 0x7f66d019a3e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e9c0 tx=0x7f66cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.253+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc00cd80 con 0x7f66d0101fe0 2024-09-21T06:11:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.254+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d01a0600 con 0x7f66d0101fe0 2024-09-21T06:11:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.254+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc0035f0 con 0x7f66d0101fe0 2024-09-21T06:11:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.254+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc010690 con 0x7f66d0101fe0 2024-09-21T06:11:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.254+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a6d90 con 0x7f66d0101fe0 2024-09-21T06:11:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.255+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d01023f0 con 0x7f66d0101fe0 2024-09-21T06:11:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.256+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f66cc0037a0 con 0x7f66d0101fe0 2024-09-21T06:11:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.257+0000 7f66c67fc640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66bc0b6560 0x7f66bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.257+0000 7f66d495e640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66bc0b6560 0x7f66bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.257+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f66cc0e7930 con 0x7f66d0101fe0 2024-09-21T06:11:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.258+0000 7f66d495e640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66bc0b6560 0x7f66bc0b8a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009230 tx=0x7f66c0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.262+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66cc0ae0e0 con 0x7f66d0101fe0 2024-09-21T06:11:14.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:13 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1578588432' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:14.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:13 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1578588432' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:14.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.449+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f66d002e640 con 0x7f66d0101fe0 2024-09-21T06:11:14.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.449+0000 7f66c67fc640 1 -- 172.21.15.31:0/3303527077 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f66cc0b24c0 con 0x7f66d0101fe0 2024-09-21T06:11:14.451 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66bc0b6560 msgr2=0x7f66bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66bc0b6560 0x7f66bc0b8a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009230 tx=0x7f66c0031400 comp rx=0 tx=0).stop 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 msgr2=0x7f66d019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 0x7f66d019a3e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e9c0 tx=0x7f66cc00ee80 comp rx=0 tx=0).stop 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 shutdown_connections 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66d0106b10 0x7f66d019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66d01061c0 0x7f66d019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66bc0b6560 0x7f66bc0b8a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 --2- 172.21.15.31:0/3303527077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d0101fe0 0x7f66d019a3e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 >> 172.21.15.31:0/3303527077 conn(0x7f66d0079e10 msgr2=0x7f66d0109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 shutdown_connections 2024-09-21T06:11:14.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.452+0000 7f66d73c3640 1 -- 172.21.15.31:0/3303527077 wait complete. 2024-09-21T06:11:14.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 -- 172.21.15.31:0/2367677531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1940108c10 msgr2=0x7f194010f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 --2- 172.21.15.31:0/2367677531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1940108c10 0x7f194010f440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1938009ca0 tx=0x7f193802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 -- 172.21.15.31:0/2367677531 shutdown_connections 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 --2- 172.21.15.31:0/2367677531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1940108c10 0x7f194010f440 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 --2- 172.21.15.31:0/2367677531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19401082c0 0x7f19401086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 --2- 172.21.15.31:0/2367677531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1940102990 0x7f1940102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 -- 172.21.15.31:0/2367677531 >> 172.21.15.31:0/2367677531 conn(0x7f19400fe3b0 msgr2=0x7f1940100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 -- 172.21.15.31:0/2367677531 shutdown_connections 2024-09-21T06:11:14.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.579+0000 7f1946739640 1 -- 172.21.15.31:0/2367677531 wait complete. 2024-09-21T06:11:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 Processor -- start 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 -- start start 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1940102990 0x7f194019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 0x7f194019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1940108c10 0x7f194019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19401128d0 con 0x7f19401082c0 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1940112750 con 0x7f1940108c10 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f1946739640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1940112a50 con 0x7f1940102990 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f193f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 0x7f194019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f193f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 0x7f194019b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39568/0 (socket says 172.21.15.31:39568) 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.580+0000 7f193f7fe640 1 -- 172.21.15.31:0/759215424 learned_addr learned my addr 172.21.15.31:0/759215424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f1944cd6640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1940108c10 0x7f194019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193ffff640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1940102990 0x7f194019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193f7fe640 1 -- 172.21.15.31:0/759215424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1940102990 msgr2=0x7f194019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193f7fe640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1940102990 0x7f194019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193f7fe640 1 -- 172.21.15.31:0/759215424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1940108c10 msgr2=0x7f194019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193f7fe640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1940108c10 0x7f194019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193f7fe640 1 -- 172.21.15.31:0/759215424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19401a6440 con 0x7f19401082c0 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193f7fe640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 0x7f194019b6c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f193400e9f0 tx=0x7f193400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193400ccc0 con 0x7f19401082c0 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19340108e0 con 0x7f19401082c0 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401a6730 con 0x7f19401082c0 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934018860 con 0x7f19401082c0 2024-09-21T06:11:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.581+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401a6c80 con 0x7f19401082c0 2024-09-21T06:11:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.582+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1940102da0 con 0x7f19401082c0 2024-09-21T06:11:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.584+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f1934010430 con 0x7f19401082c0 2024-09-21T06:11:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.585+0000 7f193d7fa640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19280b6560 0x7f19280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.585+0000 7f193ffff640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19280b6560 0x7f19280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.585+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f19340e7c90 con 0x7f19401082c0 2024-09-21T06:11:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.585+0000 7f193ffff640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19280b6560 0x7f19280b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1930005fd0 tx=0x7f1930005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.588+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19340ae3e0 con 0x7f19401082c0 2024-09-21T06:11:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.775+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f194002e640 con 0x7f19401082c0 2024-09-21T06:11:14.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.775+0000 7f193d7fa640 1 -- 172.21.15.31:0/759215424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f19340b27c0 con 0x7f19401082c0 2024-09-21T06:11:14.777 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.777+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19280b6560 msgr2=0x7f19280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.777+0000 7f1946739640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19280b6560 0x7f19280b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1930005fd0 tx=0x7f1930005850 comp rx=0 tx=0).stop 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.777+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 msgr2=0x7f194019b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.777+0000 7f1946739640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 0x7f194019b6c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f193400e9f0 tx=0x7f193400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 shutdown_connections 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1940108c10 0x7f194019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19280b6560 0x7f19280b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19401082c0 0x7f194019b6c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 --2- 172.21.15.31:0/759215424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1940102990 0x7f194019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 >> 172.21.15.31:0/759215424 conn(0x7f19400fe3b0 msgr2=0x7f19401007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 shutdown_connections 2024-09-21T06:11:14.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:14.778+0000 7f1946739640 1 -- 172.21.15.31:0/759215424 wait complete. 2024-09-21T06:11:14.791 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T06:11:14.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:11:15.051 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:15.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:14 smithi031 ceph-mon[31237]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:11:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:14 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4190299650' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:14 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3303527077' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:14 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/759215424' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.179 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:14 smithi110 ceph-mon[34550]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:11:15.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:14 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4190299650' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:14 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3303527077' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.180 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:14 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/759215424' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.190 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:14 smithi107 ceph-mon[34380]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 255 B/s wr, 2 op/s 2024-09-21T06:11:15.190 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:14 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4190299650' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.190 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:14 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3303527077' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.190 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:14 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/759215424' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 -- 172.21.15.31:0/2567264599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 msgr2=0x7f39bc1125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2567264599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1125a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009ca0 tx=0x7f39b402a950 comp rx=0 tx=0).stop 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 -- 172.21.15.31:0/2567264599 shutdown_connections 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2567264599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1125a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2567264599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc102f20 0x7f39bc10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2567264599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39bc1025d0 0x7f39bc1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 -- 172.21.15.31:0/2567264599 >> 172.21.15.31:0/2567264599 conn(0x7f39bc079730 msgr2=0x7f39bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 -- 172.21.15.31:0/2567264599 shutdown_connections 2024-09-21T06:11:15.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.554+0000 7f39c3bec640 1 -- 172.21.15.31:0/2567264599 wait complete. 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.555+0000 7f39c3bec640 1 Processor -- start 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.555+0000 7f39c3bec640 1 -- start start 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c3bec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc1025d0 0x7f39bc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c3bec640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39bc102f20 0x7f39bc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c3bec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c3bec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115c70 con 0x7f39bc110180 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c3bec640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115af0 con 0x7f39bc102f20 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c3bec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc115df0 con 0x7f39bc1025d0 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c2189640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c2189640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39590/0 (socket says 172.21.15.31:39590) 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c2189640 1 -- 172.21.15.31:0/2096131160 learned_addr learned my addr 172.21.15.31:0/2096131160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc1025d0 0x7f39bc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1187640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39bc102f20 0x7f39bc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 -- 172.21.15.31:0/2096131160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39bc102f20 msgr2=0x7f39bc19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39bc102f20 0x7f39bc19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 -- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 msgr2=0x7f39bc1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 -- 172.21.15.31:0/2096131160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1a25f0 con 0x7f39bc1025d0 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c2189640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.556+0000 7f39c1988640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc1025d0 0x7f39bc19c680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39b800d6d0 tx=0x7f39b800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.557+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8017070 con 0x7f39bc1025d0 2024-09-21T06:11:15.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.557+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a28e0 con 0x7f39bc1025d0 2024-09-21T06:11:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.557+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39b8009820 con 0x7f39bc1025d0 2024-09-21T06:11:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.557+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8004c40 con 0x7f39bc1025d0 2024-09-21T06:11:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.557+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a9040 con 0x7f39bc1025d0 2024-09-21T06:11:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.558+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc1029e0 con 0x7f39bc1025d0 2024-09-21T06:11:15.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.560+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f39b8005220 con 0x7f39bc1025d0 2024-09-21T06:11:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.560+0000 7f39b2ffd640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39a80b6560 0x7f39a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.561+0000 7f39c1187640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39a80b6560 0x7f39a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.561+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f39b80e7290 con 0x7f39bc1025d0 2024-09-21T06:11:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.561+0000 7f39c1187640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39a80b6560 0x7f39a80b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39bc103fd0 tx=0x7f39ac005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:15.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.564+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f39b80b1e20 con 0x7f39bc1025d0 2024-09-21T06:11:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.790+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39bc19d7b0 con 0x7f39bc1025d0 2024-09-21T06:11:15.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.791+0000 7f39b2ffd640 1 -- 172.21.15.31:0/2096131160 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f39b80b6d60 con 0x7f39bc1025d0 2024-09-21T06:11:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:15.794 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:11:15.794 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39a80b6560 msgr2=0x7f39a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39a80b6560 0x7f39a80b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39bc103fd0 tx=0x7f39ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc1025d0 msgr2=0x7f39bc19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc1025d0 0x7f39bc19c680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39b800d6d0 tx=0x7f39b800db90 comp rx=0 tx=0).stop 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 shutdown_connections 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f39a80b6560 0x7f39a80b8a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39bc110180 0x7f39bc1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39bc102f20 0x7f39bc19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 --2- 172.21.15.31:0/2096131160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39bc1025d0 0x7f39bc19c680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 >> 172.21.15.31:0/2096131160 conn(0x7f39bc079730 msgr2=0x7f39bc109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 shutdown_connections 2024-09-21T06:11:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:15.794+0000 7f39c3bec640 1 -- 172.21.15.31:0/2096131160 wait complete. 2024-09-21T06:11:16.064 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:15 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2096131160' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:16.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:15 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2096131160' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:16.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:15 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2096131160' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:16.520 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T06:11:16.520 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T06:11:16.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:11:16.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 -- 172.21.15.31:0/38177452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010b600 msgr2=0x7f11f010ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 --2- 172.21.15.31:0/38177452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010b600 0x7f11f010ba10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009ca0 tx=0x7f11e002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 -- 172.21.15.31:0/38177452 shutdown_connections 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 --2- 172.21.15.31:0/38177452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f010bf50 0x7f11f0112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 --2- 172.21.15.31:0/38177452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010b600 0x7f11f010ba10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 --2- 172.21.15.31:0/38177452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0105cd0 0x7f11f01060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 -- 172.21.15.31:0/38177452 >> 172.21.15.31:0/38177452 conn(0x7f11f01016f0 msgr2=0x7f11f0103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 -- 172.21.15.31:0/38177452 shutdown_connections 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.647+0000 7f11f7030640 1 -- 172.21.15.31:0/38177452 wait complete. 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 Processor -- start 2024-09-21T06:11:16.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 -- start start 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0105cd0 0x7f11f019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f010b600 0x7f11f019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 0x7f11f01a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0115c10 con 0x7f11f010bf50 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0115a90 con 0x7f11f0105cd0 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f7030640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f11f0115d90 con 0x7f11f010b600 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.648+0000 7f11f55cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 0x7f11f01a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 0x7f11f01a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39606/0 (socket says 172.21.15.31:39606) 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 -- 172.21.15.31:0/2804191291 learned_addr learned my addr 172.21.15.31:0/2804191291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f4dcc640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0105cd0 0x7f11f019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11effff640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f010b600 0x7f11f019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 -- 172.21.15.31:0/2804191291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f010b600 msgr2=0x7f11f019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f010b600 0x7f11f019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 -- 172.21.15.31:0/2804191291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0105cd0 msgr2=0x7f11f019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0105cd0 0x7f11f019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 -- 172.21.15.31:0/2804191291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f01a2520 con 0x7f11f010bf50 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f55cd640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 0x7f11f01a1e20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f11e800d750 tx=0x7f11e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8017070 con 0x7f11f010bf50 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e80054d0 con 0x7f11f010bf50 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f01a2810 con 0x7f11f010bf50 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8004c60 con 0x7f11f010bf50 2024-09-21T06:11:16.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.649+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f01a8fa0 con 0x7f11f010bf50 2024-09-21T06:11:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.651+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f0106f40 con 0x7f11f010bf50 2024-09-21T06:11:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.652+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f11e8009bd0 con 0x7f11f010bf50 2024-09-21T06:11:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.653+0000 7f11edffb640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11dc0b6560 0x7f11dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.653+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f11e80e6db0 con 0x7f11f010bf50 2024-09-21T06:11:16.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.653+0000 7f11f4dcc640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11dc0b6560 0x7f11dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:16.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.653+0000 7f11f4dcc640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11dc0b6560 0x7f11dc0b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005d00 tx=0x7f11d800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:16.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.657+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f11e80ad560 con 0x7f11f010bf50 2024-09-21T06:11:16.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.844+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11f0063800 con 0x7f11dc0b6560 2024-09-21T06:11:16.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.849+0000 7f11edffb640 1 -- 172.21.15.31:0/2804191291 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7f11f0063800 con 0x7f11dc0b6560 2024-09-21T06:11:16.854 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:16.856 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:11:16.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11dc0b6560 msgr2=0x7f11dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11dc0b6560 0x7f11dc0b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11d8005d00 tx=0x7f11d800b410 comp rx=0 tx=0).stop 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 msgr2=0x7f11f01a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 0x7f11f01a1e20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f11e800d750 tx=0x7f11e800dc10 comp rx=0 tx=0).stop 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 shutdown_connections 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11dc0b6560 0x7f11dc0b8a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11f010bf50 0x7f11f01a1e20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f010b600 0x7f11f019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 --2- 172.21.15.31:0/2804191291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11f0105cd0 0x7f11f019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 >> 172.21.15.31:0/2804191291 conn(0x7f11f01016f0 msgr2=0x7f11f0103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 shutdown_connections 2024-09-21T06:11:16.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:16.853+0000 7f11f7030640 1 -- 172.21.15.31:0/2804191291 wait complete. 2024-09-21T06:11:16.866 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-21T06:11:15.636892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658914,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":278,"num_read_kb":235,"num_write":216,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":668,"ondisk_log_size":668,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381940,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559884,"statfs":{"total":1151940427776,"available":1151549321216,"internally_reserved":0,"allocated":12177408,"data_stored":6964818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":557,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020756"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027515+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:11:06.027515+0000","last_peered":"2024-09-21T06:11:06.027515+0000","last_clean":"2024-09-21T06:11:06.027515+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:11:06.027515+0000","last_undegraded":"2024-09-21T06:11:06.027515+0000","last_fullsized":"2024-09-21T06:11:06.027515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.015871+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:11:06.015871+0000","last_peered":"2024-09-21T06:11:06.015871+0000","last_clean":"2024-09-21T06:11:06.015871+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:11:06.015871+0000","last_undegraded":"2024-09-21T06:11:06.015871+0000","last_fullsized":"2024-09-21T06:11:06.015871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026215+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:11:06.026215+0000","last_peered":"2024-09-21T06:11:06.026215+0000","last_clean":"2024-09-21T06:11:06.026215+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:11:06.026215+0000","last_undegraded":"2024-09-21T06:11:06.026215+0000","last_fullsized":"2024-09-21T06:11:06.026215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023448+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:11:06.023448+0000","last_peered":"2024-09-21T06:11:06.023448+0000","last_clean":"2024-09-21T06:11:06.023448+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:11:06.023448+0000","last_undegraded":"2024-09-21T06:11:06.023448+0000","last_fullsized":"2024-09-21T06:11:06.023448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.995137+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:11:05.995137+0000","last_peered":"2024-09-21T06:11:05.995137+0000","last_clean":"2024-09-21T06:11:05.995137+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:11:05.995137+0000","last_undegraded":"2024-09-21T06:11:05.995137+0000","last_fullsized":"2024-09-21T06:11:05.995137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021515+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:11:06.021515+0000","last_peered":"2024-09-21T06:11:06.021515+0000","last_clean":"2024-09-21T06:11:06.021515+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:11:06.021515+0000","last_undegraded":"2024-09-21T06:11:06.021515+0000","last_fullsized":"2024-09-21T06:11:06.021515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.002074+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:11:06.002074+0000","last_peered":"2024-09-21T06:11:06.002074+0000","last_clean":"2024-09-21T06:11:06.002074+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:11:06.002074+0000","last_undegraded":"2024-09-21T06:11:06.002074+0000","last_fullsized":"2024-09-21T06:11:06.002074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.008396+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:11:06.008396+0000","last_peered":"2024-09-21T06:11:06.008396+0000","last_clean":"2024-09-21T06:11:06.008396+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:11:06.008396+0000","last_undegraded":"2024-09-21T06:11:06.008396+0000","last_fullsized":"2024-09-21T06:11:06.008396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026878+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:11:06.026878+0000","last_peered":"2024-09-21T06:11:06.026878+0000","last_clean":"2024-09-21T06:11:06.026878+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:11:06.026878+0000","last_undegraded":"2024-09-21T06:11:06.026878+0000","last_fullsized":"2024-09-21T06:11:06.026878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021546+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:11:06.021546+0000","last_peered":"2024-09-21T06:11:06.021546+0000","last_clean":"2024-09-21T06:11:06.021546+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:11:06.021546+0000","last_undegraded":"2024-09-21T06:11:06.021546+0000","last_fullsized":"2024-09-21T06:11:06.021546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016861+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:11:06.016861+0000","last_peered":"2024-09-21T06:11:06.016861+0000","last_clean":"2024-09-21T06:11:06.016861+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:11:06.016861+0000","last_undegraded":"2024-09-21T06:11:06.016861+0000","last_fullsized":"2024-09-21T06:11:06.016861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023671+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:11:06.023671+0000","last_peered":"2024-09-21T06:11:06.023671+0000","last_clean":"2024-09-21T06:11:06.023671+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:11:06.023671+0000","last_undegraded":"2024-09-21T06:11:06.023671+0000","last_fullsized":"2024-09-21T06:11:06.023671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011521+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:11:06.011521+0000","last_peered":"2024-09-21T06:11:06.011521+0000","last_clean":"2024-09-21T06:11:06.011521+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:11:06.011521+0000","last_undegraded":"2024-09-21T06:11:06.011521+0000","last_fullsized":"2024-09-21T06:11:06.011521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.020387+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:11:06.020387+0000","last_peered":"2024-09-21T06:11:06.020387+0000","last_clean":"2024-09-21T06:11:06.020387+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:11:06.020387+0000","last_undegraded":"2024-09-21T06:11:06.020387+0000","last_fullsized":"2024-09-21T06:11:06.020387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021325+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:11:06.021325+0000","last_peered":"2024-09-21T06:11:06.021325+0000","last_clean":"2024-09-21T06:11:06.021325+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:11:06.021325+0000","last_undegraded":"2024-09-21T06:11:06.021325+0000","last_fullsized":"2024-09-21T06:11:06.021325+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027177+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:11:06.027177+0000","last_peered":"2024-09-21T06:11:06.027177+0000","last_clean":"2024-09-21T06:11:06.027177+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:11:06.027177+0000","last_undegraded":"2024-09-21T06:11:06.027177+0000","last_fullsized":"2024-09-21T06:11:06.027177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027659+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:11:06.027659+0000","last_peered":"2024-09-21T06:11:06.027659+0000","last_clean":"2024-09-21T06:11:06.027659+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:11:06.027659+0000","last_undegraded":"2024-09-21T06:11:06.027659+0000","last_fullsized":"2024-09-21T06:11:06.027659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.012685+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:11:06.012685+0000","last_peered":"2024-09-21T06:11:06.012685+0000","last_clean":"2024-09-21T06:11:06.012685+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:11:06.012685+0000","last_undegraded":"2024-09-21T06:11:06.012685+0000","last_fullsized":"2024-09-21T06:11:06.012685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016804+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:11:06.016804+0000","last_peered":"2024-09-21T06:11:06.016804+0000","last_clean":"2024-09-21T06:11:06.016804+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:11:06.016804+0000","last_undegraded":"2024-09-21T06:11:06.016804+0000","last_fullsized":"2024-09-21T06:11:06.016804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023293+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:11:06.023293+0000","last_peered":"2024-09-21T06:11:06.023293+0000","last_clean":"2024-09-21T06:11:06.023293+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:11:06.023293+0000","last_undegraded":"2024-09-21T06:11:06.023293+0000","last_fullsized":"2024-09-21T06:11:06.023293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016651+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:11:06.016651+0000","last_peered":"2024-09-21T06:11:06.016651+0000","last_clean":"2024-09-21T06:11:06.016651+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:11:06.016651+0000","last_undegraded":"2024-09-21T06:11:06.016651+0000","last_fullsized":"2024-09-21T06:11:06.016651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027496+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:11:06.027496+0000","last_peered":"2024-09-21T06:11:06.027496+0000","last_clean":"2024-09-21T06:11:06.027496+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:11:06.027496+0000","last_undegraded":"2024-09-21T06:11:06.027496+0000","last_fullsized":"2024-09-21T06:11:06.027496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016584+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:11:06.016584+0000","last_peered":"2024-09-21T06:11:06.016584+0000","last_clean":"2024-09-21T06:11:06.016584+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:11:06.016584+0000","last_undegraded":"2024-09-21T06:11:06.016584+0000","last_fullsized":"2024-09-21T06:11:06.016584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.007034+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:11:06.007034+0000","last_peered":"2024-09-21T06:11:06.007034+0000","last_clean":"2024-09-21T06:11:06.007034+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:11:06.007034+0000","last_undegraded":"2024-09-21T06:11:06.007034+0000","last_fullsized":"2024-09-21T06:11:06.007034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.012729+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:11:06.012729+0000","last_peered":"2024-09-21T06:11:06.012729+0000","last_clean":"2024-09-21T06:11:06.012729+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:11:06.012729+0000","last_undegraded":"2024-09-21T06:11:06.012729+0000","last_fullsized":"2024-09-21T06:11:06.012729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027364+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:11:06.027364+0000","last_peered":"2024-09-21T06:11:06.027364+0000","last_clean":"2024-09-21T06:11:06.027364+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:11:06.027364+0000","last_undegraded":"2024-09-21T06:11:06.027364+0000","last_fullsized":"2024-09-21T06:11:06.027364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.004164+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:11:06.004164+0000","last_peered":"2024-09-21T06:11:06.004164+0000","last_clean":"2024-09-21T06:11:06.004164+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:11:06.004164+0000","last_undegraded":"2024-09-21T06:11:06.004164+0000","last_fullsized":"2024-09-21T06:11:06.004164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.017987+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:11:06.017987+0000","last_peered":"2024-09-21T06:11:06.017987+0000","last_clean":"2024-09-21T06:11:06.017987+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:11:06.017987+0000","last_undegraded":"2024-09-21T06:11:06.017987+0000","last_fullsized":"2024-09-21T06:11:06.017987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023167+0000","last_change":"2024-09-21T06:10:07.403662+0000","last_active":"2024-09-21T06:11:06.023167+0000","last_peered":"2024-09-21T06:11:06.023167+0000","last_clean":"2024-09-21T06:11:06.023167+0000","last_became_active":"2024-09-21T06:10:07.403220+0000","last_became_peered":"2024-09-21T06:10:07.403220+0000","last_unstale":"2024-09-21T06:11:06.023167+0000","last_undegraded":"2024-09-21T06:11:06.023167+0000","last_fullsized":"2024-09-21T06:11:06.023167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:42.670948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021026+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:11:06.021026+0000","last_peered":"2024-09-21T06:11:06.021026+0000","last_clean":"2024-09-21T06:11:06.021026+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:11:06.021026+0000","last_undegraded":"2024-09-21T06:11:06.021026+0000","last_fullsized":"2024-09-21T06:11:06.021026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.010605+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:11:06.010605+0000","last_peered":"2024-09-21T06:11:06.010605+0000","last_clean":"2024-09-21T06:11:06.010605+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:11:06.010605+0000","last_undegraded":"2024-09-21T06:11:06.010605+0000","last_fullsized":"2024-09-21T06:11:06.010605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.006692+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:11:06.006692+0000","last_peered":"2024-09-21T06:11:06.006692+0000","last_clean":"2024-09-21T06:11:06.006692+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:11:06.006692+0000","last_undegraded":"2024-09-21T06:11:06.006692+0000","last_fullsized":"2024-09-21T06:11:06.006692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.006132+0000","last_change":"2024-09-21T06:10:07.399633+0000","last_active":"2024-09-21T06:11:06.006132+0000","last_peered":"2024-09-21T06:11:06.006132+0000","last_clean":"2024-09-21T06:11:06.006132+0000","last_became_active":"2024-09-21T06:10:07.399200+0000","last_became_peered":"2024-09-21T06:10:07.399200+0000","last_unstale":"2024-09-21T06:11:06.006132+0000","last_undegraded":"2024-09-21T06:11:06.006132+0000","last_fullsized":"2024-09-21T06:11:06.006132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:16.830371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398713+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:10:07.398713+0000","last_peered":"2024-09-21T06:10:07.398713+0000","last_clean":"2024-09-21T06:10:07.398713+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:10:07.398713+0000","last_undegraded":"2024-09-21T06:10:07.398713+0000","last_fullsized":"2024-09-21T06:10:07.398713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016929+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:11:06.016929+0000","last_peered":"2024-09-21T06:11:06.016929+0000","last_clean":"2024-09-21T06:11:06.016929+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:11:06.016929+0000","last_undegraded":"2024-09-21T06:11:06.016929+0000","last_fullsized":"2024-09-21T06:11:06.016929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587183+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:10:07.587183+0000","last_peered":"2024-09-21T06:10:07.587183+0000","last_clean":"2024-09-21T06:10:07.587183+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:10:07.587183+0000","last_undegraded":"2024-09-21T06:10:07.587183+0000","last_fullsized":"2024-09-21T06:10:07.587183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396798+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:10:07.396798+0000","last_peered":"2024-09-21T06:10:07.396798+0000","last_clean":"2024-09-21T06:10:07.396798+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:10:07.396798+0000","last_undegraded":"2024-09-21T06:10:07.396798+0000","last_fullsized":"2024-09-21T06:10:07.396798+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.020012+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:11:06.020012+0000","last_peered":"2024-09-21T06:11:06.020012+0000","last_clean":"2024-09-21T06:11:06.020012+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:11:06.020012+0000","last_undegraded":"2024-09-21T06:11:06.020012+0000","last_fullsized":"2024-09-21T06:11:06.020012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398807+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:10:07.398807+0000","last_peered":"2024-09-21T06:10:07.398807+0000","last_clean":"2024-09-21T06:10:07.398807+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:10:07.398807+0000","last_undegraded":"2024-09-21T06:10:07.398807+0000","last_fullsized":"2024-09-21T06:10:07.398807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011649+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:11:06.011649+0000","last_peered":"2024-09-21T06:11:06.011649+0000","last_clean":"2024-09-21T06:11:06.011649+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:11:06.011649+0000","last_undegraded":"2024-09-21T06:11:06.011649+0000","last_fullsized":"2024-09-21T06:11:06.011649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587232+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:10:07.587232+0000","last_peered":"2024-09-21T06:10:07.587232+0000","last_clean":"2024-09-21T06:10:07.587232+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:10:07.587232+0000","last_undegraded":"2024-09-21T06:10:07.587232+0000","last_fullsized":"2024-09-21T06:10:07.587232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.014591+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:11:06.014591+0000","last_peered":"2024-09-21T06:11:06.014591+0000","last_clean":"2024-09-21T06:11:06.014591+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:11:06.014591+0000","last_undegraded":"2024-09-21T06:11:06.014591+0000","last_fullsized":"2024-09-21T06:11:06.014591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395089+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:10:07.395089+0000","last_peered":"2024-09-21T06:10:07.395089+0000","last_clean":"2024-09-21T06:10:07.395089+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:10:07.395089+0000","last_undegraded":"2024-09-21T06:10:07.395089+0000","last_fullsized":"2024-09-21T06:10:07.395089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022625+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:11:06.022625+0000","last_peered":"2024-09-21T06:11:06.022625+0000","last_clean":"2024-09-21T06:11:06.022625+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:11:06.022625+0000","last_undegraded":"2024-09-21T06:11:06.022625+0000","last_fullsized":"2024-09-21T06:11:06.022625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399080+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:10:07.399080+0000","last_peered":"2024-09-21T06:10:07.399080+0000","last_clean":"2024-09-21T06:10:07.399080+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:10:07.399080+0000","last_undegraded":"2024-09-21T06:10:07.399080+0000","last_fullsized":"2024-09-21T06:10:07.399080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.028117+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:11:06.028117+0000","last_peered":"2024-09-21T06:11:06.028117+0000","last_clean":"2024-09-21T06:11:06.028117+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:11:06.028117+0000","last_undegraded":"2024-09-21T06:11:06.028117+0000","last_fullsized":"2024-09-21T06:11:06.028117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'31","reported_seq":247,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:14.466629+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:11:14.466629+0000","last_peered":"2024-09-21T06:11:14.466629+0000","last_clean":"2024-09-21T06:11:14.466629+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:11:14.466629+0000","last_undegraded":"2024-09-21T06:11:14.466629+0000","last_fullsized":"2024-09-21T06:11:14.466629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67514,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":162,"num_write":31,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.997726+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:11:05.997726+0000","last_peered":"2024-09-21T06:11:05.997726+0000","last_clean":"2024-09-21T06:11:05.997726+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:11:05.997726+0000","last_undegraded":"2024-09-21T06:11:05.997726+0000","last_fullsized":"2024-09-21T06:11:05.997726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394811+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:10:07.394811+0000","last_peered":"2024-09-21T06:10:07.394811+0000","last_clean":"2024-09-21T06:10:07.394811+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:10:07.394811+0000","last_undegraded":"2024-09-21T06:10:07.394811+0000","last_fullsized":"2024-09-21T06:10:07.394811+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019305+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:11:06.019305+0000","last_peered":"2024-09-21T06:11:06.019305+0000","last_clean":"2024-09-21T06:11:06.019305+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:11:06.019305+0000","last_undegraded":"2024-09-21T06:11:06.019305+0000","last_fullsized":"2024-09-21T06:11:06.019305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398372+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:10:07.398372+0000","last_peered":"2024-09-21T06:10:07.398372+0000","last_clean":"2024-09-21T06:10:07.398372+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:10:07.398372+0000","last_undegraded":"2024-09-21T06:10:07.398372+0000","last_fullsized":"2024-09-21T06:10:07.398372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019906+0000","last_change":"2024-09-21T06:10:07.401185+0000","last_active":"2024-09-21T06:11:06.019906+0000","last_peered":"2024-09-21T06:11:06.019906+0000","last_clean":"2024-09-21T06:11:06.019906+0000","last_became_active":"2024-09-21T06:10:07.400789+0000","last_became_peered":"2024-09-21T06:10:07.400789+0000","last_unstale":"2024-09-21T06:11:06.019906+0000","last_undegraded":"2024-09-21T06:11:06.019906+0000","last_fullsized":"2024-09-21T06:11:06.019906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:03.201239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394895+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:10:07.394895+0000","last_peered":"2024-09-21T06:10:07.394895+0000","last_clean":"2024-09-21T06:10:07.394895+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:10:07.394895+0000","last_undegraded":"2024-09-21T06:10:07.394895+0000","last_fullsized":"2024-09-21T06:10:07.394895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018842+0000","last_change":"2024-09-21T06:10:07.588201+0000","last_active":"2024-09-21T06:11:06.018842+0000","last_peered":"2024-09-21T06:11:06.018842+0000","last_clean":"2024-09-21T06:11:06.018842+0000","last_became_active":"2024-09-21T06:10:07.587788+0000","last_became_peered":"2024-09-21T06:10:07.587788+0000","last_unstale":"2024-09-21T06:11:06.018842+0000","last_undegraded":"2024-09-21T06:11:06.018842+0000","last_fullsized":"2024-09-21T06:11:06.018842+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:16.617435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394307+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:10:07.394307+0000","last_peered":"2024-09-21T06:10:07.394307+0000","last_clean":"2024-09-21T06:10:07.394307+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:10:07.394307+0000","last_undegraded":"2024-09-21T06:10:07.394307+0000","last_fullsized":"2024-09-21T06:10:07.394307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018670+0000","last_change":"2024-09-21T06:10:07.478289+0000","last_active":"2024-09-21T06:11:06.018670+0000","last_peered":"2024-09-21T06:11:06.018670+0000","last_clean":"2024-09-21T06:11:06.018670+0000","last_became_active":"2024-09-21T06:10:07.477828+0000","last_became_peered":"2024-09-21T06:10:07.477828+0000","last_unstale":"2024-09-21T06:11:06.018670+0000","last_undegraded":"2024-09-21T06:11:06.018670+0000","last_fullsized":"2024-09-21T06:11:06.018670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.721612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395262+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:10:07.395262+0000","last_peered":"2024-09-21T06:10:07.395262+0000","last_clean":"2024-09-21T06:10:07.395262+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:10:07.395262+0000","last_undegraded":"2024-09-21T06:10:07.395262+0000","last_fullsized":"2024-09-21T06:10:07.395262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022924+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:11:06.022924+0000","last_peered":"2024-09-21T06:11:06.022924+0000","last_clean":"2024-09-21T06:11:06.022924+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:11:06.022924+0000","last_undegraded":"2024-09-21T06:11:06.022924+0000","last_fullsized":"2024-09-21T06:11:06.022924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406063+0000","last_change":"2024-09-21T06:10:07.405946+0000","last_active":"2024-09-21T06:10:07.406063+0000","last_peered":"2024-09-21T06:10:07.406063+0000","last_clean":"2024-09-21T06:10:07.406063+0000","last_became_active":"2024-09-21T06:10:07.401620+0000","last_became_peered":"2024-09-21T06:10:07.401620+0000","last_unstale":"2024-09-21T06:10:07.406063+0000","last_undegraded":"2024-09-21T06:10:07.406063+0000","last_fullsized":"2024-09-21T06:10:07.406063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:17.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021136+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:11:06.021136+0000","last_peered":"2024-09-21T06:11:06.021136+0000","last_clean":"2024-09-21T06:11:06.021136+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:11:06.021136+0000","last_undegraded":"2024-09-21T06:11:06.021136+0000","last_fullsized":"2024-09-21T06:11:06.021136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399215+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:10:07.399215+0000","last_peered":"2024-09-21T06:10:07.399215+0000","last_clean":"2024-09-21T06:10:07.399215+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:10:07.399215+0000","last_undegraded":"2024-09-21T06:10:07.399215+0000","last_fullsized":"2024-09-21T06:10:07.399215+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.013524+0000","last_change":"2024-09-21T06:10:07.406892+0000","last_active":"2024-09-21T06:11:06.013524+0000","last_peered":"2024-09-21T06:11:06.013524+0000","last_clean":"2024-09-21T06:11:06.013524+0000","last_became_active":"2024-09-21T06:10:07.402546+0000","last_became_peered":"2024-09-21T06:10:07.402546+0000","last_unstale":"2024-09-21T06:11:06.013524+0000","last_undegraded":"2024-09-21T06:11:06.013524+0000","last_fullsized":"2024-09-21T06:11:06.013524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.081510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394662+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:10:07.394662+0000","last_peered":"2024-09-21T06:10:07.394662+0000","last_clean":"2024-09-21T06:10:07.394662+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:10:07.394662+0000","last_undegraded":"2024-09-21T06:10:07.394662+0000","last_fullsized":"2024-09-21T06:10:07.394662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019250+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:11:06.019250+0000","last_peered":"2024-09-21T06:11:06.019250+0000","last_clean":"2024-09-21T06:11:06.019250+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:11:06.019250+0000","last_undegraded":"2024-09-21T06:11:06.019250+0000","last_fullsized":"2024-09-21T06:11:06.019250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395834+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:10:07.395834+0000","last_peered":"2024-09-21T06:10:07.395834+0000","last_clean":"2024-09-21T06:10:07.395834+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:10:07.395834+0000","last_undegraded":"2024-09-21T06:10:07.395834+0000","last_fullsized":"2024-09-21T06:10:07.395834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395073+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:10:07.395073+0000","last_peered":"2024-09-21T06:10:07.395073+0000","last_clean":"2024-09-21T06:10:07.395073+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:10:07.395073+0000","last_undegraded":"2024-09-21T06:10:07.395073+0000","last_fullsized":"2024-09-21T06:10:07.395073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.005537+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:11:06.005537+0000","last_peered":"2024-09-21T06:11:06.005537+0000","last_clean":"2024-09-21T06:11:06.005537+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:11:06.005537+0000","last_undegraded":"2024-09-21T06:11:06.005537+0000","last_fullsized":"2024-09-21T06:11:06.005537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394546+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:10:07.394546+0000","last_peered":"2024-09-21T06:10:07.394546+0000","last_clean":"2024-09-21T06:10:07.394546+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:10:07.394546+0000","last_undegraded":"2024-09-21T06:10:07.394546+0000","last_fullsized":"2024-09-21T06:10:07.394546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011182+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:11:06.011182+0000","last_peered":"2024-09-21T06:11:06.011182+0000","last_clean":"2024-09-21T06:11:06.011182+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:11:06.011182+0000","last_undegraded":"2024-09-21T06:11:06.011182+0000","last_fullsized":"2024-09-21T06:11:06.011182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396950+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:10:07.396950+0000","last_peered":"2024-09-21T06:10:07.396950+0000","last_clean":"2024-09-21T06:10:07.396950+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:10:07.396950+0000","last_undegraded":"2024-09-21T06:10:07.396950+0000","last_fullsized":"2024-09-21T06:10:07.396950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.015296+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:11:06.015296+0000","last_peered":"2024-09-21T06:11:06.015296+0000","last_clean":"2024-09-21T06:11:06.015296+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:11:06.015296+0000","last_undegraded":"2024-09-21T06:11:06.015296+0000","last_fullsized":"2024-09-21T06:11:06.015296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394998+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:10:07.394998+0000","last_peered":"2024-09-21T06:10:07.394998+0000","last_clean":"2024-09-21T06:10:07.394998+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:10:07.394998+0000","last_undegraded":"2024-09-21T06:10:07.394998+0000","last_fullsized":"2024-09-21T06:10:07.394998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018487+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:11:06.018487+0000","last_peered":"2024-09-21T06:11:06.018487+0000","last_clean":"2024-09-21T06:11:06.018487+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:11:06.018487+0000","last_undegraded":"2024-09-21T06:11:06.018487+0000","last_fullsized":"2024-09-21T06:11:06.018487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401748+0000","last_change":"2024-09-21T06:10:07.401640+0000","last_active":"2024-09-21T06:10:07.401748+0000","last_peered":"2024-09-21T06:10:07.401748+0000","last_clean":"2024-09-21T06:10:07.401748+0000","last_became_active":"2024-09-21T06:10:07.401242+0000","last_became_peered":"2024-09-21T06:10:07.401242+0000","last_unstale":"2024-09-21T06:10:07.401748+0000","last_undegraded":"2024-09-21T06:10:07.401748+0000","last_fullsized":"2024-09-21T06:10:07.401748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:25.335694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.004481+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:11:06.004481+0000","last_peered":"2024-09-21T06:11:06.004481+0000","last_clean":"2024-09-21T06:11:06.004481+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:11:06.004481+0000","last_undegraded":"2024-09-21T06:11:06.004481+0000","last_fullsized":"2024-09-21T06:11:06.004481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395763+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:10:07.395763+0000","last_peered":"2024-09-21T06:10:07.395763+0000","last_clean":"2024-09-21T06:10:07.395763+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:10:07.395763+0000","last_undegraded":"2024-09-21T06:10:07.395763+0000","last_fullsized":"2024-09-21T06:10:07.395763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.010167+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:11:06.010167+0000","last_peered":"2024-09-21T06:11:06.010167+0000","last_clean":"2024-09-21T06:11:06.010167+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:11:06.010167+0000","last_undegraded":"2024-09-21T06:11:06.010167+0000","last_fullsized":"2024-09-21T06:11:06.010167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'11","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.300858+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:11:06.300858+0000","last_peered":"2024-09-21T06:11:06.300858+0000","last_clean":"2024-09-21T06:11:06.300858+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:11:06.300858+0000","last_undegraded":"2024-09-21T06:11:06.300858+0000","last_fullsized":"2024-09-21T06:11:06.300858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022143+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:11:06.022143+0000","last_peered":"2024-09-21T06:11:06.022143+0000","last_clean":"2024-09-21T06:11:06.022143+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:11:06.022143+0000","last_undegraded":"2024-09-21T06:11:06.022143+0000","last_fullsized":"2024-09-21T06:11:06.022143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395509+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:10:07.395509+0000","last_peered":"2024-09-21T06:10:07.395509+0000","last_clean":"2024-09-21T06:10:07.395509+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:10:07.395509+0000","last_undegraded":"2024-09-21T06:10:07.395509+0000","last_fullsized":"2024-09-21T06:10:07.395509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019541+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:11:06.019541+0000","last_peered":"2024-09-21T06:11:06.019541+0000","last_clean":"2024-09-21T06:11:06.019541+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:11:06.019541+0000","last_undegraded":"2024-09-21T06:11:06.019541+0000","last_fullsized":"2024-09-21T06:11:06.019541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476406+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:10:07.476406+0000","last_peered":"2024-09-21T06:10:07.476406+0000","last_clean":"2024-09-21T06:10:07.476406+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:10:07.476406+0000","last_undegraded":"2024-09-21T06:10:07.476406+0000","last_fullsized":"2024-09-21T06:10:07.476406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.970288+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:11:05.970288+0000","last_peered":"2024-09-21T06:11:05.970288+0000","last_clean":"2024-09-21T06:11:05.970288+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:11:05.970288+0000","last_undegraded":"2024-09-21T06:11:05.970288+0000","last_fullsized":"2024-09-21T06:11:05.970288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393700+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:10:07.393700+0000","last_peered":"2024-09-21T06:10:07.393700+0000","last_clean":"2024-09-21T06:10:07.393700+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:10:07.393700+0000","last_undegraded":"2024-09-21T06:10:07.393700+0000","last_fullsized":"2024-09-21T06:10:07.393700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.008331+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:11:06.008331+0000","last_peered":"2024-09-21T06:11:06.008331+0000","last_clean":"2024-09-21T06:11:06.008331+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:11:06.008331+0000","last_undegraded":"2024-09-21T06:11:06.008331+0000","last_fullsized":"2024-09-21T06:11:06.008331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394466+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:10:07.394466+0000","last_peered":"2024-09-21T06:10:07.394466+0000","last_clean":"2024-09-21T06:10:07.394466+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:10:07.394466+0000","last_undegraded":"2024-09-21T06:10:07.394466+0000","last_fullsized":"2024-09-21T06:10:07.394466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018378+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:11:06.018378+0000","last_peered":"2024-09-21T06:11:06.018378+0000","last_clean":"2024-09-21T06:11:06.018378+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:11:06.018378+0000","last_undegraded":"2024-09-21T06:11:06.018378+0000","last_fullsized":"2024-09-21T06:11:06.018378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403122+0000","last_change":"2024-09-21T06:10:07.402379+0000","last_active":"2024-09-21T06:10:07.403122+0000","last_peered":"2024-09-21T06:10:07.403122+0000","last_clean":"2024-09-21T06:10:07.403122+0000","last_became_active":"2024-09-21T06:10:07.396828+0000","last_became_peered":"2024-09-21T06:10:07.396828+0000","last_unstale":"2024-09-21T06:10:07.403122+0000","last_undegraded":"2024-09-21T06:10:07.403122+0000","last_fullsized":"2024-09-21T06:10:07.403122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:26.181343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026247+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:11:06.026247+0000","last_peered":"2024-09-21T06:11:06.026247+0000","last_clean":"2024-09-21T06:11:06.026247+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:11:06.026247+0000","last_undegraded":"2024-09-21T06:11:06.026247+0000","last_fullsized":"2024-09-21T06:11:06.026247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022587+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:11:06.022587+0000","last_peered":"2024-09-21T06:11:06.022587+0000","last_clean":"2024-09-21T06:11:06.022587+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:11:06.022587+0000","last_undegraded":"2024-09-21T06:11:06.022587+0000","last_fullsized":"2024-09-21T06:11:06.022587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395391+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:10:07.395391+0000","last_peered":"2024-09-21T06:10:07.395391+0000","last_clean":"2024-09-21T06:10:07.395391+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:10:07.395391+0000","last_undegraded":"2024-09-21T06:10:07.395391+0000","last_fullsized":"2024-09-21T06:10:07.395391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011566+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:11:06.011566+0000","last_peered":"2024-09-21T06:11:06.011566+0000","last_clean":"2024-09-21T06:11:06.011566+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:11:06.011566+0000","last_undegraded":"2024-09-21T06:11:06.011566+0000","last_fullsized":"2024-09-21T06:11:06.011566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395635+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:10:07.395635+0000","last_peered":"2024-09-21T06:10:07.395635+0000","last_clean":"2024-09-21T06:10:07.395635+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:10:07.395635+0000","last_undegraded":"2024-09-21T06:10:07.395635+0000","last_fullsized":"2024-09-21T06:10:07.395635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026621+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:11:06.026621+0000","last_peered":"2024-09-21T06:11:06.026621+0000","last_clean":"2024-09-21T06:11:06.026621+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:11:06.026621+0000","last_undegraded":"2024-09-21T06:11:06.026621+0000","last_fullsized":"2024-09-21T06:11:06.026621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394530+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:10:07.394530+0000","last_peered":"2024-09-21T06:10:07.394530+0000","last_clean":"2024-09-21T06:10:07.394530+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:10:07.394530+0000","last_undegraded":"2024-09-21T06:10:07.394530+0000","last_fullsized":"2024-09-21T06:10:07.394530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.013863+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:11:06.013863+0000","last_peered":"2024-09-21T06:11:06.013863+0000","last_clean":"2024-09-21T06:11:06.013863+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:11:06.013863+0000","last_undegraded":"2024-09-21T06:11:06.013863+0000","last_fullsized":"2024-09-21T06:11:06.013863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399889+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:10:07.399889+0000","last_peered":"2024-09-21T06:10:07.399889+0000","last_clean":"2024-09-21T06:10:07.399889+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:10:07.399889+0000","last_undegraded":"2024-09-21T06:10:07.399889+0000","last_fullsized":"2024-09-21T06:10:07.399889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68312,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":197,"num_read_kb":166,"num_write":91,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":839680,"data_stored":415673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1527808,"data_stored":1073866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":416372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266736,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":416277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1441792,"data_stored":1006740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1449984,"data_stored":1006340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":415894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378702,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":483221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":483187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:11:16.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:11:17.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 -- 172.21.15.31:0/353672796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4105260 msgr2=0x7fedd410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 --2- 172.21.15.31:0/353672796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4105260 0x7fedd410d7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009900 tx=0x7fedc802a410 comp rx=0 tx=0).stop 2024-09-21T06:11:17.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 -- 172.21.15.31:0/353672796 shutdown_connections 2024-09-21T06:11:17.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 --2- 172.21.15.31:0/353672796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fedd410dd70 0x7fedd4110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 --2- 172.21.15.31:0/353672796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd4105260 0x7fedd410d7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 --2- 172.21.15.31:0/353672796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedd4104910 0x7fedd4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 -- 172.21.15.31:0/353672796 >> 172.21.15.31:0/353672796 conn(0x7fedd407a670 msgr2=0x7fedd407aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 -- 172.21.15.31:0/353672796 shutdown_connections 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 -- 172.21.15.31:0/353672796 wait complete. 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 Processor -- start 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.021+0000 7feddacbd640 1 -- start start 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7feddacbd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedd4104910 0x7fedd419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7feddacbd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fedd4105260 0x7fedd419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7feddacbd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 0x7fedd419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7feddacbd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41139d0 con 0x7fedd410dd70 2024-09-21T06:11:17.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7feddacbd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4113850 con 0x7fedd4104910 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7feddacbd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4113b50 con 0x7fedd4105260 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 0x7fedd419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 0x7fedd419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39612/0 (socket says 172.21.15.31:39612) 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 -- 172.21.15.31:0/2614760935 learned_addr learned my addr 172.21.15.31:0/2614760935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd3fff640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fedd4105260 0x7fedd419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd8a59640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedd4104910 0x7fedd419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 -- 172.21.15.31:0/2614760935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fedd4105260 msgr2=0x7fedd419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fedd4105260 0x7fedd419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 -- 172.21.15.31:0/2614760935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedd4104910 msgr2=0x7fedd419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedd4104910 0x7fedd419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 -- 172.21.15.31:0/2614760935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41a0360 con 0x7fedd410dd70 2024-09-21T06:11:17.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.022+0000 7fedd925a640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 0x7fedd419fc60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00eaa0 tx=0x7fedcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:17.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.023+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00cdb0 con 0x7fedd410dd70 2024-09-21T06:11:17.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.023+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedcc005020 con 0x7fedd410dd70 2024-09-21T06:11:17.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.023+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a0650 con 0x7fedd410dd70 2024-09-21T06:11:17.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.023+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0116d0 con 0x7fedd410dd70 2024-09-21T06:11:17.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.023+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41a6de0 con 0x7fedd410dd70 2024-09-21T06:11:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.024+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd4104d20 con 0x7fedd410dd70 2024-09-21T06:11:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.025+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fedcc0051d0 con 0x7fedd410dd70 2024-09-21T06:11:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.026+0000 7fedd1ffb640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fedbc0b6560 0x7fedbc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.027+0000 7fedd8a59640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fedbc0b6560 0x7fedbc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.027+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fedcc0e79a0 con 0x7fedd410dd70 2024-09-21T06:11:17.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.027+0000 7fedd8a59640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fedbc0b6560 0x7fedbc0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005d00 tx=0x7fedc4005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:17.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.030+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fedcc016030 con 0x7fedd410dd70 2024-09-21T06:11:17.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:16 smithi031 ceph-mon[31237]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 425 B/s wr, 3 op/s 2024-09-21T06:11:17.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.216+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedd4100bc0 con 0x7fedbc0b6560 2024-09-21T06:11:17.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.220+0000 7fedd1ffb640 1 -- 172.21.15.31:0/2614760935 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7fedd4100bc0 con 0x7fedbc0b6560 2024-09-21T06:11:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:17.226 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:11:17.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fedbc0b6560 msgr2=0x7fedbc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fedbc0b6560 0x7fedbc0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fedc4005d00 tx=0x7fedc4005c50 comp rx=0 tx=0).stop 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 msgr2=0x7fedd419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 0x7fedd419fc60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00eaa0 tx=0x7fedcc00ef60 comp rx=0 tx=0).stop 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 shutdown_connections 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fedbc0b6560 0x7fedbc0b8a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd410dd70 0x7fedd419fc60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fedd4105260 0x7fedd419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 --2- 172.21.15.31:0/2614760935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedd4104910 0x7fedd419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.223+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 >> 172.21.15.31:0/2614760935 conn(0x7fedd407a670 msgr2=0x7fedd40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.224+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 shutdown_connections 2024-09-21T06:11:17.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.224+0000 7feddacbd640 1 -- 172.21.15.31:0/2614760935 wait complete. 2024-09-21T06:11:17.237 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-21T06:11:15.636892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658914,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":278,"num_read_kb":235,"num_write":216,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":668,"ondisk_log_size":668,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381940,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559884,"statfs":{"total":1151940427776,"available":1151549321216,"internally_reserved":0,"allocated":12177408,"data_stored":6964818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":557,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020756"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027515+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:11:06.027515+0000","last_peered":"2024-09-21T06:11:06.027515+0000","last_clean":"2024-09-21T06:11:06.027515+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:11:06.027515+0000","last_undegraded":"2024-09-21T06:11:06.027515+0000","last_fullsized":"2024-09-21T06:11:06.027515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.015871+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:11:06.015871+0000","last_peered":"2024-09-21T06:11:06.015871+0000","last_clean":"2024-09-21T06:11:06.015871+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:11:06.015871+0000","last_undegraded":"2024-09-21T06:11:06.015871+0000","last_fullsized":"2024-09-21T06:11:06.015871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026215+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:11:06.026215+0000","last_peered":"2024-09-21T06:11:06.026215+0000","last_clean":"2024-09-21T06:11:06.026215+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:11:06.026215+0000","last_undegraded":"2024-09-21T06:11:06.026215+0000","last_fullsized":"2024-09-21T06:11:06.026215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023448+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:11:06.023448+0000","last_peered":"2024-09-21T06:11:06.023448+0000","last_clean":"2024-09-21T06:11:06.023448+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:11:06.023448+0000","last_undegraded":"2024-09-21T06:11:06.023448+0000","last_fullsized":"2024-09-21T06:11:06.023448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.995137+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:11:05.995137+0000","last_peered":"2024-09-21T06:11:05.995137+0000","last_clean":"2024-09-21T06:11:05.995137+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:11:05.995137+0000","last_undegraded":"2024-09-21T06:11:05.995137+0000","last_fullsized":"2024-09-21T06:11:05.995137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021515+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:11:06.021515+0000","last_peered":"2024-09-21T06:11:06.021515+0000","last_clean":"2024-09-21T06:11:06.021515+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:11:06.021515+0000","last_undegraded":"2024-09-21T06:11:06.021515+0000","last_fullsized":"2024-09-21T06:11:06.021515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.002074+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:11:06.002074+0000","last_peered":"2024-09-21T06:11:06.002074+0000","last_clean":"2024-09-21T06:11:06.002074+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:11:06.002074+0000","last_undegraded":"2024-09-21T06:11:06.002074+0000","last_fullsized":"2024-09-21T06:11:06.002074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.008396+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:11:06.008396+0000","last_peered":"2024-09-21T06:11:06.008396+0000","last_clean":"2024-09-21T06:11:06.008396+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:11:06.008396+0000","last_undegraded":"2024-09-21T06:11:06.008396+0000","last_fullsized":"2024-09-21T06:11:06.008396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026878+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:11:06.026878+0000","last_peered":"2024-09-21T06:11:06.026878+0000","last_clean":"2024-09-21T06:11:06.026878+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:11:06.026878+0000","last_undegraded":"2024-09-21T06:11:06.026878+0000","last_fullsized":"2024-09-21T06:11:06.026878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021546+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:11:06.021546+0000","last_peered":"2024-09-21T06:11:06.021546+0000","last_clean":"2024-09-21T06:11:06.021546+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:11:06.021546+0000","last_undegraded":"2024-09-21T06:11:06.021546+0000","last_fullsized":"2024-09-21T06:11:06.021546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016861+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:11:06.016861+0000","last_peered":"2024-09-21T06:11:06.016861+0000","last_clean":"2024-09-21T06:11:06.016861+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:11:06.016861+0000","last_undegraded":"2024-09-21T06:11:06.016861+0000","last_fullsized":"2024-09-21T06:11:06.016861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023671+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:11:06.023671+0000","last_peered":"2024-09-21T06:11:06.023671+0000","last_clean":"2024-09-21T06:11:06.023671+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:11:06.023671+0000","last_undegraded":"2024-09-21T06:11:06.023671+0000","last_fullsized":"2024-09-21T06:11:06.023671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011521+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:11:06.011521+0000","last_peered":"2024-09-21T06:11:06.011521+0000","last_clean":"2024-09-21T06:11:06.011521+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:11:06.011521+0000","last_undegraded":"2024-09-21T06:11:06.011521+0000","last_fullsized":"2024-09-21T06:11:06.011521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.020387+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:11:06.020387+0000","last_peered":"2024-09-21T06:11:06.020387+0000","last_clean":"2024-09-21T06:11:06.020387+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:11:06.020387+0000","last_undegraded":"2024-09-21T06:11:06.020387+0000","last_fullsized":"2024-09-21T06:11:06.020387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021325+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:11:06.021325+0000","last_peered":"2024-09-21T06:11:06.021325+0000","last_clean":"2024-09-21T06:11:06.021325+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:11:06.021325+0000","last_undegraded":"2024-09-21T06:11:06.021325+0000","last_fullsized":"2024-09-21T06:11:06.021325+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027177+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:11:06.027177+0000","last_peered":"2024-09-21T06:11:06.027177+0000","last_clean":"2024-09-21T06:11:06.027177+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:11:06.027177+0000","last_undegraded":"2024-09-21T06:11:06.027177+0000","last_fullsized":"2024-09-21T06:11:06.027177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027659+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:11:06.027659+0000","last_peered":"2024-09-21T06:11:06.027659+0000","last_clean":"2024-09-21T06:11:06.027659+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:11:06.027659+0000","last_undegraded":"2024-09-21T06:11:06.027659+0000","last_fullsized":"2024-09-21T06:11:06.027659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.012685+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:11:06.012685+0000","last_peered":"2024-09-21T06:11:06.012685+0000","last_clean":"2024-09-21T06:11:06.012685+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:11:06.012685+0000","last_undegraded":"2024-09-21T06:11:06.012685+0000","last_fullsized":"2024-09-21T06:11:06.012685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016804+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:11:06.016804+0000","last_peered":"2024-09-21T06:11:06.016804+0000","last_clean":"2024-09-21T06:11:06.016804+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:11:06.016804+0000","last_undegraded":"2024-09-21T06:11:06.016804+0000","last_fullsized":"2024-09-21T06:11:06.016804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023293+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:11:06.023293+0000","last_peered":"2024-09-21T06:11:06.023293+0000","last_clean":"2024-09-21T06:11:06.023293+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:11:06.023293+0000","last_undegraded":"2024-09-21T06:11:06.023293+0000","last_fullsized":"2024-09-21T06:11:06.023293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016651+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:11:06.016651+0000","last_peered":"2024-09-21T06:11:06.016651+0000","last_clean":"2024-09-21T06:11:06.016651+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:11:06.016651+0000","last_undegraded":"2024-09-21T06:11:06.016651+0000","last_fullsized":"2024-09-21T06:11:06.016651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027496+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:11:06.027496+0000","last_peered":"2024-09-21T06:11:06.027496+0000","last_clean":"2024-09-21T06:11:06.027496+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:11:06.027496+0000","last_undegraded":"2024-09-21T06:11:06.027496+0000","last_fullsized":"2024-09-21T06:11:06.027496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016584+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:11:06.016584+0000","last_peered":"2024-09-21T06:11:06.016584+0000","last_clean":"2024-09-21T06:11:06.016584+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:11:06.016584+0000","last_undegraded":"2024-09-21T06:11:06.016584+0000","last_fullsized":"2024-09-21T06:11:06.016584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.007034+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:11:06.007034+0000","last_peered":"2024-09-21T06:11:06.007034+0000","last_clean":"2024-09-21T06:11:06.007034+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:11:06.007034+0000","last_undegraded":"2024-09-21T06:11:06.007034+0000","last_fullsized":"2024-09-21T06:11:06.007034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.012729+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:11:06.012729+0000","last_peered":"2024-09-21T06:11:06.012729+0000","last_clean":"2024-09-21T06:11:06.012729+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:11:06.012729+0000","last_undegraded":"2024-09-21T06:11:06.012729+0000","last_fullsized":"2024-09-21T06:11:06.012729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027364+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:11:06.027364+0000","last_peered":"2024-09-21T06:11:06.027364+0000","last_clean":"2024-09-21T06:11:06.027364+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:11:06.027364+0000","last_undegraded":"2024-09-21T06:11:06.027364+0000","last_fullsized":"2024-09-21T06:11:06.027364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.004164+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:11:06.004164+0000","last_peered":"2024-09-21T06:11:06.004164+0000","last_clean":"2024-09-21T06:11:06.004164+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:11:06.004164+0000","last_undegraded":"2024-09-21T06:11:06.004164+0000","last_fullsized":"2024-09-21T06:11:06.004164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.017987+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:11:06.017987+0000","last_peered":"2024-09-21T06:11:06.017987+0000","last_clean":"2024-09-21T06:11:06.017987+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:11:06.017987+0000","last_undegraded":"2024-09-21T06:11:06.017987+0000","last_fullsized":"2024-09-21T06:11:06.017987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023167+0000","last_change":"2024-09-21T06:10:07.403662+0000","last_active":"2024-09-21T06:11:06.023167+0000","last_peered":"2024-09-21T06:11:06.023167+0000","last_clean":"2024-09-21T06:11:06.023167+0000","last_became_active":"2024-09-21T06:10:07.403220+0000","last_became_peered":"2024-09-21T06:10:07.403220+0000","last_unstale":"2024-09-21T06:11:06.023167+0000","last_undegraded":"2024-09-21T06:11:06.023167+0000","last_fullsized":"2024-09-21T06:11:06.023167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:42.670948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021026+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:11:06.021026+0000","last_peered":"2024-09-21T06:11:06.021026+0000","last_clean":"2024-09-21T06:11:06.021026+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:11:06.021026+0000","last_undegraded":"2024-09-21T06:11:06.021026+0000","last_fullsized":"2024-09-21T06:11:06.021026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.010605+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:11:06.010605+0000","last_peered":"2024-09-21T06:11:06.010605+0000","last_clean":"2024-09-21T06:11:06.010605+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:11:06.010605+0000","last_undegraded":"2024-09-21T06:11:06.010605+0000","last_fullsized":"2024-09-21T06:11:06.010605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.006692+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:11:06.006692+0000","last_peered":"2024-09-21T06:11:06.006692+0000","last_clean":"2024-09-21T06:11:06.006692+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:11:06.006692+0000","last_undegraded":"2024-09-21T06:11:06.006692+0000","last_fullsized":"2024-09-21T06:11:06.006692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.006132+0000","last_change":"2024-09-21T06:10:07.399633+0000","last_active":"2024-09-21T06:11:06.006132+0000","last_peered":"2024-09-21T06:11:06.006132+0000","last_clean":"2024-09-21T06:11:06.006132+0000","last_became_active":"2024-09-21T06:10:07.399200+0000","last_became_peered":"2024-09-21T06:10:07.399200+0000","last_unstale":"2024-09-21T06:11:06.006132+0000","last_undegraded":"2024-09-21T06:11:06.006132+0000","last_fullsized":"2024-09-21T06:11:06.006132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:16.830371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398713+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:10:07.398713+0000","last_peered":"2024-09-21T06:10:07.398713+0000","last_clean":"2024-09-21T06:10:07.398713+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:10:07.398713+0000","last_undegraded":"2024-09-21T06:10:07.398713+0000","last_fullsized":"2024-09-21T06:10:07.398713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016929+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:11:06.016929+0000","last_peered":"2024-09-21T06:11:06.016929+0000","last_clean":"2024-09-21T06:11:06.016929+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:11:06.016929+0000","last_undegraded":"2024-09-21T06:11:06.016929+0000","last_fullsized":"2024-09-21T06:11:06.016929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587183+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:10:07.587183+0000","last_peered":"2024-09-21T06:10:07.587183+0000","last_clean":"2024-09-21T06:10:07.587183+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:10:07.587183+0000","last_undegraded":"2024-09-21T06:10:07.587183+0000","last_fullsized":"2024-09-21T06:10:07.587183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396798+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:10:07.396798+0000","last_peered":"2024-09-21T06:10:07.396798+0000","last_clean":"2024-09-21T06:10:07.396798+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:10:07.396798+0000","last_undegraded":"2024-09-21T06:10:07.396798+0000","last_fullsized":"2024-09-21T06:10:07.396798+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.020012+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:11:06.020012+0000","last_peered":"2024-09-21T06:11:06.020012+0000","last_clean":"2024-09-21T06:11:06.020012+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:11:06.020012+0000","last_undegraded":"2024-09-21T06:11:06.020012+0000","last_fullsized":"2024-09-21T06:11:06.020012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398807+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:10:07.398807+0000","last_peered":"2024-09-21T06:10:07.398807+0000","last_clean":"2024-09-21T06:10:07.398807+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:10:07.398807+0000","last_undegraded":"2024-09-21T06:10:07.398807+0000","last_fullsized":"2024-09-21T06:10:07.398807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011649+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:11:06.011649+0000","last_peered":"2024-09-21T06:11:06.011649+0000","last_clean":"2024-09-21T06:11:06.011649+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:11:06.011649+0000","last_undegraded":"2024-09-21T06:11:06.011649+0000","last_fullsized":"2024-09-21T06:11:06.011649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587232+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:10:07.587232+0000","last_peered":"2024-09-21T06:10:07.587232+0000","last_clean":"2024-09-21T06:10:07.587232+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:10:07.587232+0000","last_undegraded":"2024-09-21T06:10:07.587232+0000","last_fullsized":"2024-09-21T06:10:07.587232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.014591+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:11:06.014591+0000","last_peered":"2024-09-21T06:11:06.014591+0000","last_clean":"2024-09-21T06:11:06.014591+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:11:06.014591+0000","last_undegraded":"2024-09-21T06:11:06.014591+0000","last_fullsized":"2024-09-21T06:11:06.014591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395089+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:10:07.395089+0000","last_peered":"2024-09-21T06:10:07.395089+0000","last_clean":"2024-09-21T06:10:07.395089+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:10:07.395089+0000","last_undegraded":"2024-09-21T06:10:07.395089+0000","last_fullsized":"2024-09-21T06:10:07.395089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022625+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:11:06.022625+0000","last_peered":"2024-09-21T06:11:06.022625+0000","last_clean":"2024-09-21T06:11:06.022625+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:11:06.022625+0000","last_undegraded":"2024-09-21T06:11:06.022625+0000","last_fullsized":"2024-09-21T06:11:06.022625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399080+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:10:07.399080+0000","last_peered":"2024-09-21T06:10:07.399080+0000","last_clean":"2024-09-21T06:10:07.399080+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:10:07.399080+0000","last_undegraded":"2024-09-21T06:10:07.399080+0000","last_fullsized":"2024-09-21T06:10:07.399080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.028117+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:11:06.028117+0000","last_peered":"2024-09-21T06:11:06.028117+0000","last_clean":"2024-09-21T06:11:06.028117+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:11:06.028117+0000","last_undegraded":"2024-09-21T06:11:06.028117+0000","last_fullsized":"2024-09-21T06:11:06.028117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'31","reported_seq":247,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:14.466629+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:11:14.466629+0000","last_peered":"2024-09-21T06:11:14.466629+0000","last_clean":"2024-09-21T06:11:14.466629+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:11:14.466629+0000","last_undegraded":"2024-09-21T06:11:14.466629+0000","last_fullsized":"2024-09-21T06:11:14.466629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67514,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":162,"num_write":31,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.997726+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:11:05.997726+0000","last_peered":"2024-09-21T06:11:05.997726+0000","last_clean":"2024-09-21T06:11:05.997726+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:11:05.997726+0000","last_undegraded":"2024-09-21T06:11:05.997726+0000","last_fullsized":"2024-09-21T06:11:05.997726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394811+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:10:07.394811+0000","last_peered":"2024-09-21T06:10:07.394811+0000","last_clean":"2024-09-21T06:10:07.394811+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:10:07.394811+0000","last_undegraded":"2024-09-21T06:10:07.394811+0000","last_fullsized":"2024-09-21T06:10:07.394811+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019305+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:11:06.019305+0000","last_peered":"2024-09-21T06:11:06.019305+0000","last_clean":"2024-09-21T06:11:06.019305+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:11:06.019305+0000","last_undegraded":"2024-09-21T06:11:06.019305+0000","last_fullsized":"2024-09-21T06:11:06.019305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398372+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:10:07.398372+0000","last_peered":"2024-09-21T06:10:07.398372+0000","last_clean":"2024-09-21T06:10:07.398372+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:10:07.398372+0000","last_undegraded":"2024-09-21T06:10:07.398372+0000","last_fullsized":"2024-09-21T06:10:07.398372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019906+0000","last_change":"2024-09-21T06:10:07.401185+0000","last_active":"2024-09-21T06:11:06.019906+0000","last_peered":"2024-09-21T06:11:06.019906+0000","last_clean":"2024-09-21T06:11:06.019906+0000","last_became_active":"2024-09-21T06:10:07.400789+0000","last_became_peered":"2024-09-21T06:10:07.400789+0000","last_unstale":"2024-09-21T06:11:06.019906+0000","last_undegraded":"2024-09-21T06:11:06.019906+0000","last_fullsized":"2024-09-21T06:11:06.019906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:03.201239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394895+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:10:07.394895+0000","last_peered":"2024-09-21T06:10:07.394895+0000","last_clean":"2024-09-21T06:10:07.394895+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:10:07.394895+0000","last_undegraded":"2024-09-21T06:10:07.394895+0000","last_fullsized":"2024-09-21T06:10:07.394895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018842+0000","last_change":"2024-09-21T06:10:07.588201+0000","last_active":"2024-09-21T06:11:06.018842+0000","last_peered":"2024-09-21T06:11:06.018842+0000","last_clean":"2024-09-21T06:11:06.018842+0000","last_became_active":"2024-09-21T06:10:07.587788+0000","last_became_peered":"2024-09-21T06:10:07.587788+0000","last_unstale":"2024-09-21T06:11:06.018842+0000","last_undegraded":"2024-09-21T06:11:06.018842+0000","last_fullsized":"2024-09-21T06:11:06.018842+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:16.617435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394307+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:10:07.394307+0000","last_peered":"2024-09-21T06:10:07.394307+0000","last_clean":"2024-09-21T06:10:07.394307+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:10:07.394307+0000","last_undegraded":"2024-09-21T06:10:07.394307+0000","last_fullsized":"2024-09-21T06:10:07.394307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018670+0000","last_change":"2024-09-21T06:10:07.478289+0000","last_active":"2024-09-21T06:11:06.018670+0000","last_peered":"2024-09-21T06:11:06.018670+0000","last_clean":"2024-09-21T06:11:06.018670+0000","last_became_active":"2024-09-21T06:10:07.477828+0000","last_became_peered":"2024-09-21T06:10:07.477828+0000","last_unstale":"2024-09-21T06:11:06.018670+0000","last_undegraded":"2024-09-21T06:11:06.018670+0000","last_fullsized":"2024-09-21T06:11:06.018670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.721612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395262+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:10:07.395262+0000","last_peered":"2024-09-21T06:10:07.395262+0000","last_clean":"2024-09-21T06:10:07.395262+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:10:07.395262+0000","last_undegraded":"2024-09-21T06:10:07.395262+0000","last_fullsized":"2024-09-21T06:10:07.395262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022924+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:11:06.022924+0000","last_peered":"2024-09-21T06:11:06.022924+0000","last_clean":"2024-09-21T06:11:06.022924+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:11:06.022924+0000","last_undegraded":"2024-09-21T06:11:06.022924+0000","last_fullsized":"2024-09-21T06:11:06.022924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406063+0000","last_change":"2024-09-21T06:10:07.405946+0000","last_active":"2024-09-21T06:10:07.406063+0000","last_peered":"2024-09-21T06:10:07.406063+0000","last_clean":"2024-09-21T06:10:07.406063+0000","last_became_active":"2024-09-21T06:10:07.401620+0000","last_became_peered":"2024-09-21T06:10:07.401620+0000","last_unstale":"2024-09-21T06:10:07.406063+0000","last_undegraded":"2024-09-21T06:10:07.406063+0000","last_fullsized":"2024-09-21T06:10:07.406063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:17.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021136+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:11:06.021136+0000","last_peered":"2024-09-21T06:11:06.021136+0000","last_clean":"2024-09-21T06:11:06.021136+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:11:06.021136+0000","last_undegraded":"2024-09-21T06:11:06.021136+0000","last_fullsized":"2024-09-21T06:11:06.021136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399215+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:10:07.399215+0000","last_peered":"2024-09-21T06:10:07.399215+0000","last_clean":"2024-09-21T06:10:07.399215+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:10:07.399215+0000","last_undegraded":"2024-09-21T06:10:07.399215+0000","last_fullsized":"2024-09-21T06:10:07.399215+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.013524+0000","last_change":"2024-09-21T06:10:07.406892+0000","last_active":"2024-09-21T06:11:06.013524+0000","last_peered":"2024-09-21T06:11:06.013524+0000","last_clean":"2024-09-21T06:11:06.013524+0000","last_became_active":"2024-09-21T06:10:07.402546+0000","last_became_peered":"2024-09-21T06:10:07.402546+0000","last_unstale":"2024-09-21T06:11:06.013524+0000","last_undegraded":"2024-09-21T06:11:06.013524+0000","last_fullsized":"2024-09-21T06:11:06.013524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.081510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394662+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:10:07.394662+0000","last_peered":"2024-09-21T06:10:07.394662+0000","last_clean":"2024-09-21T06:10:07.394662+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:10:07.394662+0000","last_undegraded":"2024-09-21T06:10:07.394662+0000","last_fullsized":"2024-09-21T06:10:07.394662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019250+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:11:06.019250+0000","last_peered":"2024-09-21T06:11:06.019250+0000","last_clean":"2024-09-21T06:11:06.019250+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:11:06.019250+0000","last_undegraded":"2024-09-21T06:11:06.019250+0000","last_fullsized":"2024-09-21T06:11:06.019250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395834+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:10:07.395834+0000","last_peered":"2024-09-21T06:10:07.395834+0000","last_clean":"2024-09-21T06:10:07.395834+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:10:07.395834+0000","last_undegraded":"2024-09-21T06:10:07.395834+0000","last_fullsized":"2024-09-21T06:10:07.395834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395073+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:10:07.395073+0000","last_peered":"2024-09-21T06:10:07.395073+0000","last_clean":"2024-09-21T06:10:07.395073+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:10:07.395073+0000","last_undegraded":"2024-09-21T06:10:07.395073+0000","last_fullsized":"2024-09-21T06:10:07.395073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.005537+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:11:06.005537+0000","last_peered":"2024-09-21T06:11:06.005537+0000","last_clean":"2024-09-21T06:11:06.005537+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:11:06.005537+0000","last_undegraded":"2024-09-21T06:11:06.005537+0000","last_fullsized":"2024-09-21T06:11:06.005537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394546+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:10:07.394546+0000","last_peered":"2024-09-21T06:10:07.394546+0000","last_clean":"2024-09-21T06:10:07.394546+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:10:07.394546+0000","last_undegraded":"2024-09-21T06:10:07.394546+0000","last_fullsized":"2024-09-21T06:10:07.394546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011182+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:11:06.011182+0000","last_peered":"2024-09-21T06:11:06.011182+0000","last_clean":"2024-09-21T06:11:06.011182+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:11:06.011182+0000","last_undegraded":"2024-09-21T06:11:06.011182+0000","last_fullsized":"2024-09-21T06:11:06.011182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396950+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:10:07.396950+0000","last_peered":"2024-09-21T06:10:07.396950+0000","last_clean":"2024-09-21T06:10:07.396950+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:10:07.396950+0000","last_undegraded":"2024-09-21T06:10:07.396950+0000","last_fullsized":"2024-09-21T06:10:07.396950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.015296+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:11:06.015296+0000","last_peered":"2024-09-21T06:11:06.015296+0000","last_clean":"2024-09-21T06:11:06.015296+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:11:06.015296+0000","last_undegraded":"2024-09-21T06:11:06.015296+0000","last_fullsized":"2024-09-21T06:11:06.015296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394998+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:10:07.394998+0000","last_peered":"2024-09-21T06:10:07.394998+0000","last_clean":"2024-09-21T06:10:07.394998+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:10:07.394998+0000","last_undegraded":"2024-09-21T06:10:07.394998+0000","last_fullsized":"2024-09-21T06:10:07.394998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018487+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:11:06.018487+0000","last_peered":"2024-09-21T06:11:06.018487+0000","last_clean":"2024-09-21T06:11:06.018487+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:11:06.018487+0000","last_undegraded":"2024-09-21T06:11:06.018487+0000","last_fullsized":"2024-09-21T06:11:06.018487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401748+0000","last_change":"2024-09-21T06:10:07.401640+0000","last_active":"2024-09-21T06:10:07.401748+0000","last_peered":"2024-09-21T06:10:07.401748+0000","last_clean":"2024-09-21T06:10:07.401748+0000","last_became_active":"2024-09-21T06:10:07.401242+0000","last_became_peered":"2024-09-21T06:10:07.401242+0000","last_unstale":"2024-09-21T06:10:07.401748+0000","last_undegraded":"2024-09-21T06:10:07.401748+0000","last_fullsized":"2024-09-21T06:10:07.401748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:25.335694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.004481+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:11:06.004481+0000","last_peered":"2024-09-21T06:11:06.004481+0000","last_clean":"2024-09-21T06:11:06.004481+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:11:06.004481+0000","last_undegraded":"2024-09-21T06:11:06.004481+0000","last_fullsized":"2024-09-21T06:11:06.004481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395763+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:10:07.395763+0000","last_peered":"2024-09-21T06:10:07.395763+0000","last_clean":"2024-09-21T06:10:07.395763+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:10:07.395763+0000","last_undegraded":"2024-09-21T06:10:07.395763+0000","last_fullsized":"2024-09-21T06:10:07.395763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.010167+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:11:06.010167+0000","last_peered":"2024-09-21T06:11:06.010167+0000","last_clean":"2024-09-21T06:11:06.010167+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:11:06.010167+0000","last_undegraded":"2024-09-21T06:11:06.010167+0000","last_fullsized":"2024-09-21T06:11:06.010167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'11","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.300858+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:11:06.300858+0000","last_peered":"2024-09-21T06:11:06.300858+0000","last_clean":"2024-09-21T06:11:06.300858+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:11:06.300858+0000","last_undegraded":"2024-09-21T06:11:06.300858+0000","last_fullsized":"2024-09-21T06:11:06.300858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022143+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:11:06.022143+0000","last_peered":"2024-09-21T06:11:06.022143+0000","last_clean":"2024-09-21T06:11:06.022143+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:11:06.022143+0000","last_undegraded":"2024-09-21T06:11:06.022143+0000","last_fullsized":"2024-09-21T06:11:06.022143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395509+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:10:07.395509+0000","last_peered":"2024-09-21T06:10:07.395509+0000","last_clean":"2024-09-21T06:10:07.395509+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:10:07.395509+0000","last_undegraded":"2024-09-21T06:10:07.395509+0000","last_fullsized":"2024-09-21T06:10:07.395509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019541+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:11:06.019541+0000","last_peered":"2024-09-21T06:11:06.019541+0000","last_clean":"2024-09-21T06:11:06.019541+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:11:06.019541+0000","last_undegraded":"2024-09-21T06:11:06.019541+0000","last_fullsized":"2024-09-21T06:11:06.019541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476406+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:10:07.476406+0000","last_peered":"2024-09-21T06:10:07.476406+0000","last_clean":"2024-09-21T06:10:07.476406+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:10:07.476406+0000","last_undegraded":"2024-09-21T06:10:07.476406+0000","last_fullsized":"2024-09-21T06:10:07.476406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.970288+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:11:05.970288+0000","last_peered":"2024-09-21T06:11:05.970288+0000","last_clean":"2024-09-21T06:11:05.970288+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:11:05.970288+0000","last_undegraded":"2024-09-21T06:11:05.970288+0000","last_fullsized":"2024-09-21T06:11:05.970288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393700+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:10:07.393700+0000","last_peered":"2024-09-21T06:10:07.393700+0000","last_clean":"2024-09-21T06:10:07.393700+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:10:07.393700+0000","last_undegraded":"2024-09-21T06:10:07.393700+0000","last_fullsized":"2024-09-21T06:10:07.393700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.008331+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:11:06.008331+0000","last_peered":"2024-09-21T06:11:06.008331+0000","last_clean":"2024-09-21T06:11:06.008331+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:11:06.008331+0000","last_undegraded":"2024-09-21T06:11:06.008331+0000","last_fullsized":"2024-09-21T06:11:06.008331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394466+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:10:07.394466+0000","last_peered":"2024-09-21T06:10:07.394466+0000","last_clean":"2024-09-21T06:10:07.394466+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:10:07.394466+0000","last_undegraded":"2024-09-21T06:10:07.394466+0000","last_fullsized":"2024-09-21T06:10:07.394466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018378+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:11:06.018378+0000","last_peered":"2024-09-21T06:11:06.018378+0000","last_clean":"2024-09-21T06:11:06.018378+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:11:06.018378+0000","last_undegraded":"2024-09-21T06:11:06.018378+0000","last_fullsized":"2024-09-21T06:11:06.018378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403122+0000","last_change":"2024-09-21T06:10:07.402379+0000","last_active":"2024-09-21T06:10:07.403122+0000","last_peered":"2024-09-21T06:10:07.403122+0000","last_clean":"2024-09-21T06:10:07.403122+0000","last_became_active":"2024-09-21T06:10:07.396828+0000","last_became_peered":"2024-09-21T06:10:07.396828+0000","last_unstale":"2024-09-21T06:10:07.403122+0000","last_undegraded":"2024-09-21T06:10:07.403122+0000","last_fullsized":"2024-09-21T06:10:07.403122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:26.181343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026247+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:11:06.026247+0000","last_peered":"2024-09-21T06:11:06.026247+0000","last_clean":"2024-09-21T06:11:06.026247+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:11:06.026247+0000","last_undegraded":"2024-09-21T06:11:06.026247+0000","last_fullsized":"2024-09-21T06:11:06.026247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022587+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:11:06.022587+0000","last_peered":"2024-09-21T06:11:06.022587+0000","last_clean":"2024-09-21T06:11:06.022587+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:11:06.022587+0000","last_undegraded":"2024-09-21T06:11:06.022587+0000","last_fullsized":"2024-09-21T06:11:06.022587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395391+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:10:07.395391+0000","last_peered":"2024-09-21T06:10:07.395391+0000","last_clean":"2024-09-21T06:10:07.395391+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:10:07.395391+0000","last_undegraded":"2024-09-21T06:10:07.395391+0000","last_fullsized":"2024-09-21T06:10:07.395391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011566+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:11:06.011566+0000","last_peered":"2024-09-21T06:11:06.011566+0000","last_clean":"2024-09-21T06:11:06.011566+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:11:06.011566+0000","last_undegraded":"2024-09-21T06:11:06.011566+0000","last_fullsized":"2024-09-21T06:11:06.011566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395635+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:10:07.395635+0000","last_peered":"2024-09-21T06:10:07.395635+0000","last_clean":"2024-09-21T06:10:07.395635+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:10:07.395635+0000","last_undegraded":"2024-09-21T06:10:07.395635+0000","last_fullsized":"2024-09-21T06:10:07.395635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026621+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:11:06.026621+0000","last_peered":"2024-09-21T06:11:06.026621+0000","last_clean":"2024-09-21T06:11:06.026621+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:11:06.026621+0000","last_undegraded":"2024-09-21T06:11:06.026621+0000","last_fullsized":"2024-09-21T06:11:06.026621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394530+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:10:07.394530+0000","last_peered":"2024-09-21T06:10:07.394530+0000","last_clean":"2024-09-21T06:10:07.394530+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:10:07.394530+0000","last_undegraded":"2024-09-21T06:10:07.394530+0000","last_fullsized":"2024-09-21T06:10:07.394530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.013863+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:11:06.013863+0000","last_peered":"2024-09-21T06:11:06.013863+0000","last_clean":"2024-09-21T06:11:06.013863+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:11:06.013863+0000","last_undegraded":"2024-09-21T06:11:06.013863+0000","last_fullsized":"2024-09-21T06:11:06.013863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399889+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:10:07.399889+0000","last_peered":"2024-09-21T06:10:07.399889+0000","last_clean":"2024-09-21T06:10:07.399889+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:10:07.399889+0000","last_undegraded":"2024-09-21T06:10:07.399889+0000","last_fullsized":"2024-09-21T06:10:07.399889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68312,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":197,"num_read_kb":166,"num_write":91,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":839680,"data_stored":415673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1527808,"data_stored":1073866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":416372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266736,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":416277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1441792,"data_stored":1006740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1449984,"data_stored":1006340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":415894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378702,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":483221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":483187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:11:17.240 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T06:11:17.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:11:17.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:16 smithi110 ceph-mon[34550]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 425 B/s wr, 3 op/s 2024-09-21T06:11:17.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:16 smithi107 ceph-mon[34380]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 425 B/s wr, 3 op/s 2024-09-21T06:11:17.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 -- 172.21.15.31:0/932334825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b4102510 msgr2=0x7f82b4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 --2- 172.21.15.31:0/932334825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b4102510 0x7f82b4102920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009900 tx=0x7f82b002a480 comp rx=0 tx=0).stop 2024-09-21T06:11:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 -- 172.21.15.31:0/932334825 shutdown_connections 2024-09-21T06:11:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 --2- 172.21.15.31:0/932334825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b41100c0 0x7f82b41124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 --2- 172.21.15.31:0/932334825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102e60 0x7f82b410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 --2- 172.21.15.31:0/932334825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b4102510 0x7f82b4102920 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 -- 172.21.15.31:0/932334825 >> 172.21.15.31:0/932334825 conn(0x7f82b40fe3b0 msgr2=0x7f82b4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 -- 172.21.15.31:0/932334825 shutdown_connections 2024-09-21T06:11:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.390+0000 7f82bb430640 1 -- 172.21.15.31:0/932334825 wait complete. 2024-09-21T06:11:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 Processor -- start 2024-09-21T06:11:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 -- start start 2024-09-21T06:11:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102510 0x7f82b419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b4102e60 0x7f82b419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 0x7f82b41a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4115bb0 con 0x7f82b41100c0 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4115a30 con 0x7f82b4102e60 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82bb430640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4115d30 con 0x7f82b4102510 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 0x7f82b41a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 0x7f82b41a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39634/0 (socket says 172.21.15.31:39634) 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 -- 172.21.15.31:0/3373608673 learned_addr learned my addr 172.21.15.31:0/3373608673 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b91cc640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102510 0x7f82b419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 -- 172.21.15.31:0/3373608673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102510 msgr2=0x7f82b419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102510 0x7f82b419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 -- 172.21.15.31:0/3373608673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b4102e60 msgr2=0x7f82b419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b89cb640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b4102e60 0x7f82b419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b4102e60 0x7f82b419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b99cd640 1 -- 172.21.15.31:0/3373608673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41a24c0 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.391+0000 7f82b91cc640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102510 0x7f82b419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.392+0000 7f82b99cd640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 0x7f82b41a1dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00eaa0 tx=0x7f82ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.392+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00cdb0 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.392+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ac005020 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.392+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0116d0 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.392+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b41a27b0 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.392+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41a8f40 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.393+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b4102920 con 0x7f82b41100c0 2024-09-21T06:11:17.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.394+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f82ac0051d0 con 0x7f82b41100c0 2024-09-21T06:11:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.396+0000 7f82aa7fc640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f82a00b6560 0x7f82a00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.396+0000 7f82b91cc640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f82a00b6560 0x7f82a00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.396+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f82ac0e7b10 con 0x7f82b41100c0 2024-09-21T06:11:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.396+0000 7f82b91cc640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f82a00b6560 0x7f82a00b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82b00096c0 tx=0x7f82b0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:17.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.399+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82ac016030 con 0x7f82b41100c0 2024-09-21T06:11:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.584+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82b41092a0 con 0x7f82a00b6560 2024-09-21T06:11:17.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.589+0000 7f82aa7fc640 1 -- 172.21.15.31:0/3373608673 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258775 (secure 0 0 0) 0x7f82b41092a0 con 0x7f82a00b6560 2024-09-21T06:11:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:17.595 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-21T06:11:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f82a00b6560 msgr2=0x7f82a00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f82a00b6560 0x7f82a00b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82b00096c0 tx=0x7f82b0031400 comp rx=0 tx=0).stop 2024-09-21T06:11:17.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 msgr2=0x7f82b41a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 0x7f82b41a1dc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00eaa0 tx=0x7f82ac00ef60 comp rx=0 tx=0).stop 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 shutdown_connections 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f82a00b6560 0x7f82a00b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b41100c0 0x7f82b41a1dc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82b4102e60 0x7f82b419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 --2- 172.21.15.31:0/3373608673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82b4102510 0x7f82b419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 >> 172.21.15.31:0/3373608673 conn(0x7f82b40fe3b0 msgr2=0x7f82b4108b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 shutdown_connections 2024-09-21T06:11:17.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:17.592+0000 7f82bb430640 1 -- 172.21.15.31:0/3373608673 wait complete. 2024-09-21T06:11:17.605 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-21T06:11:15.636892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658914,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":278,"num_read_kb":235,"num_write":216,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":668,"ondisk_log_size":668,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381940,"kb_used_data":11892,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559884,"statfs":{"total":1151940427776,"available":1151549321216,"internally_reserved":0,"allocated":12177408,"data_stored":6964818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":557,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":17,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020756"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027515+0000","last_change":"2024-09-21T06:09:36.307918+0000","last_active":"2024-09-21T06:11:06.027515+0000","last_peered":"2024-09-21T06:11:06.027515+0000","last_clean":"2024-09-21T06:11:06.027515+0000","last_became_active":"2024-09-21T06:09:36.307524+0000","last_became_peered":"2024-09-21T06:09:36.307524+0000","last_unstale":"2024-09-21T06:11:06.027515+0000","last_undegraded":"2024-09-21T06:11:06.027515+0000","last_fullsized":"2024-09-21T06:11:06.027515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:52.330019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.015871+0000","last_change":"2024-09-21T06:09:36.296631+0000","last_active":"2024-09-21T06:11:06.015871+0000","last_peered":"2024-09-21T06:11:06.015871+0000","last_clean":"2024-09-21T06:11:06.015871+0000","last_became_active":"2024-09-21T06:09:36.295969+0000","last_became_peered":"2024-09-21T06:09:36.295969+0000","last_unstale":"2024-09-21T06:11:06.015871+0000","last_undegraded":"2024-09-21T06:11:06.015871+0000","last_fullsized":"2024-09-21T06:11:06.015871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:38.189488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026215+0000","last_change":"2024-09-21T06:09:36.303988+0000","last_active":"2024-09-21T06:11:06.026215+0000","last_peered":"2024-09-21T06:11:06.026215+0000","last_clean":"2024-09-21T06:11:06.026215+0000","last_became_active":"2024-09-21T06:09:36.303616+0000","last_became_peered":"2024-09-21T06:09:36.303616+0000","last_unstale":"2024-09-21T06:11:06.026215+0000","last_undegraded":"2024-09-21T06:11:06.026215+0000","last_fullsized":"2024-09-21T06:11:06.026215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:46:27.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023448+0000","last_change":"2024-09-21T06:09:36.299688+0000","last_active":"2024-09-21T06:11:06.023448+0000","last_peered":"2024-09-21T06:11:06.023448+0000","last_clean":"2024-09-21T06:11:06.023448+0000","last_became_active":"2024-09-21T06:09:36.298862+0000","last_became_peered":"2024-09-21T06:09:36.298862+0000","last_unstale":"2024-09-21T06:11:06.023448+0000","last_undegraded":"2024-09-21T06:11:06.023448+0000","last_fullsized":"2024-09-21T06:11:06.023448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:58.016246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.995137+0000","last_change":"2024-09-21T06:09:36.298624+0000","last_active":"2024-09-21T06:11:05.995137+0000","last_peered":"2024-09-21T06:11:05.995137+0000","last_clean":"2024-09-21T06:11:05.995137+0000","last_became_active":"2024-09-21T06:09:36.298296+0000","last_became_peered":"2024-09-21T06:09:36.298296+0000","last_unstale":"2024-09-21T06:11:05.995137+0000","last_undegraded":"2024-09-21T06:11:05.995137+0000","last_fullsized":"2024-09-21T06:11:05.995137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:38.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021515+0000","last_change":"2024-09-21T06:09:36.299251+0000","last_active":"2024-09-21T06:11:06.021515+0000","last_peered":"2024-09-21T06:11:06.021515+0000","last_clean":"2024-09-21T06:11:06.021515+0000","last_became_active":"2024-09-21T06:09:36.298190+0000","last_became_peered":"2024-09-21T06:09:36.298190+0000","last_unstale":"2024-09-21T06:11:06.021515+0000","last_undegraded":"2024-09-21T06:11:06.021515+0000","last_fullsized":"2024-09-21T06:11:06.021515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:11.346058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.002074+0000","last_change":"2024-09-21T06:09:36.290331+0000","last_active":"2024-09-21T06:11:06.002074+0000","last_peered":"2024-09-21T06:11:06.002074+0000","last_clean":"2024-09-21T06:11:06.002074+0000","last_became_active":"2024-09-21T06:09:36.289844+0000","last_became_peered":"2024-09-21T06:09:36.289844+0000","last_unstale":"2024-09-21T06:11:06.002074+0000","last_undegraded":"2024-09-21T06:11:06.002074+0000","last_fullsized":"2024-09-21T06:11:06.002074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:31.292954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.008396+0000","last_change":"2024-09-21T06:09:36.298503+0000","last_active":"2024-09-21T06:11:06.008396+0000","last_peered":"2024-09-21T06:11:06.008396+0000","last_clean":"2024-09-21T06:11:06.008396+0000","last_became_active":"2024-09-21T06:09:36.297255+0000","last_became_peered":"2024-09-21T06:09:36.297255+0000","last_unstale":"2024-09-21T06:11:06.008396+0000","last_undegraded":"2024-09-21T06:11:06.008396+0000","last_fullsized":"2024-09-21T06:11:06.008396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:13.782577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026878+0000","last_change":"2024-09-21T06:09:36.299743+0000","last_active":"2024-09-21T06:11:06.026878+0000","last_peered":"2024-09-21T06:11:06.026878+0000","last_clean":"2024-09-21T06:11:06.026878+0000","last_became_active":"2024-09-21T06:09:36.299398+0000","last_became_peered":"2024-09-21T06:09:36.299398+0000","last_unstale":"2024-09-21T06:11:06.026878+0000","last_undegraded":"2024-09-21T06:11:06.026878+0000","last_fullsized":"2024-09-21T06:11:06.026878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:57.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021546+0000","last_change":"2024-09-21T06:09:36.299768+0000","last_active":"2024-09-21T06:11:06.021546+0000","last_peered":"2024-09-21T06:11:06.021546+0000","last_clean":"2024-09-21T06:11:06.021546+0000","last_became_active":"2024-09-21T06:09:36.299390+0000","last_became_peered":"2024-09-21T06:09:36.299390+0000","last_unstale":"2024-09-21T06:11:06.021546+0000","last_undegraded":"2024-09-21T06:11:06.021546+0000","last_fullsized":"2024-09-21T06:11:06.021546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:00.391637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016861+0000","last_change":"2024-09-21T06:09:36.304976+0000","last_active":"2024-09-21T06:11:06.016861+0000","last_peered":"2024-09-21T06:11:06.016861+0000","last_clean":"2024-09-21T06:11:06.016861+0000","last_became_active":"2024-09-21T06:09:36.304266+0000","last_became_peered":"2024-09-21T06:09:36.304266+0000","last_unstale":"2024-09-21T06:11:06.016861+0000","last_undegraded":"2024-09-21T06:11:06.016861+0000","last_fullsized":"2024-09-21T06:11:06.016861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:24.243305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023671+0000","last_change":"2024-09-21T06:09:36.298087+0000","last_active":"2024-09-21T06:11:06.023671+0000","last_peered":"2024-09-21T06:11:06.023671+0000","last_clean":"2024-09-21T06:11:06.023671+0000","last_became_active":"2024-09-21T06:09:36.297733+0000","last_became_peered":"2024-09-21T06:09:36.297733+0000","last_unstale":"2024-09-21T06:11:06.023671+0000","last_undegraded":"2024-09-21T06:11:06.023671+0000","last_fullsized":"2024-09-21T06:11:06.023671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:32.051869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011521+0000","last_change":"2024-09-21T06:09:36.294031+0000","last_active":"2024-09-21T06:11:06.011521+0000","last_peered":"2024-09-21T06:11:06.011521+0000","last_clean":"2024-09-21T06:11:06.011521+0000","last_became_active":"2024-09-21T06:09:36.293648+0000","last_became_peered":"2024-09-21T06:09:36.293648+0000","last_unstale":"2024-09-21T06:11:06.011521+0000","last_undegraded":"2024-09-21T06:11:06.011521+0000","last_fullsized":"2024-09-21T06:11:06.011521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:00.003922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.020387+0000","last_change":"2024-09-21T06:09:36.302467+0000","last_active":"2024-09-21T06:11:06.020387+0000","last_peered":"2024-09-21T06:11:06.020387+0000","last_clean":"2024-09-21T06:11:06.020387+0000","last_became_active":"2024-09-21T06:09:36.302075+0000","last_became_peered":"2024-09-21T06:09:36.302075+0000","last_unstale":"2024-09-21T06:11:06.020387+0000","last_undegraded":"2024-09-21T06:11:06.020387+0000","last_fullsized":"2024-09-21T06:11:06.020387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:18.560985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021325+0000","last_change":"2024-09-21T06:09:36.304100+0000","last_active":"2024-09-21T06:11:06.021325+0000","last_peered":"2024-09-21T06:11:06.021325+0000","last_clean":"2024-09-21T06:11:06.021325+0000","last_became_active":"2024-09-21T06:09:36.302796+0000","last_became_peered":"2024-09-21T06:09:36.302796+0000","last_unstale":"2024-09-21T06:11:06.021325+0000","last_undegraded":"2024-09-21T06:11:06.021325+0000","last_fullsized":"2024-09-21T06:11:06.021325+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:17.659458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027177+0000","last_change":"2024-09-21T06:09:36.308404+0000","last_active":"2024-09-21T06:11:06.027177+0000","last_peered":"2024-09-21T06:11:06.027177+0000","last_clean":"2024-09-21T06:11:06.027177+0000","last_became_active":"2024-09-21T06:09:36.308017+0000","last_became_peered":"2024-09-21T06:09:36.308017+0000","last_unstale":"2024-09-21T06:11:06.027177+0000","last_undegraded":"2024-09-21T06:11:06.027177+0000","last_fullsized":"2024-09-21T06:11:06.027177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:17.480631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027659+0000","last_change":"2024-09-21T06:09:36.307929+0000","last_active":"2024-09-21T06:11:06.027659+0000","last_peered":"2024-09-21T06:11:06.027659+0000","last_clean":"2024-09-21T06:11:06.027659+0000","last_became_active":"2024-09-21T06:09:36.307514+0000","last_became_peered":"2024-09-21T06:09:36.307514+0000","last_unstale":"2024-09-21T06:11:06.027659+0000","last_undegraded":"2024-09-21T06:11:06.027659+0000","last_fullsized":"2024-09-21T06:11:06.027659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:04.235694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.012685+0000","last_change":"2024-09-21T06:09:36.302085+0000","last_active":"2024-09-21T06:11:06.012685+0000","last_peered":"2024-09-21T06:11:06.012685+0000","last_clean":"2024-09-21T06:11:06.012685+0000","last_became_active":"2024-09-21T06:09:36.301738+0000","last_became_peered":"2024-09-21T06:09:36.301738+0000","last_unstale":"2024-09-21T06:11:06.012685+0000","last_undegraded":"2024-09-21T06:11:06.012685+0000","last_fullsized":"2024-09-21T06:11:06.012685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:37.227967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016804+0000","last_change":"2024-09-21T06:09:36.299276+0000","last_active":"2024-09-21T06:11:06.016804+0000","last_peered":"2024-09-21T06:11:06.016804+0000","last_clean":"2024-09-21T06:11:06.016804+0000","last_became_active":"2024-09-21T06:09:36.298776+0000","last_became_peered":"2024-09-21T06:09:36.298776+0000","last_unstale":"2024-09-21T06:11:06.016804+0000","last_undegraded":"2024-09-21T06:11:06.016804+0000","last_fullsized":"2024-09-21T06:11:06.016804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:38.346772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023293+0000","last_change":"2024-09-21T06:09:36.298035+0000","last_active":"2024-09-21T06:11:06.023293+0000","last_peered":"2024-09-21T06:11:06.023293+0000","last_clean":"2024-09-21T06:11:06.023293+0000","last_became_active":"2024-09-21T06:09:36.297654+0000","last_became_peered":"2024-09-21T06:09:36.297654+0000","last_unstale":"2024-09-21T06:11:06.023293+0000","last_undegraded":"2024-09-21T06:11:06.023293+0000","last_fullsized":"2024-09-21T06:11:06.023293+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:22.477291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016651+0000","last_change":"2024-09-21T06:09:36.291712+0000","last_active":"2024-09-21T06:11:06.016651+0000","last_peered":"2024-09-21T06:11:06.016651+0000","last_clean":"2024-09-21T06:11:06.016651+0000","last_became_active":"2024-09-21T06:09:36.291062+0000","last_became_peered":"2024-09-21T06:09:36.291062+0000","last_unstale":"2024-09-21T06:11:06.016651+0000","last_undegraded":"2024-09-21T06:11:06.016651+0000","last_fullsized":"2024-09-21T06:11:06.016651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:28.188773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027496+0000","last_change":"2024-09-21T06:09:36.301475+0000","last_active":"2024-09-21T06:11:06.027496+0000","last_peered":"2024-09-21T06:11:06.027496+0000","last_clean":"2024-09-21T06:11:06.027496+0000","last_became_active":"2024-09-21T06:09:36.300825+0000","last_became_peered":"2024-09-21T06:09:36.300825+0000","last_unstale":"2024-09-21T06:11:06.027496+0000","last_undegraded":"2024-09-21T06:11:06.027496+0000","last_fullsized":"2024-09-21T06:11:06.027496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:04.282037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016584+0000","last_change":"2024-09-21T06:09:36.300994+0000","last_active":"2024-09-21T06:11:06.016584+0000","last_peered":"2024-09-21T06:11:06.016584+0000","last_clean":"2024-09-21T06:11:06.016584+0000","last_became_active":"2024-09-21T06:09:36.298672+0000","last_became_peered":"2024-09-21T06:09:36.298672+0000","last_unstale":"2024-09-21T06:11:06.016584+0000","last_undegraded":"2024-09-21T06:11:06.016584+0000","last_fullsized":"2024-09-21T06:11:06.016584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:47:06.473696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.007034+0000","last_change":"2024-09-21T06:09:36.291001+0000","last_active":"2024-09-21T06:11:06.007034+0000","last_peered":"2024-09-21T06:11:06.007034+0000","last_clean":"2024-09-21T06:11:06.007034+0000","last_became_active":"2024-09-21T06:09:36.290596+0000","last_became_peered":"2024-09-21T06:09:36.290596+0000","last_unstale":"2024-09-21T06:11:06.007034+0000","last_undegraded":"2024-09-21T06:11:06.007034+0000","last_fullsized":"2024-09-21T06:11:06.007034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:03.696077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.012729+0000","last_change":"2024-09-21T06:09:36.298689+0000","last_active":"2024-09-21T06:11:06.012729+0000","last_peered":"2024-09-21T06:11:06.012729+0000","last_clean":"2024-09-21T06:11:06.012729+0000","last_became_active":"2024-09-21T06:09:36.297137+0000","last_became_peered":"2024-09-21T06:09:36.297137+0000","last_unstale":"2024-09-21T06:11:06.012729+0000","last_undegraded":"2024-09-21T06:11:06.012729+0000","last_fullsized":"2024-09-21T06:11:06.012729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:42.154776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.027364+0000","last_change":"2024-09-21T06:09:36.302962+0000","last_active":"2024-09-21T06:11:06.027364+0000","last_peered":"2024-09-21T06:11:06.027364+0000","last_clean":"2024-09-21T06:11:06.027364+0000","last_became_active":"2024-09-21T06:09:36.302367+0000","last_became_peered":"2024-09-21T06:09:36.302367+0000","last_unstale":"2024-09-21T06:11:06.027364+0000","last_undegraded":"2024-09-21T06:11:06.027364+0000","last_fullsized":"2024-09-21T06:11:06.027364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:46.829206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.004164+0000","last_change":"2024-09-21T06:09:36.301653+0000","last_active":"2024-09-21T06:11:06.004164+0000","last_peered":"2024-09-21T06:11:06.004164+0000","last_clean":"2024-09-21T06:11:06.004164+0000","last_became_active":"2024-09-21T06:09:36.301259+0000","last_became_peered":"2024-09-21T06:09:36.301259+0000","last_unstale":"2024-09-21T06:11:06.004164+0000","last_undegraded":"2024-09-21T06:11:06.004164+0000","last_fullsized":"2024-09-21T06:11:06.004164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:28.758518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.017987+0000","last_change":"2024-09-21T06:09:36.291566+0000","last_active":"2024-09-21T06:11:06.017987+0000","last_peered":"2024-09-21T06:11:06.017987+0000","last_clean":"2024-09-21T06:11:06.017987+0000","last_became_active":"2024-09-21T06:09:36.291124+0000","last_became_peered":"2024-09-21T06:09:36.291124+0000","last_unstale":"2024-09-21T06:11:06.017987+0000","last_undegraded":"2024-09-21T06:11:06.017987+0000","last_fullsized":"2024-09-21T06:11:06.017987+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:35.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.023167+0000","last_change":"2024-09-21T06:10:07.403662+0000","last_active":"2024-09-21T06:11:06.023167+0000","last_peered":"2024-09-21T06:11:06.023167+0000","last_clean":"2024-09-21T06:11:06.023167+0000","last_became_active":"2024-09-21T06:10:07.403220+0000","last_became_peered":"2024-09-21T06:10:07.403220+0000","last_unstale":"2024-09-21T06:11:06.023167+0000","last_undegraded":"2024-09-21T06:11:06.023167+0000","last_fullsized":"2024-09-21T06:11:06.023167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:42.670948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021026+0000","last_change":"2024-09-21T06:09:36.305788+0000","last_active":"2024-09-21T06:11:06.021026+0000","last_peered":"2024-09-21T06:11:06.021026+0000","last_clean":"2024-09-21T06:11:06.021026+0000","last_became_active":"2024-09-21T06:09:36.305329+0000","last_became_peered":"2024-09-21T06:09:36.305329+0000","last_unstale":"2024-09-21T06:11:06.021026+0000","last_undegraded":"2024-09-21T06:11:06.021026+0000","last_fullsized":"2024-09-21T06:11:06.021026+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:02.938750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.010605+0000","last_change":"2024-09-21T06:09:36.293291+0000","last_active":"2024-09-21T06:11:06.010605+0000","last_peered":"2024-09-21T06:11:06.010605+0000","last_clean":"2024-09-21T06:11:06.010605+0000","last_became_active":"2024-09-21T06:09:36.292850+0000","last_became_peered":"2024-09-21T06:09:36.292850+0000","last_unstale":"2024-09-21T06:11:06.010605+0000","last_undegraded":"2024-09-21T06:11:06.010605+0000","last_fullsized":"2024-09-21T06:11:06.010605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:59.694908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.006692+0000","last_change":"2024-09-21T06:09:36.292833+0000","last_active":"2024-09-21T06:11:06.006692+0000","last_peered":"2024-09-21T06:11:06.006692+0000","last_clean":"2024-09-21T06:11:06.006692+0000","last_became_active":"2024-09-21T06:09:36.292176+0000","last_became_peered":"2024-09-21T06:09:36.292176+0000","last_unstale":"2024-09-21T06:11:06.006692+0000","last_undegraded":"2024-09-21T06:11:06.006692+0000","last_fullsized":"2024-09-21T06:11:06.006692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:28.900797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.006132+0000","last_change":"2024-09-21T06:10:07.399633+0000","last_active":"2024-09-21T06:11:06.006132+0000","last_peered":"2024-09-21T06:11:06.006132+0000","last_clean":"2024-09-21T06:11:06.006132+0000","last_became_active":"2024-09-21T06:10:07.399200+0000","last_became_peered":"2024-09-21T06:10:07.399200+0000","last_unstale":"2024-09-21T06:11:06.006132+0000","last_undegraded":"2024-09-21T06:11:06.006132+0000","last_fullsized":"2024-09-21T06:11:06.006132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:16.830371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398713+0000","last_change":"2024-09-21T06:09:34.274004+0000","last_active":"2024-09-21T06:10:07.398713+0000","last_peered":"2024-09-21T06:10:07.398713+0000","last_clean":"2024-09-21T06:10:07.398713+0000","last_became_active":"2024-09-21T06:09:34.273207+0000","last_became_peered":"2024-09-21T06:09:34.273207+0000","last_unstale":"2024-09-21T06:10:07.398713+0000","last_undegraded":"2024-09-21T06:10:07.398713+0000","last_fullsized":"2024-09-21T06:10:07.398713+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:55:37.626539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.016929+0000","last_change":"2024-09-21T06:09:36.294289+0000","last_active":"2024-09-21T06:11:06.016929+0000","last_peered":"2024-09-21T06:11:06.016929+0000","last_clean":"2024-09-21T06:11:06.016929+0000","last_became_active":"2024-09-21T06:09:36.293937+0000","last_became_peered":"2024-09-21T06:09:36.293937+0000","last_unstale":"2024-09-21T06:11:06.016929+0000","last_undegraded":"2024-09-21T06:11:06.016929+0000","last_fullsized":"2024-09-21T06:11:06.016929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:35.103743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587183+0000","last_change":"2024-09-21T06:09:34.273776+0000","last_active":"2024-09-21T06:10:07.587183+0000","last_peered":"2024-09-21T06:10:07.587183+0000","last_clean":"2024-09-21T06:10:07.587183+0000","last_became_active":"2024-09-21T06:09:34.273390+0000","last_became_peered":"2024-09-21T06:09:34.273390+0000","last_unstale":"2024-09-21T06:10:07.587183+0000","last_undegraded":"2024-09-21T06:10:07.587183+0000","last_fullsized":"2024-09-21T06:10:07.587183+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:29.639228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396798+0000","last_change":"2024-09-21T06:09:34.268184+0000","last_active":"2024-09-21T06:10:07.396798+0000","last_peered":"2024-09-21T06:10:07.396798+0000","last_clean":"2024-09-21T06:10:07.396798+0000","last_became_active":"2024-09-21T06:09:34.267680+0000","last_became_peered":"2024-09-21T06:09:34.267680+0000","last_unstale":"2024-09-21T06:10:07.396798+0000","last_undegraded":"2024-09-21T06:10:07.396798+0000","last_fullsized":"2024-09-21T06:10:07.396798+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:49.432116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.020012+0000","last_change":"2024-09-21T06:09:36.297179+0000","last_active":"2024-09-21T06:11:06.020012+0000","last_peered":"2024-09-21T06:11:06.020012+0000","last_clean":"2024-09-21T06:11:06.020012+0000","last_became_active":"2024-09-21T06:09:36.296324+0000","last_became_peered":"2024-09-21T06:09:36.296324+0000","last_unstale":"2024-09-21T06:11:06.020012+0000","last_undegraded":"2024-09-21T06:11:06.020012+0000","last_fullsized":"2024-09-21T06:11:06.020012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:06.447133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398807+0000","last_change":"2024-09-21T06:09:34.275068+0000","last_active":"2024-09-21T06:10:07.398807+0000","last_peered":"2024-09-21T06:10:07.398807+0000","last_clean":"2024-09-21T06:10:07.398807+0000","last_became_active":"2024-09-21T06:09:34.274401+0000","last_became_peered":"2024-09-21T06:09:34.274401+0000","last_unstale":"2024-09-21T06:10:07.398807+0000","last_undegraded":"2024-09-21T06:10:07.398807+0000","last_fullsized":"2024-09-21T06:10:07.398807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:22:26.066583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011649+0000","last_change":"2024-09-21T06:09:36.294033+0000","last_active":"2024-09-21T06:11:06.011649+0000","last_peered":"2024-09-21T06:11:06.011649+0000","last_clean":"2024-09-21T06:11:06.011649+0000","last_became_active":"2024-09-21T06:09:36.293668+0000","last_became_peered":"2024-09-21T06:09:36.293668+0000","last_unstale":"2024-09-21T06:11:06.011649+0000","last_undegraded":"2024-09-21T06:11:06.011649+0000","last_fullsized":"2024-09-21T06:11:06.011649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:56.300720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.587232+0000","last_change":"2024-09-21T06:09:34.272574+0000","last_active":"2024-09-21T06:10:07.587232+0000","last_peered":"2024-09-21T06:10:07.587232+0000","last_clean":"2024-09-21T06:10:07.587232+0000","last_became_active":"2024-09-21T06:09:34.272162+0000","last_became_peered":"2024-09-21T06:09:34.272162+0000","last_unstale":"2024-09-21T06:10:07.587232+0000","last_undegraded":"2024-09-21T06:10:07.587232+0000","last_fullsized":"2024-09-21T06:10:07.587232+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:55:17.108027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.014591+0000","last_change":"2024-09-21T06:09:36.302035+0000","last_active":"2024-09-21T06:11:06.014591+0000","last_peered":"2024-09-21T06:11:06.014591+0000","last_clean":"2024-09-21T06:11:06.014591+0000","last_became_active":"2024-09-21T06:09:36.301616+0000","last_became_peered":"2024-09-21T06:09:36.301616+0000","last_unstale":"2024-09-21T06:11:06.014591+0000","last_undegraded":"2024-09-21T06:11:06.014591+0000","last_fullsized":"2024-09-21T06:11:06.014591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:47.658000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395089+0000","last_change":"2024-09-21T06:09:34.272365+0000","last_active":"2024-09-21T06:10:07.395089+0000","last_peered":"2024-09-21T06:10:07.395089+0000","last_clean":"2024-09-21T06:10:07.395089+0000","last_became_active":"2024-09-21T06:09:34.271923+0000","last_became_peered":"2024-09-21T06:09:34.271923+0000","last_unstale":"2024-09-21T06:10:07.395089+0000","last_undegraded":"2024-09-21T06:10:07.395089+0000","last_fullsized":"2024-09-21T06:10:07.395089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:32.325170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022625+0000","last_change":"2024-09-21T06:09:36.303737+0000","last_active":"2024-09-21T06:11:06.022625+0000","last_peered":"2024-09-21T06:11:06.022625+0000","last_clean":"2024-09-21T06:11:06.022625+0000","last_became_active":"2024-09-21T06:09:36.301640+0000","last_became_peered":"2024-09-21T06:09:36.301640+0000","last_unstale":"2024-09-21T06:11:06.022625+0000","last_undegraded":"2024-09-21T06:11:06.022625+0000","last_fullsized":"2024-09-21T06:11:06.022625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:03.148061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399080+0000","last_change":"2024-09-21T06:09:34.333216+0000","last_active":"2024-09-21T06:10:07.399080+0000","last_peered":"2024-09-21T06:10:07.399080+0000","last_clean":"2024-09-21T06:10:07.399080+0000","last_became_active":"2024-09-21T06:09:34.332703+0000","last_became_peered":"2024-09-21T06:09:34.332703+0000","last_unstale":"2024-09-21T06:10:07.399080+0000","last_undegraded":"2024-09-21T06:10:07.399080+0000","last_fullsized":"2024-09-21T06:10:07.399080+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:59.743957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.028117+0000","last_change":"2024-09-21T06:09:36.303131+0000","last_active":"2024-09-21T06:11:06.028117+0000","last_peered":"2024-09-21T06:11:06.028117+0000","last_clean":"2024-09-21T06:11:06.028117+0000","last_became_active":"2024-09-21T06:09:36.302769+0000","last_became_peered":"2024-09-21T06:09:36.302769+0000","last_unstale":"2024-09-21T06:11:06.028117+0000","last_undegraded":"2024-09-21T06:11:06.028117+0000","last_fullsized":"2024-09-21T06:11:06.028117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:49.449247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'31","reported_seq":247,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:14.466629+0000","last_change":"2024-09-21T06:09:34.271810+0000","last_active":"2024-09-21T06:11:14.466629+0000","last_peered":"2024-09-21T06:11:14.466629+0000","last_clean":"2024-09-21T06:11:14.466629+0000","last_became_active":"2024-09-21T06:09:34.271142+0000","last_became_peered":"2024-09-21T06:09:34.271142+0000","last_unstale":"2024-09-21T06:11:14.466629+0000","last_undegraded":"2024-09-21T06:11:14.466629+0000","last_fullsized":"2024-09-21T06:11:14.466629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:52.749907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67514,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":162,"num_write":31,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.997726+0000","last_change":"2024-09-21T06:09:36.298274+0000","last_active":"2024-09-21T06:11:05.997726+0000","last_peered":"2024-09-21T06:11:05.997726+0000","last_clean":"2024-09-21T06:11:05.997726+0000","last_became_active":"2024-09-21T06:09:36.297925+0000","last_became_peered":"2024-09-21T06:09:36.297925+0000","last_unstale":"2024-09-21T06:11:05.997726+0000","last_undegraded":"2024-09-21T06:11:05.997726+0000","last_fullsized":"2024-09-21T06:11:05.997726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:49.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394811+0000","last_change":"2024-09-21T06:09:34.268272+0000","last_active":"2024-09-21T06:10:07.394811+0000","last_peered":"2024-09-21T06:10:07.394811+0000","last_clean":"2024-09-21T06:10:07.394811+0000","last_became_active":"2024-09-21T06:09:34.266698+0000","last_became_peered":"2024-09-21T06:09:34.266698+0000","last_unstale":"2024-09-21T06:10:07.394811+0000","last_undegraded":"2024-09-21T06:10:07.394811+0000","last_fullsized":"2024-09-21T06:10:07.394811+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:05.209098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019305+0000","last_change":"2024-09-21T06:09:36.288844+0000","last_active":"2024-09-21T06:11:06.019305+0000","last_peered":"2024-09-21T06:11:06.019305+0000","last_clean":"2024-09-21T06:11:06.019305+0000","last_became_active":"2024-09-21T06:09:36.288187+0000","last_became_peered":"2024-09-21T06:09:36.288187+0000","last_unstale":"2024-09-21T06:11:06.019305+0000","last_undegraded":"2024-09-21T06:11:06.019305+0000","last_fullsized":"2024-09-21T06:11:06.019305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.758116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.398372+0000","last_change":"2024-09-21T06:09:34.274726+0000","last_active":"2024-09-21T06:10:07.398372+0000","last_peered":"2024-09-21T06:10:07.398372+0000","last_clean":"2024-09-21T06:10:07.398372+0000","last_became_active":"2024-09-21T06:09:34.274198+0000","last_became_peered":"2024-09-21T06:09:34.274198+0000","last_unstale":"2024-09-21T06:10:07.398372+0000","last_undegraded":"2024-09-21T06:10:07.398372+0000","last_fullsized":"2024-09-21T06:10:07.398372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:06.050338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019906+0000","last_change":"2024-09-21T06:10:07.401185+0000","last_active":"2024-09-21T06:11:06.019906+0000","last_peered":"2024-09-21T06:11:06.019906+0000","last_clean":"2024-09-21T06:11:06.019906+0000","last_became_active":"2024-09-21T06:10:07.400789+0000","last_became_peered":"2024-09-21T06:10:07.400789+0000","last_unstale":"2024-09-21T06:11:06.019906+0000","last_undegraded":"2024-09-21T06:11:06.019906+0000","last_fullsized":"2024-09-21T06:11:06.019906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:03.201239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394895+0000","last_change":"2024-09-21T06:09:34.270250+0000","last_active":"2024-09-21T06:10:07.394895+0000","last_peered":"2024-09-21T06:10:07.394895+0000","last_clean":"2024-09-21T06:10:07.394895+0000","last_became_active":"2024-09-21T06:09:34.269752+0000","last_became_peered":"2024-09-21T06:09:34.269752+0000","last_unstale":"2024-09-21T06:10:07.394895+0000","last_undegraded":"2024-09-21T06:10:07.394895+0000","last_fullsized":"2024-09-21T06:10:07.394895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:46.417181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018842+0000","last_change":"2024-09-21T06:10:07.588201+0000","last_active":"2024-09-21T06:11:06.018842+0000","last_peered":"2024-09-21T06:11:06.018842+0000","last_clean":"2024-09-21T06:11:06.018842+0000","last_became_active":"2024-09-21T06:10:07.587788+0000","last_became_peered":"2024-09-21T06:10:07.587788+0000","last_unstale":"2024-09-21T06:11:06.018842+0000","last_undegraded":"2024-09-21T06:11:06.018842+0000","last_fullsized":"2024-09-21T06:11:06.018842+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:16.617435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394307+0000","last_change":"2024-09-21T06:09:34.333194+0000","last_active":"2024-09-21T06:10:07.394307+0000","last_peered":"2024-09-21T06:10:07.394307+0000","last_clean":"2024-09-21T06:10:07.394307+0000","last_became_active":"2024-09-21T06:09:34.332668+0000","last_became_peered":"2024-09-21T06:09:34.332668+0000","last_unstale":"2024-09-21T06:10:07.394307+0000","last_undegraded":"2024-09-21T06:10:07.394307+0000","last_fullsized":"2024-09-21T06:10:07.394307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:44.549284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018670+0000","last_change":"2024-09-21T06:10:07.478289+0000","last_active":"2024-09-21T06:11:06.018670+0000","last_peered":"2024-09-21T06:11:06.018670+0000","last_clean":"2024-09-21T06:11:06.018670+0000","last_became_active":"2024-09-21T06:10:07.477828+0000","last_became_peered":"2024-09-21T06:10:07.477828+0000","last_unstale":"2024-09-21T06:11:06.018670+0000","last_undegraded":"2024-09-21T06:11:06.018670+0000","last_fullsized":"2024-09-21T06:11:06.018670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:03.721612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395262+0000","last_change":"2024-09-21T06:09:34.273446+0000","last_active":"2024-09-21T06:10:07.395262+0000","last_peered":"2024-09-21T06:10:07.395262+0000","last_clean":"2024-09-21T06:10:07.395262+0000","last_became_active":"2024-09-21T06:09:34.272794+0000","last_became_peered":"2024-09-21T06:09:34.272794+0000","last_unstale":"2024-09-21T06:10:07.395262+0000","last_undegraded":"2024-09-21T06:10:07.395262+0000","last_fullsized":"2024-09-21T06:10:07.395262+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.203684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022924+0000","last_change":"2024-09-21T06:09:36.292203+0000","last_active":"2024-09-21T06:11:06.022924+0000","last_peered":"2024-09-21T06:11:06.022924+0000","last_clean":"2024-09-21T06:11:06.022924+0000","last_became_active":"2024-09-21T06:09:36.291809+0000","last_became_peered":"2024-09-21T06:09:36.291809+0000","last_unstale":"2024-09-21T06:11:06.022924+0000","last_undegraded":"2024-09-21T06:11:06.022924+0000","last_fullsized":"2024-09-21T06:11:06.022924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:00.712689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.406063+0000","last_change":"2024-09-21T06:10:07.405946+0000","last_active":"2024-09-21T06:10:07.406063+0000","last_peered":"2024-09-21T06:10:07.406063+0000","last_clean":"2024-09-21T06:10:07.406063+0000","last_became_active":"2024-09-21T06:10:07.401620+0000","last_became_peered":"2024-09-21T06:10:07.401620+0000","last_unstale":"2024-09-21T06:10:07.406063+0000","last_undegraded":"2024-09-21T06:10:07.406063+0000","last_fullsized":"2024-09-21T06:10:07.406063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:17.853826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.021136+0000","last_change":"2024-09-21T06:09:36.291591+0000","last_active":"2024-09-21T06:11:06.021136+0000","last_peered":"2024-09-21T06:11:06.021136+0000","last_clean":"2024-09-21T06:11:06.021136+0000","last_became_active":"2024-09-21T06:09:36.291204+0000","last_became_peered":"2024-09-21T06:09:36.291204+0000","last_unstale":"2024-09-21T06:11:06.021136+0000","last_undegraded":"2024-09-21T06:11:06.021136+0000","last_fullsized":"2024-09-21T06:11:06.021136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:53.752822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399215+0000","last_change":"2024-09-21T06:09:34.270426+0000","last_active":"2024-09-21T06:10:07.399215+0000","last_peered":"2024-09-21T06:10:07.399215+0000","last_clean":"2024-09-21T06:10:07.399215+0000","last_became_active":"2024-09-21T06:09:34.270023+0000","last_became_peered":"2024-09-21T06:09:34.270023+0000","last_unstale":"2024-09-21T06:10:07.399215+0000","last_undegraded":"2024-09-21T06:10:07.399215+0000","last_fullsized":"2024-09-21T06:10:07.399215+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:15.927761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.013524+0000","last_change":"2024-09-21T06:10:07.406892+0000","last_active":"2024-09-21T06:11:06.013524+0000","last_peered":"2024-09-21T06:11:06.013524+0000","last_clean":"2024-09-21T06:11:06.013524+0000","last_became_active":"2024-09-21T06:10:07.402546+0000","last_became_peered":"2024-09-21T06:10:07.402546+0000","last_unstale":"2024-09-21T06:11:06.013524+0000","last_undegraded":"2024-09-21T06:11:06.013524+0000","last_fullsized":"2024-09-21T06:11:06.013524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:30.081510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394662+0000","last_change":"2024-09-21T06:09:34.334980+0000","last_active":"2024-09-21T06:10:07.394662+0000","last_peered":"2024-09-21T06:10:07.394662+0000","last_clean":"2024-09-21T06:10:07.394662+0000","last_became_active":"2024-09-21T06:09:34.334512+0000","last_became_peered":"2024-09-21T06:09:34.334512+0000","last_unstale":"2024-09-21T06:10:07.394662+0000","last_undegraded":"2024-09-21T06:10:07.394662+0000","last_fullsized":"2024-09-21T06:10:07.394662+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:31.843850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019250+0000","last_change":"2024-09-21T06:09:36.305221+0000","last_active":"2024-09-21T06:11:06.019250+0000","last_peered":"2024-09-21T06:11:06.019250+0000","last_clean":"2024-09-21T06:11:06.019250+0000","last_became_active":"2024-09-21T06:09:36.303175+0000","last_became_peered":"2024-09-21T06:09:36.303175+0000","last_unstale":"2024-09-21T06:11:06.019250+0000","last_undegraded":"2024-09-21T06:11:06.019250+0000","last_fullsized":"2024-09-21T06:11:06.019250+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:49.850780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395834+0000","last_change":"2024-09-21T06:08:25.558873+0000","last_active":"2024-09-21T06:10:07.395834+0000","last_peered":"2024-09-21T06:10:07.395834+0000","last_clean":"2024-09-21T06:10:07.395834+0000","last_became_active":"2024-09-21T06:08:25.553012+0000","last_became_peered":"2024-09-21T06:08:25.553012+0000","last_unstale":"2024-09-21T06:10:07.395834+0000","last_undegraded":"2024-09-21T06:10:07.395834+0000","last_fullsized":"2024-09-21T06:10:07.395834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:05:06.963445+0000","last_clean_scrub_stamp":"2024-09-21T06:05:06.963445+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:04.339147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395073+0000","last_change":"2024-09-21T06:09:34.268576+0000","last_active":"2024-09-21T06:10:07.395073+0000","last_peered":"2024-09-21T06:10:07.395073+0000","last_clean":"2024-09-21T06:10:07.395073+0000","last_became_active":"2024-09-21T06:09:34.268195+0000","last_became_peered":"2024-09-21T06:09:34.268195+0000","last_unstale":"2024-09-21T06:10:07.395073+0000","last_undegraded":"2024-09-21T06:10:07.395073+0000","last_fullsized":"2024-09-21T06:10:07.395073+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:19:22.875474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.005537+0000","last_change":"2024-09-21T06:09:36.302288+0000","last_active":"2024-09-21T06:11:06.005537+0000","last_peered":"2024-09-21T06:11:06.005537+0000","last_clean":"2024-09-21T06:11:06.005537+0000","last_became_active":"2024-09-21T06:09:36.301867+0000","last_became_peered":"2024-09-21T06:09:36.301867+0000","last_unstale":"2024-09-21T06:11:06.005537+0000","last_undegraded":"2024-09-21T06:11:06.005537+0000","last_fullsized":"2024-09-21T06:11:06.005537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:19.776793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394546+0000","last_change":"2024-09-21T06:09:34.271099+0000","last_active":"2024-09-21T06:10:07.394546+0000","last_peered":"2024-09-21T06:10:07.394546+0000","last_clean":"2024-09-21T06:10:07.394546+0000","last_became_active":"2024-09-21T06:09:34.270700+0000","last_became_peered":"2024-09-21T06:09:34.270700+0000","last_unstale":"2024-09-21T06:10:07.394546+0000","last_undegraded":"2024-09-21T06:10:07.394546+0000","last_fullsized":"2024-09-21T06:10:07.394546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:09.486591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011182+0000","last_change":"2024-09-21T06:09:36.295250+0000","last_active":"2024-09-21T06:11:06.011182+0000","last_peered":"2024-09-21T06:11:06.011182+0000","last_clean":"2024-09-21T06:11:06.011182+0000","last_became_active":"2024-09-21T06:09:36.294890+0000","last_became_peered":"2024-09-21T06:09:36.294890+0000","last_unstale":"2024-09-21T06:11:06.011182+0000","last_undegraded":"2024-09-21T06:11:06.011182+0000","last_fullsized":"2024-09-21T06:11:06.011182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:08.369447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.396950+0000","last_change":"2024-09-21T06:09:34.332528+0000","last_active":"2024-09-21T06:10:07.396950+0000","last_peered":"2024-09-21T06:10:07.396950+0000","last_clean":"2024-09-21T06:10:07.396950+0000","last_became_active":"2024-09-21T06:09:34.332107+0000","last_became_peered":"2024-09-21T06:09:34.332107+0000","last_unstale":"2024-09-21T06:10:07.396950+0000","last_undegraded":"2024-09-21T06:10:07.396950+0000","last_fullsized":"2024-09-21T06:10:07.396950+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:41.850416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.015296+0000","last_change":"2024-09-21T06:09:36.292249+0000","last_active":"2024-09-21T06:11:06.015296+0000","last_peered":"2024-09-21T06:11:06.015296+0000","last_clean":"2024-09-21T06:11:06.015296+0000","last_became_active":"2024-09-21T06:09:36.291502+0000","last_became_peered":"2024-09-21T06:09:36.291502+0000","last_unstale":"2024-09-21T06:11:06.015296+0000","last_undegraded":"2024-09-21T06:11:06.015296+0000","last_fullsized":"2024-09-21T06:11:06.015296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:06.569266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394998+0000","last_change":"2024-09-21T06:09:34.272172+0000","last_active":"2024-09-21T06:10:07.394998+0000","last_peered":"2024-09-21T06:10:07.394998+0000","last_clean":"2024-09-21T06:10:07.394998+0000","last_became_active":"2024-09-21T06:09:34.271747+0000","last_became_peered":"2024-09-21T06:09:34.271747+0000","last_unstale":"2024-09-21T06:10:07.394998+0000","last_undegraded":"2024-09-21T06:10:07.394998+0000","last_fullsized":"2024-09-21T06:10:07.394998+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:49.620311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018487+0000","last_change":"2024-09-21T06:09:36.297800+0000","last_active":"2024-09-21T06:11:06.018487+0000","last_peered":"2024-09-21T06:11:06.018487+0000","last_clean":"2024-09-21T06:11:06.018487+0000","last_became_active":"2024-09-21T06:09:36.297428+0000","last_became_peered":"2024-09-21T06:09:36.297428+0000","last_unstale":"2024-09-21T06:11:06.018487+0000","last_undegraded":"2024-09-21T06:11:06.018487+0000","last_fullsized":"2024-09-21T06:11:06.018487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:48.403021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.401748+0000","last_change":"2024-09-21T06:10:07.401640+0000","last_active":"2024-09-21T06:10:07.401748+0000","last_peered":"2024-09-21T06:10:07.401748+0000","last_clean":"2024-09-21T06:10:07.401748+0000","last_became_active":"2024-09-21T06:10:07.401242+0000","last_became_peered":"2024-09-21T06:10:07.401242+0000","last_unstale":"2024-09-21T06:10:07.401748+0000","last_undegraded":"2024-09-21T06:10:07.401748+0000","last_fullsized":"2024-09-21T06:10:07.401748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:25.335694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.004481+0000","last_change":"2024-09-21T06:09:36.302894+0000","last_active":"2024-09-21T06:11:06.004481+0000","last_peered":"2024-09-21T06:11:06.004481+0000","last_clean":"2024-09-21T06:11:06.004481+0000","last_became_active":"2024-09-21T06:09:36.302477+0000","last_became_peered":"2024-09-21T06:09:36.302477+0000","last_unstale":"2024-09-21T06:11:06.004481+0000","last_undegraded":"2024-09-21T06:11:06.004481+0000","last_fullsized":"2024-09-21T06:11:06.004481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:58.990354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395763+0000","last_change":"2024-09-21T06:09:34.333761+0000","last_active":"2024-09-21T06:10:07.395763+0000","last_peered":"2024-09-21T06:10:07.395763+0000","last_clean":"2024-09-21T06:10:07.395763+0000","last_became_active":"2024-09-21T06:09:34.332708+0000","last_became_peered":"2024-09-21T06:09:34.332708+0000","last_unstale":"2024-09-21T06:10:07.395763+0000","last_undegraded":"2024-09-21T06:10:07.395763+0000","last_fullsized":"2024-09-21T06:10:07.395763+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:50.757098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.010167+0000","last_change":"2024-09-21T06:09:36.295229+0000","last_active":"2024-09-21T06:11:06.010167+0000","last_peered":"2024-09-21T06:11:06.010167+0000","last_clean":"2024-09-21T06:11:06.010167+0000","last_became_active":"2024-09-21T06:09:36.294825+0000","last_became_peered":"2024-09-21T06:09:36.294825+0000","last_unstale":"2024-09-21T06:11:06.010167+0000","last_undegraded":"2024-09-21T06:11:06.010167+0000","last_fullsized":"2024-09-21T06:11:06.010167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:05.454156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'11","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.300858+0000","last_change":"2024-09-21T06:09:34.268548+0000","last_active":"2024-09-21T06:11:06.300858+0000","last_peered":"2024-09-21T06:11:06.300858+0000","last_clean":"2024-09-21T06:11:06.300858+0000","last_became_active":"2024-09-21T06:09:34.268191+0000","last_became_peered":"2024-09-21T06:09:34.268191+0000","last_unstale":"2024-09-21T06:11:06.300858+0000","last_undegraded":"2024-09-21T06:11:06.300858+0000","last_fullsized":"2024-09-21T06:11:06.300858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:35.628558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022143+0000","last_change":"2024-09-21T06:09:36.298711+0000","last_active":"2024-09-21T06:11:06.022143+0000","last_peered":"2024-09-21T06:11:06.022143+0000","last_clean":"2024-09-21T06:11:06.022143+0000","last_became_active":"2024-09-21T06:09:36.296926+0000","last_became_peered":"2024-09-21T06:09:36.296926+0000","last_unstale":"2024-09-21T06:11:06.022143+0000","last_undegraded":"2024-09-21T06:11:06.022143+0000","last_fullsized":"2024-09-21T06:11:06.022143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:01.959224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395509+0000","last_change":"2024-09-21T06:09:34.318529+0000","last_active":"2024-09-21T06:10:07.395509+0000","last_peered":"2024-09-21T06:10:07.395509+0000","last_clean":"2024-09-21T06:10:07.395509+0000","last_became_active":"2024-09-21T06:09:34.318162+0000","last_became_peered":"2024-09-21T06:09:34.318162+0000","last_unstale":"2024-09-21T06:10:07.395509+0000","last_undegraded":"2024-09-21T06:10:07.395509+0000","last_fullsized":"2024-09-21T06:10:07.395509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:23.335548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.019541+0000","last_change":"2024-09-21T06:09:36.302246+0000","last_active":"2024-09-21T06:11:06.019541+0000","last_peered":"2024-09-21T06:11:06.019541+0000","last_clean":"2024-09-21T06:11:06.019541+0000","last_became_active":"2024-09-21T06:09:36.301779+0000","last_became_peered":"2024-09-21T06:09:36.301779+0000","last_unstale":"2024-09-21T06:11:06.019541+0000","last_undegraded":"2024-09-21T06:11:06.019541+0000","last_fullsized":"2024-09-21T06:11:06.019541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:08.785356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.476406+0000","last_change":"2024-09-21T06:09:34.334819+0000","last_active":"2024-09-21T06:10:07.476406+0000","last_peered":"2024-09-21T06:10:07.476406+0000","last_clean":"2024-09-21T06:10:07.476406+0000","last_became_active":"2024-09-21T06:09:34.334451+0000","last_became_peered":"2024-09-21T06:09:34.334451+0000","last_unstale":"2024-09-21T06:10:07.476406+0000","last_undegraded":"2024-09-21T06:10:07.476406+0000","last_fullsized":"2024-09-21T06:10:07.476406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:51.025005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:05.970288+0000","last_change":"2024-09-21T06:09:36.298617+0000","last_active":"2024-09-21T06:11:05.970288+0000","last_peered":"2024-09-21T06:11:05.970288+0000","last_clean":"2024-09-21T06:11:05.970288+0000","last_became_active":"2024-09-21T06:09:36.298230+0000","last_became_peered":"2024-09-21T06:09:36.298230+0000","last_unstale":"2024-09-21T06:11:05.970288+0000","last_undegraded":"2024-09-21T06:11:05.970288+0000","last_fullsized":"2024-09-21T06:11:05.970288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:49.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.393700+0000","last_change":"2024-09-21T06:09:34.270199+0000","last_active":"2024-09-21T06:10:07.393700+0000","last_peered":"2024-09-21T06:10:07.393700+0000","last_clean":"2024-09-21T06:10:07.393700+0000","last_became_active":"2024-09-21T06:09:34.269732+0000","last_became_peered":"2024-09-21T06:09:34.269732+0000","last_unstale":"2024-09-21T06:10:07.393700+0000","last_undegraded":"2024-09-21T06:10:07.393700+0000","last_fullsized":"2024-09-21T06:10:07.393700+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:15.242049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.008331+0000","last_change":"2024-09-21T06:09:36.292566+0000","last_active":"2024-09-21T06:11:06.008331+0000","last_peered":"2024-09-21T06:11:06.008331+0000","last_clean":"2024-09-21T06:11:06.008331+0000","last_became_active":"2024-09-21T06:09:36.291759+0000","last_became_peered":"2024-09-21T06:09:36.291759+0000","last_unstale":"2024-09-21T06:11:06.008331+0000","last_undegraded":"2024-09-21T06:11:06.008331+0000","last_fullsized":"2024-09-21T06:11:06.008331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.351166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394466+0000","last_change":"2024-09-21T06:09:34.270516+0000","last_active":"2024-09-21T06:10:07.394466+0000","last_peered":"2024-09-21T06:10:07.394466+0000","last_clean":"2024-09-21T06:10:07.394466+0000","last_became_active":"2024-09-21T06:09:34.269948+0000","last_became_peered":"2024-09-21T06:09:34.269948+0000","last_unstale":"2024-09-21T06:10:07.394466+0000","last_undegraded":"2024-09-21T06:10:07.394466+0000","last_fullsized":"2024-09-21T06:10:07.394466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:13.638325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.018378+0000","last_change":"2024-09-21T06:09:36.298060+0000","last_active":"2024-09-21T06:11:06.018378+0000","last_peered":"2024-09-21T06:11:06.018378+0000","last_clean":"2024-09-21T06:11:06.018378+0000","last_became_active":"2024-09-21T06:09:36.296969+0000","last_became_peered":"2024-09-21T06:09:36.296969+0000","last_unstale":"2024-09-21T06:11:06.018378+0000","last_undegraded":"2024-09-21T06:11:06.018378+0000","last_fullsized":"2024-09-21T06:11:06.018378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:36.193259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.403122+0000","last_change":"2024-09-21T06:10:07.402379+0000","last_active":"2024-09-21T06:10:07.403122+0000","last_peered":"2024-09-21T06:10:07.403122+0000","last_clean":"2024-09-21T06:10:07.403122+0000","last_became_active":"2024-09-21T06:10:07.396828+0000","last_became_peered":"2024-09-21T06:10:07.396828+0000","last_unstale":"2024-09-21T06:10:07.403122+0000","last_undegraded":"2024-09-21T06:10:07.403122+0000","last_fullsized":"2024-09-21T06:10:07.403122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:26.181343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026247+0000","last_change":"2024-09-21T06:09:36.306245+0000","last_active":"2024-09-21T06:11:06.026247+0000","last_peered":"2024-09-21T06:11:06.026247+0000","last_clean":"2024-09-21T06:11:06.026247+0000","last_became_active":"2024-09-21T06:09:36.305772+0000","last_became_peered":"2024-09-21T06:09:36.305772+0000","last_unstale":"2024-09-21T06:11:06.026247+0000","last_undegraded":"2024-09-21T06:11:06.026247+0000","last_fullsized":"2024-09-21T06:11:06.026247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:45.078769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.022587+0000","last_change":"2024-09-21T06:09:36.302367+0000","last_active":"2024-09-21T06:11:06.022587+0000","last_peered":"2024-09-21T06:11:06.022587+0000","last_clean":"2024-09-21T06:11:06.022587+0000","last_became_active":"2024-09-21T06:09:36.301924+0000","last_became_peered":"2024-09-21T06:09:36.301924+0000","last_unstale":"2024-09-21T06:11:06.022587+0000","last_undegraded":"2024-09-21T06:11:06.022587+0000","last_fullsized":"2024-09-21T06:11:06.022587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:44.963170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395391+0000","last_change":"2024-09-21T06:09:34.270668+0000","last_active":"2024-09-21T06:10:07.395391+0000","last_peered":"2024-09-21T06:10:07.395391+0000","last_clean":"2024-09-21T06:10:07.395391+0000","last_became_active":"2024-09-21T06:09:34.270256+0000","last_became_peered":"2024-09-21T06:09:34.270256+0000","last_unstale":"2024-09-21T06:10:07.395391+0000","last_undegraded":"2024-09-21T06:10:07.395391+0000","last_fullsized":"2024-09-21T06:10:07.395391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:58:27.497157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.011566+0000","last_change":"2024-09-21T06:09:36.302320+0000","last_active":"2024-09-21T06:11:06.011566+0000","last_peered":"2024-09-21T06:11:06.011566+0000","last_clean":"2024-09-21T06:11:06.011566+0000","last_became_active":"2024-09-21T06:09:36.301729+0000","last_became_peered":"2024-09-21T06:09:36.301729+0000","last_unstale":"2024-09-21T06:11:06.011566+0000","last_undegraded":"2024-09-21T06:11:06.011566+0000","last_fullsized":"2024-09-21T06:11:06.011566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:00.015644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.395635+0000","last_change":"2024-09-21T06:09:34.274466+0000","last_active":"2024-09-21T06:10:07.395635+0000","last_peered":"2024-09-21T06:10:07.395635+0000","last_clean":"2024-09-21T06:10:07.395635+0000","last_became_active":"2024-09-21T06:09:34.274001+0000","last_became_peered":"2024-09-21T06:09:34.274001+0000","last_unstale":"2024-09-21T06:10:07.395635+0000","last_undegraded":"2024-09-21T06:10:07.395635+0000","last_fullsized":"2024-09-21T06:10:07.395635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:20.314653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.026621+0000","last_change":"2024-09-21T06:09:36.304832+0000","last_active":"2024-09-21T06:11:06.026621+0000","last_peered":"2024-09-21T06:11:06.026621+0000","last_clean":"2024-09-21T06:11:06.026621+0000","last_became_active":"2024-09-21T06:09:36.304298+0000","last_became_peered":"2024-09-21T06:09:36.304298+0000","last_unstale":"2024-09-21T06:11:06.026621+0000","last_undegraded":"2024-09-21T06:11:06.026621+0000","last_fullsized":"2024-09-21T06:11:06.026621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:04.202200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.394530+0000","last_change":"2024-09-21T06:09:34.332517+0000","last_active":"2024-09-21T06:10:07.394530+0000","last_peered":"2024-09-21T06:10:07.394530+0000","last_clean":"2024-09-21T06:10:07.394530+0000","last_became_active":"2024-09-21T06:09:34.332162+0000","last_became_peered":"2024-09-21T06:09:34.332162+0000","last_unstale":"2024-09-21T06:10:07.394530+0000","last_undegraded":"2024-09-21T06:10:07.394530+0000","last_fullsized":"2024-09-21T06:10:07.394530+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:18.266892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:11:06.013863+0000","last_change":"2024-09-21T06:09:36.300238+0000","last_active":"2024-09-21T06:11:06.013863+0000","last_peered":"2024-09-21T06:11:06.013863+0000","last_clean":"2024-09-21T06:11:06.013863+0000","last_became_active":"2024-09-21T06:09:36.298436+0000","last_became_peered":"2024-09-21T06:09:36.298436+0000","last_unstale":"2024-09-21T06:11:06.013863+0000","last_undegraded":"2024-09-21T06:11:06.013863+0000","last_fullsized":"2024-09-21T06:11:06.013863+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:35.255730+0000","last_clean_scrub_stamp":"2024-09-21T06:09:35.255730+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:13.301971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T06:10:07.399889+0000","last_change":"2024-09-21T06:09:34.267254+0000","last_active":"2024-09-21T06:10:07.399889+0000","last_peered":"2024-09-21T06:10:07.399889+0000","last_clean":"2024-09-21T06:10:07.399889+0000","last_became_active":"2024-09-21T06:09:34.266787+0000","last_became_peered":"2024-09-21T06:09:34.266787+0000","last_unstale":"2024-09-21T06:10:07.399889+0000","last_undegraded":"2024-09-21T06:10:07.399889+0000","last_fullsized":"2024-09-21T06:10:07.399889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:09:33.244563+0000","last_clean_scrub_stamp":"2024-09-21T06:09:33.244563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:04.123799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68312,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":197,"num_read_kb":166,"num_write":91,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743456,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":839680,"data_stored":415673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1527808,"data_stored":1073866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":416372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266736,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":416277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1441792,"data_stored":1006740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1449984,"data_stored":1006340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":415894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378702,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40404,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704748,"statfs":{"total":95995035648,"available":95953661952,"internally_reserved":0,"allocated":921600,"data_stored":483221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":483187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":415763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:11:17.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:11:17.904 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:18.191 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:17 smithi031 ceph-mon[31237]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:18.191 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:17 smithi031 ceph-mon[31237]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:18.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:17 smithi110 ceph-mon[34550]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:18.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:17 smithi110 ceph-mon[34550]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:18.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:17 smithi107 ceph-mon[34380]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:18.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:17 smithi107 ceph-mon[34380]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 -- 172.21.15.31:0/2205882414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c10ddf0 msgr2=0x7fd52c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 --2- 172.21.15.31:0/2205882414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c10ddf0 0x7fd52c110300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd524009ca0 tx=0x7fd52402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 -- 172.21.15.31:0/2205882414 shutdown_connections 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 --2- 172.21.15.31:0/2205882414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c10ddf0 0x7fd52c110300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 --2- 172.21.15.31:0/2205882414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c105270 0x7fd52c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 --2- 172.21.15.31:0/2205882414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c104840 0x7fd52c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 -- 172.21.15.31:0/2205882414 >> 172.21.15.31:0/2205882414 conn(0x7fd52c079730 msgr2=0x7fd52c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 -- 172.21.15.31:0/2205882414 shutdown_connections 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 -- 172.21.15.31:0/2205882414 wait complete. 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.412+0000 7fd53429e640 1 Processor -- start 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 -- start start 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c104840 0x7fd52c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 0x7fd52c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c10ddf0 0x7fd52c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c113a90 con 0x7fd52c105270 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c113910 con 0x7fd52c10ddf0 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53429e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c113c10 con 0x7fd52c104840 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 0x7fd52c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 0x7fd52c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39652/0 (socket says 172.21.15.31:39652) 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 -- 172.21.15.31:0/4085431622 learned_addr learned my addr 172.21.15.31:0/4085431622 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd53203a640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c104840 0x7fd52c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 -- 172.21.15.31:0/4085431622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c104840 msgr2=0x7fd52c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c104840 0x7fd52c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 -- 172.21.15.31:0/4085431622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c10ddf0 msgr2=0x7fd52c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c10ddf0 0x7fd52c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 -- 172.21.15.31:0/4085431622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c1a0470 con 0x7fd52c105270 2024-09-21T06:11:18.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.413+0000 7fd531839640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 0x7fd52c19aaa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00d740 tx=0x7fd51c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.414+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c017070 con 0x7fd52c105270 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.414+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c1a0760 con 0x7fd52c105270 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.414+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd51c00de50 con 0x7fd52c105270 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.414+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00f620 con 0x7fd52c105270 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.414+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1a6ec0 con 0x7fd52c105270 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.415+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c104c50 con 0x7fd52c105270 2024-09-21T06:11:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.417+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fd51c01a070 con 0x7fd52c105270 2024-09-21T06:11:18.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.417+0000 7fd5237fe640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6560 0x7fd5180b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:18.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.418+0000 7fd53203a640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6560 0x7fd5180b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:18.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.418+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd51c0e6e40 con 0x7fd52c105270 2024-09-21T06:11:18.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.418+0000 7fd53203a640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6560 0x7fd5180b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd528005fd0 tx=0x7fd528005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:18.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.420+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd51c0ad5f0 con 0x7fd52c105270 2024-09-21T06:11:18.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.653+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd52c19b630 con 0x7fd52c105270 2024-09-21T06:11:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.654+0000 7fd5237fe640 1 -- 172.21.15.31:0/4085431622 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fd51c0b19d0 con 0x7fd52c105270 2024-09-21T06:11:18.656 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:18.656 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:11:18.656 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6560 msgr2=0x7fd5180b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6560 0x7fd5180b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd528005fd0 tx=0x7fd528005850 comp rx=0 tx=0).stop 2024-09-21T06:11:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 msgr2=0x7fd52c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 0x7fd52c19aaa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00d740 tx=0x7fd51c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:11:18.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 shutdown_connections 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd52c10ddf0 0x7fd52c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5180b6560 0x7fd5180b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c105270 0x7fd52c19aaa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 --2- 172.21.15.31:0/4085431622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd52c104840 0x7fd52c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.656+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 >> 172.21.15.31:0/4085431622 conn(0x7fd52c079730 msgr2=0x7fd52c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.657+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 shutdown_connections 2024-09-21T06:11:18.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:18.657+0000 7fd53429e640 1 -- 172.21.15.31:0/4085431622 wait complete. 2024-09-21T06:11:19.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:18 smithi110 ceph-mon[34550]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:19.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:18 smithi110 ceph-mon[34550]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:11:19.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:18 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4085431622' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:19.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:18 smithi107 ceph-mon[34380]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:19.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:18 smithi107 ceph-mon[34380]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:11:19.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:18 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4085431622' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:19.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:18 smithi031 ceph-mon[31237]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:11:19.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:18 smithi031 ceph-mon[31237]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:11:19.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:18 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4085431622' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:19.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:11:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.164+0000 7feb78acc640 1 -- 172.21.15.31:0/104558246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb74108d20 msgr2=0x7feb7410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.164+0000 7feb78acc640 1 --2- 172.21.15.31:0/104558246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb74108d20 0x7feb7410f550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009ca0 tx=0x7feb6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 -- 172.21.15.31:0/104558246 shutdown_connections 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 --2- 172.21.15.31:0/104558246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb74108d20 0x7feb7410f550 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 --2- 172.21.15.31:0/104558246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb741083d0 0x7feb741087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 --2- 172.21.15.31:0/104558246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb74102ae0 0x7feb74102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 -- 172.21.15.31:0/104558246 >> 172.21.15.31:0/104558246 conn(0x7feb74079760 msgr2=0x7feb74079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 -- 172.21.15.31:0/104558246 shutdown_connections 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.165+0000 7feb78acc640 1 -- 172.21.15.31:0/104558246 wait complete. 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 Processor -- start 2024-09-21T06:11:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 -- start start 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74102ae0 0x7feb7419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 0x7feb7419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb74108d20 0x7feb7419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb741129f0 con 0x7feb741083d0 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb74112870 con 0x7feb74102ae0 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb78acc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb74112b70 con 0x7feb74108d20 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb71d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 0x7feb7419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb71d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 0x7feb7419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46192/0 (socket says 172.21.15.31:46192) 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb71d9b640 1 -- 172.21.15.31:0/1386773859 learned_addr learned my addr 172.21.15.31:0/1386773859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb72d9d640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb74108d20 0x7feb7419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.166+0000 7feb7259c640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74102ae0 0x7feb7419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb71d9b640 1 -- 172.21.15.31:0/1386773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb74108d20 msgr2=0x7feb7419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb71d9b640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb74108d20 0x7feb7419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb71d9b640 1 -- 172.21.15.31:0/1386773859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74102ae0 msgr2=0x7feb7419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb71d9b640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74102ae0 0x7feb7419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb71d9b640 1 -- 172.21.15.31:0/1386773859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb741a6570 con 0x7feb741083d0 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb71d9b640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 0x7feb7419b820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feb6800b450 tx=0x7feb6800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68015020 con 0x7feb741083d0 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb68011d40 con 0x7feb741083d0 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb741a6860 con 0x7feb741083d0 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800f340 con 0x7feb741083d0 2024-09-21T06:11:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.167+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741a6da0 con 0x7feb741083d0 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.168+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb74103d10 con 0x7feb741083d0 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.170+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7feb68018020 con 0x7feb741083d0 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.170+0000 7feb637fe640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb5c0b6590 0x7feb5c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.171+0000 7feb7259c640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb5c0b6590 0x7feb5c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.171+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feb680e6e20 con 0x7feb741083d0 2024-09-21T06:11:20.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.171+0000 7feb7259c640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb5c0b6590 0x7feb5c0b8aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb64005fd0 tx=0x7feb64005e40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.175+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feb6801f020 con 0x7feb741083d0 2024-09-21T06:11:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.366+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb7410a580 con 0x7feb741083d0 2024-09-21T06:11:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.368+0000 7feb637fe640 1 -- 172.21.15.31:0/1386773859 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23089 (secure 0 0 0) 0x7feb6800d040 con 0x7feb741083d0 2024-09-21T06:11:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:20.371 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:10:07.387838+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:11:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb5c0b6590 msgr2=0x7feb5c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb5c0b6590 0x7feb5c0b8aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb64005fd0 tx=0x7feb64005e40 comp rx=0 tx=0).stop 2024-09-21T06:11:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 msgr2=0x7feb7419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 0x7feb7419b820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feb6800b450 tx=0x7feb6800b910 comp rx=0 tx=0).stop 2024-09-21T06:11:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 shutdown_connections 2024-09-21T06:11:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb74108d20 0x7feb7419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb5c0b6590 0x7feb5c0b8aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb741083d0 0x7feb7419b820 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 --2- 172.21.15.31:0/1386773859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74102ae0 0x7feb7419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 >> 172.21.15.31:0/1386773859 conn(0x7feb74079760 msgr2=0x7feb741052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 shutdown_connections 2024-09-21T06:11:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:20.370+0000 7feb78acc640 1 -- 172.21.15.31:0/1386773859 wait complete. 2024-09-21T06:11:21.021 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:20 smithi031 ceph-mon[31237]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:11:21.021 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:20 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1386773859' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:21.058 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T06:11:21.058 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:11:21.068 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:11:21.068 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:11:21.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T06:11:21.069 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T06:11:21.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:11:21.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:11:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 -- 172.21.15.31:0/1297723136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 msgr2=0x7f9934104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 --2- 172.21.15.31:0/1297723136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f9934104d10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f991c009900 tx=0x7f991c02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 -- 172.21.15.31:0/1297723136 shutdown_connections 2024-09-21T06:11:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 --2- 172.21.15.31:0/1297723136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f993410dd60 0x7f9934110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 --2- 172.21.15.31:0/1297723136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934105250 0x7f993410d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 --2- 172.21.15.31:0/1297723136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f9934104d10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 -- 172.21.15.31:0/1297723136 >> 172.21.15.31:0/1297723136 conn(0x7f993407a740 msgr2=0x7f993407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 -- 172.21.15.31:0/1297723136 shutdown_connections 2024-09-21T06:11:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.195+0000 7f993ac41640 1 -- 172.21.15.31:0/1297723136 wait complete. 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 Processor -- start 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 -- start start 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f993419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934105250 0x7f993419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f993410dd60 0x7f993419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f99341139e0 con 0x7f9934104900 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9934113860 con 0x7f9934105250 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f993ac41640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9934113b60 con 0x7f993410dd60 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f99389dd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f993419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f99389dd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f993419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46206/0 (socket says 172.21.15.31:46206) 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f99389dd640 1 -- 172.21.15.31:0/1639812051 learned_addr learned my addr 172.21.15.31:0/1639812051 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.196+0000 7f99391de640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f993410dd60 0x7f993419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f9933fff640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934105250 0x7f993419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f99389dd640 1 -- 172.21.15.31:0/1639812051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f993410dd60 msgr2=0x7f993419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f99389dd640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f993410dd60 0x7f993419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f99389dd640 1 -- 172.21.15.31:0/1639812051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934105250 msgr2=0x7f993419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f99389dd640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934105250 0x7f993419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f99389dd640 1 -- 172.21.15.31:0/1639812051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99341a0330 con 0x7f9934104900 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f99389dd640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f993419a400 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f991c005940 tx=0x7f991c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c03f030 con 0x7f9934104900 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99341a05c0 con 0x7f9934104900 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991c003aa0 con 0x7f9934104900 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.197+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c035070 con 0x7f9934104900 2024-09-21T06:11:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.198+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a6db0 con 0x7f9934104900 2024-09-21T06:11:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.199+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9934104d10 con 0x7f9934104900 2024-09-21T06:11:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.200+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f991c042030 con 0x7f9934104900 2024-09-21T06:11:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.201+0000 7f9931ffb640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99200b6560 0x7f99200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.201+0000 7f9933fff640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99200b6560 0x7f99200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.201+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f991c107960 con 0x7f9934104900 2024-09-21T06:11:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.202+0000 7f9933fff640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99200b6560 0x7f99200b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f993410fb90 tx=0x7f9924005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.210+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f991c0d2490 con 0x7f9934104900 2024-09-21T06:11:21.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:20 smithi110 ceph-mon[34550]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:11:21.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:20 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1386773859' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:21.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:20 smithi107 ceph-mon[34380]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-09-21T06:11:21.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:20 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1386773859' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:11:21.355 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:21.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.439+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9934101c70 con 0x7f99200b6560 2024-09-21T06:11:21.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.604+0000 7f9931ffb640 1 -- 172.21.15.31:0/1639812051 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9934101c70 con 0x7f99200b6560 2024-09-21T06:11:21.607 INFO:teuthology.orchestra.run.smithi031.stdout:Schedule set for path / 2024-09-21T06:11:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99200b6560 msgr2=0x7f99200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99200b6560 0x7f99200b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f993410fb90 tx=0x7f9924005f90 comp rx=0 tx=0).stop 2024-09-21T06:11:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 msgr2=0x7f993419a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f993419a400 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f991c005940 tx=0x7f991c004270 comp rx=0 tx=0).stop 2024-09-21T06:11:21.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 shutdown_connections 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f993410dd60 0x7f993419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934105250 0x7f993419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f99200b6560 0x7f99200b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 --2- 172.21.15.31:0/1639812051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9934104900 0x7f993419a400 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 >> 172.21.15.31:0/1639812051 conn(0x7f993407a740 msgr2=0x7f9934100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 shutdown_connections 2024-09-21T06:11:21.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.606+0000 7f993ac41640 1 -- 172.21.15.31:0/1639812051 wait complete. 2024-09-21T06:11:21.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T06:11:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 -- 172.21.15.31:0/2185947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a80108310 msgr2=0x7f8a80108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 --2- 172.21.15.31:0/2185947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a80108310 0x7f8a80108720 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009900 tx=0x7f8a7402a410 comp rx=0 tx=0).stop 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 -- 172.21.15.31:0/2185947572 shutdown_connections 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 --2- 172.21.15.31:0/2185947572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a80108c60 0x7f8a8010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 --2- 172.21.15.31:0/2185947572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a80108310 0x7f8a80108720 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 --2- 172.21.15.31:0/2185947572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a801029e0 0x7f8a80102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 -- 172.21.15.31:0/2185947572 >> 172.21.15.31:0/2185947572 conn(0x7f8a800fe3c0 msgr2=0x7f8a80100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 -- 172.21.15.31:0/2185947572 shutdown_connections 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.738+0000 7f8a852bd640 1 -- 172.21.15.31:0/2185947572 wait complete. 2024-09-21T06:11:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 Processor -- start 2024-09-21T06:11:21.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 -- start start 2024-09-21T06:11:21.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 0x7f8a80199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a80108310 0x7f8a801998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a80108c60 0x7f8a8019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80112920 con 0x7f8a801029e0 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801127a0 con 0x7f8a80108310 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a852bd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80112aa0 con 0x7f8a80108c60 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.739+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 0x7f8a80199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 0x7f8a80199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46244/0 (socket says 172.21.15.31:46244) 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 -- 172.21.15.31:0/4147435066 learned_addr learned my addr 172.21.15.31:0/4147435066 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7e7fc640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a80108310 0x7f8a801998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7f7fe640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a80108c60 0x7f8a8019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 -- 172.21.15.31:0/4147435066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a80108c60 msgr2=0x7f8a8019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a80108c60 0x7f8a8019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 -- 172.21.15.31:0/4147435066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a80108310 msgr2=0x7f8a801998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a80108310 0x7f8a801998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 -- 172.21.15.31:0/4147435066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8019f270 con 0x7f8a801029e0 2024-09-21T06:11:21.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a7effd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 0x7f8a80199340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000d950 tx=0x7f8a7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a70017070 con 0x7f8a801029e0 2024-09-21T06:11:21.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8019f560 con 0x7f8a801029e0 2024-09-21T06:11:21.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a70009c30 con 0x7f8a801029e0 2024-09-21T06:11:21.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000f620 con 0x7f8a801029e0 2024-09-21T06:11:21.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.740+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801a5cf0 con 0x7f8a801029e0 2024-09-21T06:11:21.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.741+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a80103c50 con 0x7f8a801029e0 2024-09-21T06:11:21.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.743+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f8a70010460 con 0x7f8a801029e0 2024-09-21T06:11:21.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.744+0000 7f8a67fff640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a680b6560 0x7f8a680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.744+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8a700e70b0 con 0x7f8a801029e0 2024-09-21T06:11:21.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.744+0000 7f8a7e7fc640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a680b6560 0x7f8a680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.744+0000 7f8a7e7fc640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a680b6560 0x7f8a680b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009230 tx=0x7f8a74031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.747+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8a700ad860 con 0x7f8a801029e0 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 -- 172.21.15.31:0/2631127810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f035010de10 msgr2=0x7f0350110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 --2- 172.21.15.31:0/2631127810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f035010de10 0x7f0350110300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0348009ca0 tx=0x7f034802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 -- 172.21.15.31:0/2631127810 shutdown_connections 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 --2- 172.21.15.31:0/2631127810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f035010de10 0x7f0350110300 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 --2- 172.21.15.31:0/2631127810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0350105300 0x7f035010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 --2- 172.21.15.31:0/2631127810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03501049b0 0x7f0350104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 -- 172.21.15.31:0/2631127810 >> 172.21.15.31:0/2631127810 conn(0x7f0350079730 msgr2=0x7f0350079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.904+0000 7f0357cae640 1 -- 172.21.15.31:0/2631127810 shutdown_connections 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.905+0000 7f0357cae640 1 -- 172.21.15.31:0/2631127810 wait complete. 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.905+0000 7f0357cae640 1 Processor -- start 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.905+0000 7f0357cae640 1 -- start start 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0357cae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 0x7f035019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0357cae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0350105300 0x7f035019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0357cae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f035010de10 0x7f035019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0357cae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0350113a90 con 0x7f03501049b0 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0357cae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0350113910 con 0x7f035010de10 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0357cae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0350113c10 con 0x7f0350105300 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 0x7f035019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 0x7f035019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46262/0 (socket says 172.21.15.31:46262) 2024-09-21T06:11:21.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 -- 172.21.15.31:0/2839241644 learned_addr learned my addr 172.21.15.31:0/2839241644 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355249640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0350105300 0x7f035019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f035624b640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f035010de10 0x7f035019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 -- 172.21.15.31:0/2839241644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0350105300 msgr2=0x7f035019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0350105300 0x7f035019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 -- 172.21.15.31:0/2839241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f035010de10 msgr2=0x7f035019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f035010de10 0x7f035019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 -- 172.21.15.31:0/2839241644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501a03d0 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f035624b640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f035010de10 0x7f035019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.906+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 0x7f035019a460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e9c0 tx=0x7f034c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.907+0000 7f0346ffd640 1 -- 172.21.15.31:0/2839241644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c00cd80 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.907+0000 7f0346ffd640 1 -- 172.21.15.31:0/2839241644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034c0035f0 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.907+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501a06c0 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.907+0000 7f0346ffd640 1 -- 172.21.15.31:0/2839241644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c010690 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.907+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a6e20 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.909+0000 7f035e222740 1 Processor -- start 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.909+0000 7f035e222740 1 -- start start 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0346ffd640 1 -- 172.21.15.31:0/2839241644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f034c004000 con 0x7f03501049b0 2024-09-21T06:11:21.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035e222740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d2063430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035e222740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d2063970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035e222740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2063eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035e222740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5573d1fd9e60 con 0x5573d1c93490 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035e222740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5573d1fc9290 con 0x5573d1cbc350 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035e222740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5573d1fcd0a0 con 0x5573d1c9b670 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d2063970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d2063970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46270/0 (socket says 172.21.15.31:46270) 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355a4a640 1 -- 172.21.15.31:0/2586771758 learned_addr learned my addr 172.21.15.31:0/2586771758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355249640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2063eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035624b640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d2063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355249640 1 -- 172.21.15.31:0/2586771758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 msgr2=0x5573d2063430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355249640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d2063430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355249640 1 -- 172.21.15.31:0/2586771758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 msgr2=0x5573d2063970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355249640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d2063970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355249640 1 -- 172.21.15.31:0/2586771758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d1d27930 con 0x5573d1c9b670 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d2063970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.910+0000 7f035624b640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d2063430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f0355249640 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2063eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f035006aaa0 tx=0x7f034000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=798e341867c55d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f033bfff640 1 -- 172.21.15.31:0/2586771758 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340015070 con 0x5573d1c9b670 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f0346ffd640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f033c0b65e0 0x7f033c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f033bfff640 1 -- 172.21.15.31:0/2586771758 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400035f0 con 0x5573d1c9b670 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f033bfff640 1 -- 172.21.15.31:0/2586771758 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400116a0 con 0x5573d1c9b670 2024-09-21T06:11:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f035624b640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f033c0b65e0 0x7f033c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.911+0000 7f0346ffd640 1 -- 172.21.15.31:0/2839241644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f034c0e7a20 con 0x7f03501049b0 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035624b640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f033c0b65e0 0x7f033c0b8af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f034802f500 tx=0x7f034802f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 -- 172.21.15.31:0/2586771758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 msgr2=0x5573d2063eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2063eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f035006aaa0 tx=0x7f034000f730 comp rx=0 tx=0).stop 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 -- 172.21.15.31:0/2586771758 shutdown_connections 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2063eb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d2063970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 --2- 172.21.15.31:0/2586771758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d2063430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 -- 172.21.15.31:0/2586771758 >> 172.21.15.31:0/2586771758 conn(0x5573d1cb0740 msgr2=0x5573d2061320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 -- 172.21.15.31:0/2586771758 shutdown_connections 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 -- 172.21.15.31:0/2586771758 wait complete. 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 -- start start 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.912+0000 7f035e222740 1 Processor -- start 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d206efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d206f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d207af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5573d1e2a890 con 0x5573d1c93490 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5573d1f63d40 con 0x5573d1cbc350 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5573d2071c10 con 0x5573d1c9b670 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035e222740 10 client.0 _ll_register_callbacks cb 0x5573d1fc7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f0355a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d206efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f0355249640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d206f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f0355249640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d206f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:39254/0 (socket says 172.21.15.31:39254) 2024-09-21T06:11:21.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f0355a4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d206efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46276/0 (socket says 172.21.15.31:46276) 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f0355249640 1 -- 172.21.15.31:0/2394381113 learned_addr learned my addr 172.21.15.31:0/2394381113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.913+0000 7f035624b640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d207af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355249640 1 -- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 msgr2=0x5573d207af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355249640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d207af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355249640 1 -- 172.21.15.31:0/2394381113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 msgr2=0x5573d206efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355249640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d206efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355249640 1 -- 172.21.15.31:0/2394381113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d207b4a0 con 0x5573d1c9b670 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d206efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f0355249640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d206f520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f034000bed0 tx=0x7f034000bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b98b2ae1d955d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f03397fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f03397fa640 1 -- 172.21.15.31:0/2394381113 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340015030 con 0x5573d1c9b670 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f03397fa640 1 -- 172.21.15.31:0/2394381113 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f03400039b0 con 0x5573d1c9b670 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03400035f0 con 0x5573d1c9b670 2024-09-21T06:11:21.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f03397fa640 1 -- 172.21.15.31:0/2394381113 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0340003f90 con 0x5573d1c9b670 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f035e222740 10 client.25105 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5573d207ddd0 con 0x5573d1c9b670 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.914+0000 7f03397fa640 1 -- 172.21.15.31:0/2394381113 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400035f0 con 0x5573d1c9b670 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f03397fa640 1 -- 172.21.15.31:0/2394381113 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f03400046c0 con 0x5573d1c9b670 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f035e222740 10 client.25105 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f035e222740 10 client.25105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f035e222740 20 client.25105 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f035e222740 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1c7acd0 0x5573d2081180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f035e222740 4 client.25105 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573d1fdbdb0 con 0x5573d1c7acd0 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.915+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1c7acd0 0x5573d2081180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.916+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1c7acd0 0x5573d2081180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc4fa109cea6edc server_cookie=992a4b6b6ad4383a in_seq=0 out_seq=0 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.916+0000 7f03397fa640 10 client.25105 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:21.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.916+0000 7f03397fa640 1 -- 172.21.15.31:0/2394381113 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573d1fdbdb0 con 0x5573d1c7acd0 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.916+0000 7f03397fa640 10 client.25105 handle_command_reply: tid=0 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.916+0000 7f035e222740 1 client.25105 shutdown 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 msgr2=0x5573d206f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d206f520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f034000bed0 tx=0x7f034000bf00 comp rx=0 tx=0).stop 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 shutdown_connections 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1c7acd0 0x5573d2081180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1cbc350 0x5573d207af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d206f520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 --2- 172.21.15.31:0/2394381113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d206efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f03397fa640 0 client.25105 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 >> 172.21.15.31:0/2394381113 conn(0x5573d1cb0740 msgr2=0x5573d20665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 shutdown_connections 2024-09-21T06:11:21.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 1 -- 172.21.15.31:0/2394381113 wait complete. 2024-09-21T06:11:21.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.917+0000 7f035e222740 20 client.25105 trim_cache size 0 max 16384 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 Processor -- start 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 -- start start 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1cb0740 0x5573d206a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d207ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c9b670 0x5573d2064e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03501ad600 con 0x5573d1c93490 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03501afe80 con 0x5573d1c9b670 2024-09-21T06:11:21.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f035e222740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03501adc70 con 0x5573d1cb0740 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f0355249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d207ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f0355249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d207ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46288/0 (socket says 172.21.15.31:46288) 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.935+0000 7f0355249640 1 -- 172.21.15.31:0/2221375807 learned_addr learned my addr 172.21.15.31:0/2221375807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1cb0740 0x5573d206a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f035624b640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c9b670 0x5573d2064e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355249640 1 -- 172.21.15.31:0/2221375807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1cb0740 msgr2=0x5573d206a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355249640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1cb0740 0x5573d206a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355249640 1 -- 172.21.15.31:0/2221375807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c9b670 msgr2=0x5573d2064e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355249640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c9b670 0x5573d2064e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355249640 1 -- 172.21.15.31:0/2221375807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573d2065720 con 0x5573d1c93490 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355a4a640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1cb0740 0x5573d206a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f035624b640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c9b670 0x5573d2064e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f0355249640 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d207ae30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f03400114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea491faf2490802 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f033b7fe640 1 -- 172.21.15.31:0/2221375807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340031030 con 0x5573d1c93490 2024-09-21T06:11:21.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f033b7fe640 1 -- 172.21.15.31:0/2221375807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400039b0 con 0x5573d1c93490 2024-09-21T06:11:21.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.936+0000 7f033b7fe640 1 -- 172.21.15.31:0/2221375807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340015040 con 0x5573d1c93490 2024-09-21T06:11:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 -- 172.21.15.31:0/2221375807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 msgr2=0x5573d207ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d207ae30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f03400114a0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 -- 172.21.15.31:0/2221375807 shutdown_connections 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c9b670 0x5573d2064e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1c93490 0x5573d207ae30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 --2- 172.21.15.31:0/2221375807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1cb0740 0x5573d206a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 -- 172.21.15.31:0/2221375807 >> 172.21.15.31:0/2221375807 conn(0x5573d20613c0 msgr2=0x5573d2070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 -- 172.21.15.31:0/2221375807 shutdown_connections 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.937+0000 7f035e222740 1 -- 172.21.15.31:0/2221375807 wait complete. 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 -- start start 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 Processor -- start 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c93490 0x5573d2072f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2073480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 0x5573d20739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03501ae340 con 0x5573d1cb0740 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03501af7b0 con 0x5573d1c93490 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03501aea10 con 0x5573d1c9b670 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035e222740 10 client.0 _ll_register_callbacks cb 0x7f034c0eb3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.938+0000 7f035624b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 0x5573d20739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 0x5573d20739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46294/0 (socket says 172.21.15.31:46294) 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 -- 172.21.15.31:0/3979880555 learned_addr learned my addr 172.21.15.31:0/3979880555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f0355249640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f0355a4a640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c93490 0x5573d2072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 -- 172.21.15.31:0/3979880555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 msgr2=0x5573d2073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 -- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c93490 msgr2=0x5573d2072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c93490 0x5573d2072f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 -- 172.21.15.31:0/3979880555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573d20eef30 con 0x5573d1cb0740 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f0355249640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f035624b640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 0x5573d20739c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0348031700 tx=0x7f03480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d663cddd0b977f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.939+0000 7f03397fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.942+0000 7f03397fa640 1 -- 172.21.15.31:0/3979880555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348054060 con 0x5573d1cb0740 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.942+0000 7f03397fa640 1 -- 172.21.15.31:0/3979880555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f03480305c0 con 0x5573d1cb0740 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.942+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03400039b0 con 0x5573d1cb0740 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f03397fa640 1 -- 172.21.15.31:0/3979880555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f03480307c0 con 0x5573d1cb0740 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 10 client.15378 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5573d20f0160 con 0x5573d1cb0740 2024-09-21T06:11:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f03397fa640 1 -- 172.21.15.31:0/3979880555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400039b0 con 0x5573d1cb0740 2024-09-21T06:11:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f03397fa640 1 -- 172.21.15.31:0/3979880555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0348043040 con 0x5573d1cb0740 2024-09-21T06:11:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 10 client.15378 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 10 client.15378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 20 client.15378 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1cbc350 0x5573d20f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 4 client.15378 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:11:21.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573d1fd7860 con 0x5573d1cbc350 2024-09-21T06:11:21.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.943+0000 7f0355a4a640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1cbc350 0x5573d20f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:21.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.944+0000 7f03397fa640 10 client.15378 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:21.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.944+0000 7f0355a4a640 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1cbc350 0x5573d20f29d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bb5067644a3f6d5 server_cookie=7f9d06f3bdf4a9ad in_seq=0 out_seq=0 2024-09-21T06:11:21.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f03397fa640 1 -- 172.21.15.31:0/3979880555 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573d1fd7860 con 0x5573d1cbc350 2024-09-21T06:11:21.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f03397fa640 10 client.15378 handle_command_reply: tid=0 2024-09-21T06:11:21.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 client.15378 shutdown 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 msgr2=0x5573d20739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 0x5573d20739c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0348031700 tx=0x7f03480095c0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 shutdown_connections 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573d1cbc350 0x5573d20f29d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573d1cb0740 0x5573d20739c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573d1c9b670 0x5573d2073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 --2- 172.21.15.31:0/3979880555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573d1c93490 0x5573d2072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f03397fa640 0 client.15378 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.946+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 >> 172.21.15.31:0/3979880555 conn(0x5573d20613c0 msgr2=0x5573d2064bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 shutdown_connections 2024-09-21T06:11:21.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f035e222740 1 -- 172.21.15.31:0/3979880555 wait complete. 2024-09-21T06:11:21.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f035e222740 20 client.15378 trim_cache size 0 max 16384 2024-09-21T06:11:21.951 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:11:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "cdc477b6-9b80-4f72-8219-838502bc271a", 2024-09-21T06:11:21.951 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:11:21.951 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f033c0b65e0 msgr2=0x7f033c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f0357cae640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f033c0b65e0 0x7f033c0b8af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f034802f500 tx=0x7f034802f310 comp rx=0 tx=0).stop 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 msgr2=0x7f035019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.947+0000 7f0357cae640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 0x7f035019a460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e9c0 tx=0x7f034c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 shutdown_connections 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f035010de10 0x7f035019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0350105300 0x7f035019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f033c0b65e0 0x7f033c0b8af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 --2- 172.21.15.31:0/2839241644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03501049b0 0x7f035019a460 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:21.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 >> 172.21.15.31:0/2839241644 conn(0x7f0350079730 msgr2=0x7f03500ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:21.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 shutdown_connections 2024-09-21T06:11:21.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.948+0000 7f0357cae640 1 -- 172.21.15.31:0/2839241644 wait complete. 2024-09-21T06:11:21.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.965+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f8a700a93d0 con 0x7f8a801029e0 2024-09-21T06:11:21.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:21.976+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8a80107290 con 0x7f8a680b6560 2024-09-21T06:11:22.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.040+0000 7f8a67fff640 1 -- 172.21.15.31:0/4147435066 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8a80107290 con 0x7f8a680b6560 2024-09-21T06:11:22.043 INFO:teuthology.orchestra.run.smithi031.stdout:Retention added to path / 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a680b6560 msgr2=0x7f8a680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a680b6560 0x7f8a680b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009230 tx=0x7f8a74031400 comp rx=0 tx=0).stop 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 msgr2=0x7f8a80199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 0x7f8a80199340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000d950 tx=0x7f8a7000de10 comp rx=0 tx=0).stop 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 shutdown_connections 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a80108c60 0x7f8a8019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a80108310 0x7f8a801998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8a680b6560 0x7f8a680b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 --2- 172.21.15.31:0/4147435066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a801029e0 0x7f8a80199340 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 >> 172.21.15.31:0/4147435066 conn(0x7f8a800fe3c0 msgr2=0x7f8a80106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:22.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 shutdown_connections 2024-09-21T06:11:22.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.042+0000 7f8a852bd640 1 -- 172.21.15.31:0/4147435066 wait complete. 2024-09-21T06:11:22.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T06:11:22.081 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:21 smithi031 ceph-mon[31237]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 -- 172.21.15.31:0/603507428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88103b00 msgr2=0x7f7e88103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/603507428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88103b00 0x7f7e88103f10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009900 tx=0x7f7e7802a410 comp rx=0 tx=0).stop 2024-09-21T06:11:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 -- 172.21.15.31:0/603507428 shutdown_connections 2024-09-21T06:11:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/603507428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88109d80 0x7f7e881105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/603507428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 0x7f7e88109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/603507428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88103b00 0x7f7e88103f10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 -- 172.21.15.31:0/603507428 >> 172.21.15.31:0/603507428 conn(0x7f7e880ff500 msgr2=0x7f7e88101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 -- 172.21.15.31:0/603507428 shutdown_connections 2024-09-21T06:11:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.172+0000 7f7e8d0de640 1 -- 172.21.15.31:0/603507428 wait complete. 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 Processor -- start 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 -- start start 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 0x7f7e8819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 0x7f7e8819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 0x7f7e8819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88113a40 con 0x7f7e88109d80 2024-09-21T06:11:22.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881138c0 con 0x7f7e88103b00 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.173+0000 7f7e8d0de640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88113bc0 con 0x7f7e88109430 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 0x7f7e8819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e86d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 0x7f7e8819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 0x7f7e8819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46310/0 (socket says 172.21.15.31:46310) 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e86d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 0x7f7e8819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51376/0 (socket says 172.21.15.31:51376) 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 -- 172.21.15.31:0/1676299790 learned_addr learned my addr 172.21.15.31:0/1676299790 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 0x7f7e8819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 -- 172.21.15.31:0/1676299790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 msgr2=0x7f7e8819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 0x7f7e8819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 -- 172.21.15.31:0/1676299790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 msgr2=0x7f7e8819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 0x7f7e8819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 -- 172.21.15.31:0/1676299790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881a03d0 con 0x7f7e88109d80 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e86d9d640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 0x7f7e8819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8659c640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 0x7f7e8819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8759e640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 0x7f7e8819fcd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000eaa0 tx=0x7f7e8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000cdb0 con 0x7f7e88109d80 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e80005020 con 0x7f7e88109d80 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881a06c0 con 0x7f7e88109d80 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.174+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800116d0 con 0x7f7e88109d80 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.175+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a6e50 con 0x7f7e88109d80 2024-09-21T06:11:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.175+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e88103f10 con 0x7f7e88109d80 2024-09-21T06:11:22.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.178+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f7e800051d0 con 0x7f7e88109d80 2024-09-21T06:11:22.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.179+0000 7f7e7ffff640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e740b65f0 0x7f7e740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.179+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7e800e7b50 con 0x7f7e88109d80 2024-09-21T06:11:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.179+0000 7f7e86d9d640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e740b65f0 0x7f7e740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.179+0000 7f7e86d9d640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e740b65f0 0x7f7e740b8b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e78031440 tx=0x7f7e78032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:22.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.182+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7e80016030 con 0x7f7e88109d80 2024-09-21T06:11:22.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:21 smithi110 ceph-mon[34550]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:22.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:21 smithi107 ceph-mon[34380]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:22.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.393+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7e88063800 con 0x7f7e740b65f0 2024-09-21T06:11:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.400+0000 7f7e7ffff640 1 -- 172.21.15.31:0/1676299790 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7e88063800 con 0x7f7e740b65f0 2024-09-21T06:11:22.403 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T06:11:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T06:11:22.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.402+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e740b65f0 msgr2=0x7f7e740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.402+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e740b65f0 0x7f7e740b8b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e78031440 tx=0x7f7e78032ed0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.402+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 msgr2=0x7f7e8819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.402+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 0x7f7e8819fcd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000eaa0 tx=0x7f7e8000ef60 comp rx=0 tx=0).stop 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 shutdown_connections 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e740b65f0 0x7f7e740b8b00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e88109d80 0x7f7e8819fcd0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e88109430 0x7f7e8819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 --2- 172.21.15.31:0/1676299790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e88103b00 0x7f7e8819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 >> 172.21.15.31:0/1676299790 conn(0x7f7e880ff500 msgr2=0x7f7e88108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 shutdown_connections 2024-09-21T06:11:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.403+0000 7f7e8d0de640 1 -- 172.21.15.31:0/1676299790 wait complete. 2024-09-21T06:11:22.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T06:11:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 -- 172.21.15.31:0/599947963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 msgr2=0x7f55e81086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 --2- 172.21.15.31:0/599947963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81086f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 -- 172.21.15.31:0/599947963 shutdown_connections 2024-09-21T06:11:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 --2- 172.21.15.31:0/599947963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55e8108c30 0x7f55e810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 --2- 172.21.15.31:0/599947963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81086f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 --2- 172.21.15.31:0/599947963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 0x7f55e8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 -- 172.21.15.31:0/599947963 >> 172.21.15.31:0/599947963 conn(0x7f55e807a740 msgr2=0x7f55e807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 -- 172.21.15.31:0/599947963 shutdown_connections 2024-09-21T06:11:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.570+0000 7f55ecec5640 1 -- 172.21.15.31:0/599947963 wait complete. 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 Processor -- start 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 -- start start 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 0x7f55e8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55e8108c30 0x7f55e819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55e81128f0 con 0x7f55e81082e0 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8112770 con 0x7f55e81029b0 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55ecec5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55e8112a70 con 0x7f55e8108c30 2024-09-21T06:11:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55e5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55e5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46330/0 (socket says 172.21.15.31:46330) 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55e5d9b640 1 -- 172.21.15.31:0/2324481347 learned_addr learned my addr 172.21.15.31:0/2324481347 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.571+0000 7f55e6d9d640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55e8108c30 0x7f55e819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e659c640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 0x7f55e8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e5d9b640 1 -- 172.21.15.31:0/2324481347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55e8108c30 msgr2=0x7f55e819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e5d9b640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55e8108c30 0x7f55e819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e5d9b640 1 -- 172.21.15.31:0/2324481347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 msgr2=0x7f55e8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e5d9b640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 0x7f55e8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e5d9b640 1 -- 172.21.15.31:0/2324481347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e819f260 con 0x7f55e81082e0 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e659c640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 0x7f55e8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55e5d9b640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81998d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f55dc006010 tx=0x7f55dc004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc03f030 con 0x7f55e81082e0 2024-09-21T06:11:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e819f4f0 con 0x7f55e81082e0 2024-09-21T06:11:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55dc003aa0 con 0x7f55e81082e0 2024-09-21T06:11:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e81a5ce0 con 0x7f55e81082e0 2024-09-21T06:11:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.572+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc049060 con 0x7f55e81082e0 2024-09-21T06:11:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.573+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e8102dc0 con 0x7f55e81082e0 2024-09-21T06:11:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.575+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f55dc042030 con 0x7f55e81082e0 2024-09-21T06:11:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.575+0000 7f55d77fe640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55d00b65f0 0x7f55d00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.575+0000 7f55e659c640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55d00b65f0 0x7f55d00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.576+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f55dc107e60 con 0x7f55e81082e0 2024-09-21T06:11:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.576+0000 7f55e659c640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55d00b65f0 0x7f55d00b8b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009b40 tx=0x7f55d8005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:22.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.579+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55dc0ce4b0 con 0x7f55e81082e0 2024-09-21T06:11:22.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:11:22.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.795+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f55e810aa80 con 0x7f55d00b65f0 2024-09-21T06:11:22.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.803+0000 7f55d77fe640 1 -- 172.21.15.31:0/2324481347 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f55e810aa80 con 0x7f55d00b65f0 2024-09-21T06:11:22.806 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-09-21T06:11:22.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.805+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55d00b65f0 msgr2=0x7f55d00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.805+0000 7f55ecec5640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55d00b65f0 0x7f55d00b8b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009b40 tx=0x7f55d8005e50 comp rx=0 tx=0).stop 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.805+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 msgr2=0x7f55e81998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.805+0000 7f55ecec5640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81998d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f55dc006010 tx=0x7f55dc004060 comp rx=0 tx=0).stop 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 shutdown_connections 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55e8108c30 0x7f55e819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55d00b65f0 0x7f55d00b8b00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81082e0 0x7f55e81998d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 --2- 172.21.15.31:0/2324481347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55e81029b0 0x7f55e8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 >> 172.21.15.31:0/2324481347 conn(0x7f55e807a740 msgr2=0x7f55e81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 shutdown_connections 2024-09-21T06:11:22.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:22.806+0000 7f55ecec5640 1 -- 172.21.15.31:0/2324481347 wait complete. 2024-09-21T06:11:22.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T06:11:22.850 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T06:11:22.860 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T06:11:22.899 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:11:22.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T06:11:22.902 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:11:22.938 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:11:22.938 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:11:22.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-21T06:11:22.939 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:22.974 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:22.986 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-21T06:11:22.986 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:3300,172.21.15.107:3300,172.21.15.110:3300:/volumes/qa/sv_1/11ff3a3e-12f9-435b-bdf0-22b9c15d836e 340G 0 340G 0% /tmp/cephfs 2024-09-21T06:11:22.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-21T06:11:23.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-21T06:11:23.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:11:23.186 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:22 smithi031 ceph-mon[31237]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-21T06:11:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:11:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:11:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T06:11:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:11:23.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:22 smithi110 ceph-mon[34550]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: pgmap v313: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 938 B/s wr, 3 op/s 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T06:11:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:11:23.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:22 smithi107 ceph-mon[34380]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 -- 172.21.15.31:0/1253324013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc10de10 msgr2=0x7f8cbc110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1253324013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc10de10 0x7f8cbc110300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009ca0 tx=0x7f8cb402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 -- 172.21.15.31:0/1253324013 shutdown_connections 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1253324013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc10de10 0x7f8cbc110300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1253324013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cbc105300 0x7f8cbc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1253324013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cbc1049b0 0x7f8cbc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 -- 172.21.15.31:0/1253324013 >> 172.21.15.31:0/1253324013 conn(0x7f8cbc079730 msgr2=0x7f8cbc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 -- 172.21.15.31:0/1253324013 shutdown_connections 2024-09-21T06:11:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.462+0000 7f8cc1301640 1 -- 172.21.15.31:0/1253324013 wait complete. 2024-09-21T06:11:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.463+0000 7f8cc1301640 1 Processor -- start 2024-09-21T06:11:23.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.463+0000 7f8cc1301640 1 -- start start 2024-09-21T06:11:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cc1301640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cbc1049b0 0x7f8cbc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cc1301640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 0x7f8cbc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cc1301640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cbc10de10 0x7f8cbc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cc1301640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc113a90 con 0x7f8cbc105300 2024-09-21T06:11:23.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cc1301640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc113910 con 0x7f8cbc1049b0 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cc1301640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc113c10 con 0x7f8cbc10de10 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 0x7f8cbc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 0x7f8cbc19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46346/0 (socket says 172.21.15.31:46346) 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1840799613 learned_addr learned my addr 172.21.15.31:0/1840799613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cbc10de10 0x7f8cbc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cbc1049b0 0x7f8cbc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1840799613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cbc10de10 msgr2=0x7f8cbc19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cbc10de10 0x7f8cbc19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1840799613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cbc1049b0 msgr2=0x7f8cbc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cbc1049b0 0x7f8cbc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.464+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1840799613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc1a03d0 con 0x7f8cbc105300 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.465+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 0x7f8cbc19aa00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d950 tx=0x7f8cb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.465+0000 7f8ca3fff640 1 -- 172.21.15.31:0/1840799613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0017070 con 0x7f8cbc105300 2024-09-21T06:11:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.465+0000 7f8ca3fff640 1 -- 172.21.15.31:0/1840799613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb0009c30 con 0x7f8cbc105300 2024-09-21T06:11:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.465+0000 7f8ca3fff640 1 -- 172.21.15.31:0/1840799613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00047a0 con 0x7f8cbc105300 2024-09-21T06:11:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.465+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc1a06c0 con 0x7f8cbc105300 2024-09-21T06:11:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.465+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a6e20 con 0x7f8cbc105300 2024-09-21T06:11:23.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 Processor -- start 2024-09-21T06:11:23.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 -- start start 2024-09-21T06:11:23.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8ca3fff640 1 -- 172.21.15.31:0/1840799613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f8cb0004020 con 0x7f8cbc105300 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f2e070 0x55e4e42d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42d60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e4e424be50 con 0x55e4e3f0d390 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4e423b280 con 0x55e4e3f051b0 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cc7875740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e4e423f090 con 0x55e4e3f2e070 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.468+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42d60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42d60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46348/0 (socket says 172.21.15.31:46348) 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 -- 172.21.15.31:0/2241885347 learned_addr learned my addr 172.21.15.31:0/2241885347 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f2e070 0x55e4e42d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 -- 172.21.15.31:0/2241885347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f2e070 msgr2=0x55e4e42d5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f2e070 0x55e4e42d5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 -- 172.21.15.31:0/2241885347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 msgr2=0x55e4e42d5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42d5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 -- 172.21.15.31:0/2241885347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e3f99650 con 0x55e4e3f0d390 2024-09-21T06:11:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8ca3fff640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8ca40baa80 0x7f8ca40bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42d60d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00e4980 tx=0x7f8cb00ef850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7011f96897d44ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.469+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8ca40baa80 0x7f8ca40bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.470+0000 7f8ca3fff640 1 -- 172.21.15.31:0/1840799613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8cb00e70d0 con 0x7f8cbc105300 2024-09-21T06:11:23.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.470+0000 7f8ca17fa640 1 -- 172.21.15.31:0/2241885347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00a9710 con 0x55e4e3f0d390 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.470+0000 7f8ca17fa640 1 -- 172.21.15.31:0/2241885347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb00acbd0 con 0x55e4e3f0d390 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.470+0000 7f8ca17fa640 1 -- 172.21.15.31:0/2241885347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00799a0 con 0x55e4e3f0d390 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 -- 172.21.15.31:0/2241885347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 msgr2=0x55e4e42d60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8ca40baa80 0x7f8ca40bcf90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8cb402f500 tx=0x7f8cb402f310 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42d60d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00e4980 tx=0x7f8cb00ef850 comp rx=0 tx=0).stop 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 -- 172.21.15.31:0/2241885347 shutdown_connections 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42d60d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 --2- 172.21.15.31:0/2241885347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f2e070 0x55e4e42d5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 -- 172.21.15.31:0/2241885347 >> 172.21.15.31:0/2241885347 conn(0x55e4e3f22460 msgr2=0x55e4e42d3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 -- 172.21.15.31:0/2241885347 shutdown_connections 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.471+0000 7f8cc7875740 1 -- 172.21.15.31:0/2241885347 wait complete. 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 -- start start 2024-09-21T06:11:23.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 Processor -- start 2024-09-21T06:11:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f051b0 0x55e4e42e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f2e070 0x55e4e42ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e4e409c700 con 0x55e4e3f0d390 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e4e41d6110 con 0x55e4e3f2e070 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e4e42e3e30 con 0x55e4e3f051b0 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cc7875740 10 client.0 _ll_register_callbacks cb 0x55e4e41ef990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42e1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46360/0 (socket says 172.21.15.31:46360) 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1826387390 learned_addr learned my addr 172.21.15.31:0/1826387390 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.472+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f2e070 0x55e4e42ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f051b0 0x55e4e42e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1826387390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f051b0 msgr2=0x55e4e42e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f051b0 0x55e4e42e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f2e070 msgr2=0x55e4e42ed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f2e070 0x55e4e42ed150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cba7fc640 1 -- 172.21.15.31:0/1826387390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e4203c40 con 0x55e4e3f0d390 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42e1290 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b640 tx=0x7f8cb00e7490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17be886d5d16155 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8c9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8c9eff5640 1 -- 172.21.15.31:0/1826387390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0079b80 con 0x55e4e3f0d390 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8c9eff5640 1 -- 172.21.15.31:0/1826387390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8cb00b6850 con 0x55e4e3f0d390 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb00acbd0 con 0x55e4e3f0d390 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8c9eff5640 1 -- 172.21.15.31:0/1826387390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8cb0079de0 con 0x55e4e3f0d390 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cc7875740 10 client.15402 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.473+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e4e42efbd0 con 0x55e4e3f0d390 2024-09-21T06:11:23.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8c9eff5640 1 -- 172.21.15.31:0/1826387390 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb00acbd0 con 0x55e4e3f0d390 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8c9eff5640 1 -- 172.21.15.31:0/1826387390 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8cb0098120 con 0x55e4e3f0d390 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cc7875740 10 client.15402 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cc7875740 10 client.15402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cc7875740 20 client.15402 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cc7875740 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3eec9f0 0x55e4e42f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cc7875740 4 client.15402 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e424dda0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.474+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3eec9f0 0x55e4e42f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.475+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3eec9f0 0x55e4e42f2f30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c8761c4662b2566 server_cookie=376ae8fdba220f91 in_seq=0 out_seq=0 2024-09-21T06:11:23.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.475+0000 7f8c9eff5640 10 client.15402 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8c9eff5640 1 -- 172.21.15.31:0/1826387390 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4e424dda0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8c9eff5640 10 client.15402 handle_command_reply: tid=0 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 client.15402 shutdown 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 msgr2=0x55e4e42e1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42e1290 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000b640 tx=0x7f8cb00e7490 comp rx=0 tx=0).stop 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 shutdown_connections 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3eec9f0 0x55e4e42f2f30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f2e070 0x55e4e42ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8c9eff5640 0 client.15402 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f0d390 0x55e4e42e1290 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 --2- 172.21.15.31:0/1826387390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f051b0 0x55e4e42e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 >> 172.21.15.31:0/1826387390 conn(0x55e4e3f22460 msgr2=0x55e4e42d87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 shutdown_connections 2024-09-21T06:11:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 1 -- 172.21.15.31:0/1826387390 wait complete. 2024-09-21T06:11:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.476+0000 7f8cc7875740 20 client.15402 trim_cache size 0 max 16384 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 Processor -- start 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 -- start start 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3eec9f0 0x55e4e42ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 0x55e4e42ed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f0d390 0x55e4e42ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ab410 con 0x55e4e3f051b0 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1adc90 con 0x55e4e3f0d390 2024-09-21T06:11:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cc7875740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1aba80 con 0x55e4e3eec9f0 2024-09-21T06:11:23.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.511+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 0x55e4e42ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 0x55e4e42ed600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46372/0 (socket says 172.21.15.31:46372) 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 -- 172.21.15.31:0/4262021800 learned_addr learned my addr 172.21.15.31:0/4262021800 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f0d390 0x55e4e42ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3eec9f0 0x55e4e42ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 -- 172.21.15.31:0/4262021800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3eec9f0 msgr2=0x55e4e42ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3eec9f0 0x55e4e42ed0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 -- 172.21.15.31:0/4262021800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f0d390 msgr2=0x55e4e42ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f0d390 0x55e4e42ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 -- 172.21.15.31:0/4262021800 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e42d7940 con 0x55e4e3f051b0 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 0x55e4e42ed600 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00e4980 tx=0x7f8cb00e4ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76ef5e76dedb41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8ca0ff9640 1 -- 172.21.15.31:0/4262021800 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00ac830 con 0x55e4e3f051b0 2024-09-21T06:11:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8ca0ff9640 1 -- 172.21.15.31:0/4262021800 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb00eaa80 con 0x55e4e3f051b0 2024-09-21T06:11:23.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.512+0000 7f8ca0ff9640 1 -- 172.21.15.31:0/4262021800 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00aca10 con 0x55e4e3f051b0 2024-09-21T06:11:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 -- 172.21.15.31:0/4262021800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 msgr2=0x55e4e42ed600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 0x55e4e42ed600 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00e4980 tx=0x7f8cb00e4ae0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 -- 172.21.15.31:0/4262021800 shutdown_connections 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f0d390 0x55e4e42ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3f051b0 0x55e4e42ed600 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 --2- 172.21.15.31:0/4262021800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3eec9f0 0x55e4e42ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 -- 172.21.15.31:0/4262021800 >> 172.21.15.31:0/4262021800 conn(0x55e4e3f22460 msgr2=0x55e4e42df5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 -- 172.21.15.31:0/4262021800 shutdown_connections 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.513+0000 7f8cc7875740 1 -- 172.21.15.31:0/4262021800 wait complete. 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.514+0000 7f8cc7875740 1 -- start start 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.514+0000 7f8cc7875740 1 Processor -- start 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 0x55e4e42f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f0d390 0x55e4e42f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ac150 con 0x55e4e3eec9f0 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ad5c0 con 0x55e4e3f051b0 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ac820 con 0x55e4e3f0d390 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cc7875740 10 client.0 _ll_register_callbacks cb 0x7f8cac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 0x55e4e42f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f0d390 0x55e4e42f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 0x55e4e42f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46382/0 (socket says 172.21.15.31:46382) 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 -- 172.21.15.31:0/3547997336 learned_addr learned my addr 172.21.15.31:0/3547997336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 -- 172.21.15.31:0/3547997336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f0d390 msgr2=0x55e4e42f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f0d390 0x55e4e42f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 -- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 msgr2=0x55e4e42f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 -- 172.21.15.31:0/3547997336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e435c6b0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbb7fe640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f0d390 0x55e4e42f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8cbaffd640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 0x55e4e42f3f50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0105a0 tx=0x7f8cac010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549daff826a22314 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.515+0000 7f8c9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8c9eff5640 1 -- 172.21.15.31:0/3547997336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac004220 con 0x55e4e3eec9f0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb00eaa80 con 0x55e4e3eec9f0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8c9eff5640 1 -- 172.21.15.31:0/3547997336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8cac0043c0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8c9eff5640 1 -- 172.21.15.31:0/3547997336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8cac00f7d0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 10 client.15408 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e4e435d9d0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8c9eff5640 1 -- 172.21.15.31:0/3547997336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb00eaa80 con 0x55e4e3eec9f0 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8c9eff5640 1 -- 172.21.15.31:0/3547997336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8cac003cc0 con 0x55e4e3eec9f0 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 10 client.15408 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 20 client.15408 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.516+0000 7f8cc7875740 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3f2e070 0x55e4e4360120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8cc7875740 4 client.15408 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4e4249850 con 0x55e4e3f2e070 2024-09-21T06:11:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3f2e070 0x55e4e4360120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8cba7fc640 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3f2e070 0x55e4e4360120 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edd0dd209fb24d9 server_cookie=a65388905309e16c in_seq=0 out_seq=0 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8c9eff5640 10 client.15408 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8c9eff5640 1 -- 172.21.15.31:0/3547997336 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55e4e4249850 con 0x55e4e3f2e070 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8c9eff5640 10 client.15408 handle_command_reply: tid=0 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.517+0000 7f8cc7875740 1 client.15408 shutdown 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 msgr2=0x55e4e42f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 0x55e4e42f3f50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0105a0 tx=0x7f8cac010a60 comp rx=0 tx=0).stop 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 shutdown_connections 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e4e3f2e070 0x55e4e4360120 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e4e3f0d390 0x55e4e42f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4e3f051b0 0x55e4e42f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 --2- 172.21.15.31:0/3547997336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e4e3eec9f0 0x55e4e42f3f50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 >> 172.21.15.31:0/3547997336 conn(0x55e4e3f22460 msgr2=0x55e4e42e2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8c9eff5640 0 client.15408 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 shutdown_connections 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 1 -- 172.21.15.31:0/3547997336 wait complete. 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.518+0000 7f8cc7875740 20 client.15408 trim_cache size 0 max 16384 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:11:23.526 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:23.527 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8ca40baa80 msgr2=0x7f8ca40bcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8ca40baa80 0x7f8ca40bcf90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8cb402f500 tx=0x7f8cb402f310 comp rx=0 tx=0).stop 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 msgr2=0x7f8cbc19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 0x7f8cbc19aa00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d950 tx=0x7f8cb000de10 comp rx=0 tx=0).stop 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 shutdown_connections 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cbc10de10 0x7f8cbc19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8ca40baa80 0x7f8ca40bcf90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cbc105300 0x7f8cbc19aa00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 --2- 172.21.15.31:0/1840799613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cbc1049b0 0x7f8cbc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:23.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 >> 172.21.15.31:0/1840799613 conn(0x7f8cbc079730 msgr2=0x7f8cbc0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:23.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 shutdown_connections 2024-09-21T06:11:23.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:23.519+0000 7f8cc1301640 1 -- 172.21.15.31:0/1840799613 wait complete. 2024-09-21T06:11:24.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:24 smithi031 ceph-mon[31237]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:24.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:24 smithi110 ceph-mon[34550]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:24.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:24 smithi107 ceph-mon[34380]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:11:25.017 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:11:25.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:11:25.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:25 smithi031 ceph-mon[31237]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:11:25.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-21T06:11:25.296 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-21T06:11:25.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-21T06:11:25.336 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:25.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T06:11:25.453 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/: 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 .. 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 data 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data: 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 .. 2024-09-21T06:11:25.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-21T06:11:25.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:25 smithi110 ceph-mon[34550]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:11:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:25 smithi107 ceph-mon[34380]: pgmap v314: 97 pgs: 97 active+clean; 649 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T06:11:25.541 INFO:teuthology.orchestra.run.smithi031.stdout:[Service] 2024-09-21T06:11:25.542 INFO:teuthology.orchestra.run.smithi031.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-21T06:11:25.542 INFO:teuthology.orchestra.run.smithi031.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-21T06:11:25.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-21T06:11:25.638 INFO:teuthology.orchestra.run.smithi031.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-21T06:11:25.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 -- 172.21.15.31:0/2891703267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 msgr2=0x7f162c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 --2- 172.21.15.31:0/2891703267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c1029e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f161c009ca0 tx=0x7f161c02a950 comp rx=0 tx=0).stop 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 -- 172.21.15.31:0/2891703267 shutdown_connections 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 --2- 172.21.15.31:0/2891703267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c110180 0x7f162c1125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 --2- 172.21.15.31:0/2891703267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c102f20 0x7f162c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 --2- 172.21.15.31:0/2891703267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c1029e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 -- 172.21.15.31:0/2891703267 >> 172.21.15.31:0/2891703267 conn(0x7f162c079730 msgr2=0x7f162c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 -- 172.21.15.31:0/2891703267 shutdown_connections 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.879+0000 7f1633a50640 1 -- 172.21.15.31:0/2891703267 wait complete. 2024-09-21T06:11:25.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.880+0000 7f1633a50640 1 Processor -- start 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.880+0000 7f1633a50640 1 -- start start 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1633a50640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1633a50640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c102f20 0x7f162c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1633a50640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c110180 0x7f162c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1633a50640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f162c115c40 con 0x7f162c1025d0 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1633a50640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f162c115ac0 con 0x7f162c110180 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1633a50640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f162c115dc0 con 0x7f162c102f20 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46404/0 (socket says 172.21.15.31:46404) 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 -- 172.21.15.31:0/901696960 learned_addr learned my addr 172.21.15.31:0/901696960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1631fed640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c110180 0x7f162c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1630feb640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c102f20 0x7f162c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 -- 172.21.15.31:0/901696960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c102f20 msgr2=0x7f162c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c102f20 0x7f162c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 -- 172.21.15.31:0/901696960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c110180 msgr2=0x7f162c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c110180 0x7f162c1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f16317ec640 1 -- 172.21.15.31:0/901696960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1a24c0 con 0x7f162c1025d0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.881+0000 7f1630feb640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c102f20 0x7f162c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.882+0000 7f16317ec640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c19c610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f161c005cc0 tx=0x7f161c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.882+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c034040 con 0x7f162c1025d0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.882+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1a2750 con 0x7f162c1025d0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.882+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161c003e40 con 0x7f162c1025d0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.882+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c045480 con 0x7f162c1025d0 2024-09-21T06:11:25.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.882+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a2c30 con 0x7f162c1025d0 2024-09-21T06:11:25.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.883+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c1029e0 con 0x7f162c1025d0 2024-09-21T06:11:25.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.885+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f161c037020 con 0x7f162c1025d0 2024-09-21T06:11:25.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.886+0000 7f162a7fc640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f16140b6620 0x7f16140b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:25.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.886+0000 7f1630feb640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f16140b6620 0x7f16140b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:25.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.886+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f161c108890 con 0x7f162c1025d0 2024-09-21T06:11:25.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.887+0000 7f1630feb640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f16140b6620 0x7f16140b8b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f162c07a040 tx=0x7f162000a5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:25.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:25.889+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f161c083710 con 0x7f162c1025d0 2024-09-21T06:11:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.123+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f162c02e640 con 0x7f162c1025d0 2024-09-21T06:11:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.124+0000 7f162a7fc640 1 -- 172.21.15.31:0/901696960 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f161c0ceee0 con 0x7f162c1025d0 2024-09-21T06:11:26.126 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:26.126 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:11:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f16140b6620 msgr2=0x7f16140b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f16140b6620 0x7f16140b8b30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f162c07a040 tx=0x7f162000a5c0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 msgr2=0x7f162c19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c19c610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f161c005cc0 tx=0x7f161c02f650 comp rx=0 tx=0).stop 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 shutdown_connections 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c110180 0x7f162c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f162c102f20 0x7f162c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f16140b6620 0x7f16140b8b30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 --2- 172.21.15.31:0/901696960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1025d0 0x7f162c19c610 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 >> 172.21.15.31:0/901696960 conn(0x7f162c079730 msgr2=0x7f162c111580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 shutdown_connections 2024-09-21T06:11:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:26.127+0000 7f1633a50640 1 -- 172.21.15.31:0/901696960 wait complete. 2024-09-21T06:11:26.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:11:27.240 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:27.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:27 smithi031 ceph-mon[31237]: pgmap v315: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 176 KiB/s wr, 50 op/s 2024-09-21T06:11:27.362 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:27 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/901696960' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:27 smithi110 ceph-mon[34550]: pgmap v315: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 176 KiB/s wr, 50 op/s 2024-09-21T06:11:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:27 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/901696960' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:27.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:27 smithi107 ceph-mon[34380]: pgmap v315: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 176 KiB/s wr, 50 op/s 2024-09-21T06:11:27.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:27 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/901696960' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 -- 172.21.15.31:0/2891179703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 msgr2=0x7fd3281098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/2891179703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd3281098b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd310009900 tx=0x7fd31002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 -- 172.21.15.31:0/2891179703 shutdown_connections 2024-09-21T06:11:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/2891179703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd328109df0 0x7fd328110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/2891179703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd3281098b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/2891179703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 0x7fd328103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 -- 172.21.15.31:0/2891179703 >> 172.21.15.31:0/2891179703 conn(0x7fd32807b850 msgr2=0x7fd32807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:27.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 -- 172.21.15.31:0/2891179703 shutdown_connections 2024-09-21T06:11:27.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.700+0000 7fd32d4f3640 1 -- 172.21.15.31:0/2891179703 wait complete. 2024-09-21T06:11:27.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 Processor -- start 2024-09-21T06:11:27.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 -- start start 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 0x7fd32819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd32819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd328109df0 0x7fd32819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd328113ab0 con 0x7fd3281094a0 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd328113930 con 0x7fd328109df0 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd32d4f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd328113c30 con 0x7fd328103b60 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd31ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd32819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd31ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd32819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46414/0 (socket says 172.21.15.31:46414) 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd31ffff640 1 -- 172.21.15.31:0/4218115077 learned_addr learned my addr 172.21.15.31:0/4218115077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd3277fe640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd328109df0 0x7fd32819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.701+0000 7fd326ffd640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 0x7fd32819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd31ffff640 1 -- 172.21.15.31:0/4218115077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 msgr2=0x7fd32819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd31ffff640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 0x7fd32819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd31ffff640 1 -- 172.21.15.31:0/4218115077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd328109df0 msgr2=0x7fd32819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd31ffff640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd328109df0 0x7fd32819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd31ffff640 1 -- 172.21.15.31:0/4218115077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3281a04d0 con 0x7fd3281094a0 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd326ffd640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 0x7fd32819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd31ffff640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd32819ab00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd31002a930 tx=0x7fd310004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd324ff9640 1 -- 172.21.15.31:0/4218115077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31003f030 con 0x7fd3281094a0 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3281a0760 con 0x7fd3281094a0 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd324ff9640 1 -- 172.21.15.31:0/4218115077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd310003aa0 con 0x7fd3281094a0 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3281a6f20 con 0x7fd3281094a0 2024-09-21T06:11:27.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.702+0000 7fd324ff9640 1 -- 172.21.15.31:0/4218115077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310035070 con 0x7fd3281094a0 2024-09-21T06:11:27.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.705+0000 7fd324ff9640 1 -- 172.21.15.31:0/4218115077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fd310042030 con 0x7fd3281094a0 2024-09-21T06:11:27.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.705+0000 7fd333a67740 1 Processor -- start 2024-09-21T06:11:27.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.705+0000 7fd333a67740 1 -- start start 2024-09-21T06:11:27.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd333a67740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c1f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd333a67740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c1f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd333a67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be31390 0x55727c1fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd333a67740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55727c16fea0 con 0x55727be291b0 2024-09-21T06:11:27.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd333a67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55727c15f2d0 con 0x55727be31390 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd333a67740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55727c1630e0 con 0x55727be52070 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd326ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c1f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd326ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c1f9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46422/0 (socket says 172.21.15.31:46422) 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd326ffd640 1 -- 172.21.15.31:0/1620028472 learned_addr learned my addr 172.21.15.31:0/1620028472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd3277fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c1f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd324ff9640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3140b65f0 0x7fd3140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.706+0000 7fd31ffff640 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be31390 0x55727c1fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd326ffd640 1 -- 172.21.15.31:0/1620028472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 msgr2=0x55727c1f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c1f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd326ffd640 1 -- 172.21.15.31:0/1620028472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be31390 msgr2=0x55727c1fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be31390 0x55727c1fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd326ffd640 1 -- 172.21.15.31:0/1620028472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727bebd650 con 0x55727be291b0 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd3277fe640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3140b65f0 0x7fd3140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd31ffff640 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be31390 0x55727c1fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd324ff9640 1 -- 172.21.15.31:0/4218115077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd310107790 con 0x7fd3281094a0 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c1f9be0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd31800bcc0 tx=0x7fd31800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c184ab64d57cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd31d7fa640 1 -- 172.21.15.31:0/1620028472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318005020 con 0x55727be291b0 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd31d7fa640 1 -- 172.21.15.31:0/1620028472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3180047a0 con 0x55727be291b0 2024-09-21T06:11:27.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.707+0000 7fd31d7fa640 1 -- 172.21.15.31:0/1620028472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318011640 con 0x55727be291b0 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 -- 172.21.15.31:0/1620028472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 msgr2=0x55727c1f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd3277fe640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3140b65f0 0x7fd3140b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd320005fd0 tx=0x7fd320004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c1f9be0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd31800bcc0 tx=0x7fd31800f730 comp rx=0 tx=0).stop 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 -- 172.21.15.31:0/1620028472 shutdown_connections 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be31390 0x55727c1fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c1f9be0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 --2- 172.21.15.31:0/1620028472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c1f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.708+0000 7fd333a67740 1 -- 172.21.15.31:0/1620028472 >> 172.21.15.31:0/1620028472 conn(0x55727be46460 msgr2=0x55727c1f7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:27.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.709+0000 7fd333a67740 1 -- 172.21.15.31:0/1620028472 shutdown_connections 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.709+0000 7fd333a67740 1 -- 172.21.15.31:0/1620028472 wait complete. 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.709+0000 7fd333a67740 1 -- start start 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.709+0000 7fd333a67740 1 Processor -- start 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.709+0000 7fd333a67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be291b0 0x55727c204cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd333a67740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c205230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd333a67740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c205770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd333a67740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55727bfc0700 con 0x55727be31390 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd333a67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55727c0fa160 con 0x55727be291b0 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd333a67740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55727c207e80 con 0x55727be52070 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd333a67740 10 client.0 _ll_register_callbacks cb 0x55727c1139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c205230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c205230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46424/0 (socket says 172.21.15.31:46424) 2024-09-21T06:11:27.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 -- 172.21.15.31:0/2708943484 learned_addr learned my addr 172.21.15.31:0/2708943484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd3277fe640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c205770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd326ffd640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be291b0 0x55727c204cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 -- 172.21.15.31:0/2708943484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 msgr2=0x55727c205770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c205770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 -- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be291b0 msgr2=0x55727c204cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be291b0 0x55727c204cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 -- 172.21.15.31:0/2708943484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727c127c90 con 0x55727be31390 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd326ffd640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be291b0 0x55727c204cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd3277fe640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c205770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd31ffff640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c205230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd3100c9cb0 tx=0x7fd31002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c4c35f64dac186 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.710+0000 7fd30e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd30e7fc640 1 -- 172.21.15.31:0/2708943484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310049070 con 0x55727be31390 2024-09-21T06:11:27.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd30e7fc640 1 -- 172.21.15.31:0/2708943484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd3100cd940 con 0x55727be31390 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3180047a0 con 0x55727be31390 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd30e7fc640 1 -- 172.21.15.31:0/2708943484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd3100c9d30 con 0x55727be31390 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 10 client.15426 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55727c213b00 con 0x55727be31390 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd30e7fc640 1 -- 172.21.15.31:0/2708943484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3180047a0 con 0x55727be31390 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd30e7fc640 1 -- 172.21.15.31:0/2708943484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fd31004c060 con 0x55727be31390 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 10 client.15426 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 10 client.15426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 20 client.15426 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.711+0000 7fd333a67740 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be109f0 0x55727c217080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.712+0000 7fd333a67740 4 client.15426 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.712+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55727c171df0 con 0x55727be109f0 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.712+0000 7fd326ffd640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be109f0 0x55727c217080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.712+0000 7fd30e7fc640 10 client.15426 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:27.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.712+0000 7fd326ffd640 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be109f0 0x55727c217080 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128399c031b80624 server_cookie=3c0b87e9c1e2c888 in_seq=0 out_seq=0 2024-09-21T06:11:27.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd30e7fc640 1 -- 172.21.15.31:0/2708943484 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55727c171df0 con 0x55727be109f0 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd30e7fc640 10 client.15426 handle_command_reply: tid=0 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 client.15426 shutdown 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 msgr2=0x55727c205230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c205230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd3100c9cb0 tx=0x7fd31002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 shutdown_connections 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be109f0 0x55727c217080 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be52070 0x55727c205770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c205230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd30e7fc640 0 client.15426 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 --2- 172.21.15.31:0/2708943484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be291b0 0x55727c204cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 >> 172.21.15.31:0/2708943484 conn(0x55727be46460 msgr2=0x55727c1fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 shutdown_connections 2024-09-21T06:11:27.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.713+0000 7fd333a67740 1 -- 172.21.15.31:0/2708943484 wait complete. 2024-09-21T06:11:27.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.714+0000 7fd333a67740 20 client.15426 trim_cache size 0 max 16384 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 Processor -- start 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 -- start start 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c1fb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be291b0 0x55727c206bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c1fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd3281ad780 con 0x55727be31390 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3281b0000 con 0x55727be109f0 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.727+0000 7fd333a67740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3281addf0 con 0x55727be291b0 2024-09-21T06:11:27.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c1fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c1fceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46434/0 (socket says 172.21.15.31:46434) 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd326ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c1fb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 -- 172.21.15.31:0/3274365381 learned_addr learned my addr 172.21.15.31:0/3274365381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd31ffff640 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be291b0 0x55727c206bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 -- 172.21.15.31:0/3274365381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be291b0 msgr2=0x55727c206bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be291b0 0x55727c206bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 -- 172.21.15.31:0/3274365381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 msgr2=0x55727c1fb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c1fb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 -- 172.21.15.31:0/3274365381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727c1fb990 con 0x55727be31390 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd31ffff640 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be291b0 0x55727c206bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd3277fe640 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c1fceb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd320012600 tx=0x7fd320016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6665dc3b39258b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd31cff9640 1 -- 172.21.15.31:0/3274365381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320021810 con 0x55727be31390 2024-09-21T06:11:27.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd31cff9640 1 -- 172.21.15.31:0/3274365381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd320010400 con 0x55727be31390 2024-09-21T06:11:27.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.728+0000 7fd31cff9640 1 -- 172.21.15.31:0/3274365381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320021e60 con 0x55727be31390 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 -- 172.21.15.31:0/3274365381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 msgr2=0x55727c1fceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c1fceb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd320012600 tx=0x7fd320016000 comp rx=0 tx=0).stop 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 -- 172.21.15.31:0/3274365381 shutdown_connections 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be31390 0x55727c1fceb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be291b0 0x55727c206bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 --2- 172.21.15.31:0/3274365381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c1fb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 -- 172.21.15.31:0/3274365381 >> 172.21.15.31:0/3274365381 conn(0x55727be46460 msgr2=0x55727c1fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 -- 172.21.15.31:0/3274365381 shutdown_connections 2024-09-21T06:11:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.729+0000 7fd333a67740 1 -- 172.21.15.31:0/3274365381 wait complete. 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 -- start start 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 Processor -- start 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c20d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c20d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be31390 0x55727c27aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd3281ae4c0 con 0x55727be291b0 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3281af930 con 0x55727be109f0 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3281aeb90 con 0x55727be31390 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd333a67740 10 client.0 _ll_register_callbacks cb 0x7fd310005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd31ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c20d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd31ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c20d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46444/0 (socket says 172.21.15.31:46444) 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd31ffff640 1 -- 172.21.15.31:0/1347554291 learned_addr learned my addr 172.21.15.31:0/1347554291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.730+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c20d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd3277fe640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be31390 0x55727c27aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd31ffff640 1 -- 172.21.15.31:0/1347554291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be31390 msgr2=0x55727c27aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd31ffff640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be31390 0x55727c27aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd31ffff640 1 -- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 msgr2=0x55727c20d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd31ffff640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c20d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd31ffff640 1 -- 172.21.15.31:0/1347554291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727c20db80 con 0x55727be291b0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd3277fe640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be31390 0x55727c27aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c20d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd31ffff640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c20d640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd3101050e0 tx=0x7fd310003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d5813686d7fd57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd30e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd30e7fc640 1 -- 172.21.15.31:0/1347554291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310049070 con 0x55727be291b0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd30e7fc640 1 -- 172.21.15.31:0/1347554291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd3100cd6e0 con 0x55727be291b0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd320010400 con 0x55727be291b0 2024-09-21T06:11:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd30e7fc640 1 -- 172.21.15.31:0/1347554291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd3100c9580 con 0x55727be291b0 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd333a67740 10 client.15432 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55727c2860f0 con 0x55727be291b0 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.731+0000 7fd30e7fc640 1 -- 172.21.15.31:0/1347554291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd320010400 con 0x55727be291b0 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd30e7fc640 1 -- 172.21.15.31:0/1347554291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fd3100cc940 con 0x55727be291b0 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 10 client.15432 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 20 client.15432 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be52070 0x55727c288750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 4 client.15432 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55727c2008a0 con 0x55727be52070 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be52070 0x55727c288750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd30e7fc640 10 client.15432 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd326ffd640 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be52070 0x55727c288750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744ed2d2f4aca98a server_cookie=4bfc750826de40da in_seq=0 out_seq=0 2024-09-21T06:11:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd30e7fc640 1 -- 172.21.15.31:0/1347554291 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55727c2008a0 con 0x55727be52070 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd30e7fc640 10 client.15432 handle_command_reply: tid=0 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.732+0000 7fd333a67740 1 client.15432 shutdown 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 msgr2=0x55727c20d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c20d640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd3101050e0 tx=0x7fd310003e60 comp rx=0 tx=0).stop 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 shutdown_connections 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55727be52070 0x55727c288750 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55727be31390 0x55727c27aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd30e7fc640 0 client.15432 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55727be291b0 0x55727c20d640 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 --2- 172.21.15.31:0/1347554291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55727be109f0 0x55727c20d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 >> 172.21.15.31:0/1347554291 conn(0x55727be46460 msgr2=0x55727c206930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 shutdown_connections 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 1 -- 172.21.15.31:0/1347554291 wait complete. 2024-09-21T06:11:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd333a67740 20 client.15432 trim_cache size 0 max 16384 2024-09-21T06:11:27.740 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:11:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3140b65f0 msgr2=0x7fd3140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3140b65f0 0x7fd3140b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd320005fd0 tx=0x7fd320004a40 comp rx=0 tx=0).stop 2024-09-21T06:11:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 msgr2=0x7fd32819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:27.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd32819ab00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd31002a930 tx=0x7fd310004270 comp rx=0 tx=0).stop 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 shutdown_connections 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd328109df0 0x7fd32819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3140b65f0 0x7fd3140b8b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3281094a0 0x7fd32819ab00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 --2- 172.21.15.31:0/4218115077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd328103b60 0x7fd32819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.733+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 >> 172.21.15.31:0/4218115077 conn(0x7fd32807b850 msgr2=0x7fd3281075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.734+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 shutdown_connections 2024-09-21T06:11:27.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:27.734+0000 7fd32d4f3640 1 -- 172.21.15.31:0/4218115077 wait complete. 2024-09-21T06:11:29.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:29 smithi031 ceph-mon[31237]: pgmap v316: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T06:11:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:29 smithi110 ceph-mon[34550]: pgmap v316: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T06:11:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:29 smithi107 ceph-mon[34380]: pgmap v316: 97 pgs: 97 active+clean; 1.4 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T06:11:29.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:11:29.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:11:29.848 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:30.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 -- 172.21.15.31:0/2225549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d801094c0 msgr2=0x7f4d801098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 --2- 172.21.15.31:0/2225549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d801094c0 0x7f4d801098d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7402a410 comp rx=0 tx=0).stop 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 -- 172.21.15.31:0/2225549216 shutdown_connections 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 --2- 172.21.15.31:0/2225549216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d80109e10 0x7f4d80110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 --2- 172.21.15.31:0/2225549216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d801094c0 0x7f4d801098d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 --2- 172.21.15.31:0/2225549216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 0x7f4d80103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 -- 172.21.15.31:0/2225549216 >> 172.21.15.31:0/2225549216 conn(0x7f4d800ff610 msgr2=0x7f4d80101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 -- 172.21.15.31:0/2225549216 shutdown_connections 2024-09-21T06:11:30.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.340+0000 7f4d8525c640 1 -- 172.21.15.31:0/2225549216 wait complete. 2024-09-21T06:11:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.341+0000 7f4d8525c640 1 Processor -- start 2024-09-21T06:11:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.341+0000 7f4d8525c640 1 -- start start 2024-09-21T06:11:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d8525c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 0x7f4d8019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d8525c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d801094c0 0x7f4d8019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d8525c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 0x7f4d8019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d8525c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80113ad0 con 0x7f4d80109e10 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d8525c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80113950 con 0x7f4d80103b80 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d8525c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80113c50 con 0x7f4d801094c0 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 0x7f4d8019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d801094c0 0x7f4d8019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 0x7f4d8019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 0x7f4d8019fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44696/0 (socket says 172.21.15.31:44696) 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d801094c0 0x7f4d8019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41220/0 (socket says 172.21.15.31:41220) 2024-09-21T06:11:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 -- 172.21.15.31:0/1896124276 learned_addr learned my addr 172.21.15.31:0/1896124276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 -- 172.21.15.31:0/1896124276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d801094c0 msgr2=0x7f4d8019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d801094c0 0x7f4d8019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 -- 172.21.15.31:0/1896124276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 msgr2=0x7f4d8019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 0x7f4d8019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7f59e640 1 -- 172.21.15.31:0/1896124276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801a0570 con 0x7f4d80109e10 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.342+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 0x7f4d8019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.343+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 0x7f4d8019fe40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009fd0 tx=0x7f4d7800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.343+0000 7f4d67fff640 1 -- 172.21.15.31:0/1896124276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78017070 con 0x7f4d80109e10 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.343+0000 7f4d67fff640 1 -- 172.21.15.31:0/1896124276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d78003e20 con 0x7f4d80109e10 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.343+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a0860 con 0x7f4d80109e10 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.343+0000 7f4d67fff640 1 -- 172.21.15.31:0/1896124276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78010460 con 0x7f4d80109e10 2024-09-21T06:11:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.343+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a6fc0 con 0x7f4d80109e10 2024-09-21T06:11:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.345+0000 7f4d8b7d0740 1 Processor -- start 2024-09-21T06:11:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d67fff640 1 -- 172.21.15.31:0/1896124276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f4d78003970 con 0x7f4d80109e10 2024-09-21T06:11:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 -- start start 2024-09-21T06:11:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560cccb7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccb7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5560ccc2de60 con 0x5560cc8e7490 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5560ccc1d290 con 0x5560cc8ef670 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560ccc210a0 con 0x5560cc910350 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccb7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccb7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44712/0 (socket says 172.21.15.31:44712) 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3782150541 learned_addr learned my addr 172.21.15.31:0/3782150541 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560cccb7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3782150541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 msgr2=0x5560cccb7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560cccb7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3782150541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 msgr2=0x5560cccb7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccb7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3782150541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cc97b930 con 0x5560cc8e7490 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.346+0000 7f4d67fff640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4d680b65f0 0x7f4d680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560cccb7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d67fff640 1 -- 172.21.15.31:0/1896124276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4d780e7220 con 0x7f4d80109e10 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4d680b65f0 0x7f4d680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccb7970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4d700115f0 tx=0x7f4d70011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eba2942a50bda1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d657fa640 1 -- 172.21.15.31:0/3782150541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7001e070 con 0x5560cc8e7490 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d657fa640 1 -- 172.21.15.31:0/3782150541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7000faf0 con 0x5560cc8e7490 2024-09-21T06:11:30.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.347+0000 7f4d657fa640 1 -- 172.21.15.31:0/3782150541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70019ac0 con 0x5560cc8e7490 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3782150541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 msgr2=0x5560cccb7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4d680b65f0 0x7f4d680b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009230 tx=0x7f4d74000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccb7970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4d700115f0 tx=0x7f4d70011ab0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3782150541 shutdown_connections 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccb7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccb7970 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3782150541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560cccb7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3782150541 >> 172.21.15.31:0/3782150541 conn(0x5560cc904740 msgr2=0x5560cccb5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3782150541 shutdown_connections 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.348+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3782150541 wait complete. 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 -- start start 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 Processor -- start 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccc3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560ccccef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5560cca7e890 con 0x5560cc8e7490 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5560ccbb7d40 con 0x5560cc8ef670 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5560cccc5c10 con 0x5560cc910350 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d8b7d0740 10 client.0 _ll_register_callbacks cb 0x5560ccc1bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:30.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d7ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d7ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44726/0 (socket says 172.21.15.31:44726) 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3840440357 learned_addr learned my addr 172.21.15.31:0/3840440357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560ccccef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.349+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccc3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3840440357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 msgr2=0x5560ccccef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560ccccef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 msgr2=0x5560cccc3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccc3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/3840440357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ccccf4c0 con 0x5560cc8e7490 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560ccccef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc2fd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000f8f0 tx=0x7f4d7000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e7782107625721 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:30.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d62ff5640 1 -- 172.21.15.31:0/3840440357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7001e040 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d62ff5640 1 -- 172.21.15.31:0/3840440357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4d70004040 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7000faf0 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d62ff5640 1 -- 172.21.15.31:0/3840440357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4d70030050 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d8b7d0740 10 client.15444 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.350+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5560cccd1d60 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d62ff5640 1 -- 172.21.15.31:0/3840440357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7000faf0 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d62ff5640 1 -- 172.21.15.31:0/3840440357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4d70021070 con 0x5560cc8e7490 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d8b7d0740 10 client.15444 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d8b7d0740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d8b7d0740 20 client.15444 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc8cecd0 0x5560cccd5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d8b7d0740 4 client.15444 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560ccc2fdb0 con 0x5560cc8cecd0 2024-09-21T06:11:30.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc8cecd0 0x5560cccd5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc8cecd0 0x5560cccd5570 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87ce0e83b32c723 server_cookie=24ab7394d5b50f56 in_seq=0 out_seq=0 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.351+0000 7f4d62ff5640 10 client.15444 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.352+0000 7f4d62ff5640 1 -- 172.21.15.31:0/3840440357 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560ccc2fdb0 con 0x5560cc8cecd0 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.352+0000 7f4d62ff5640 10 client.15444 handle_command_reply: tid=0 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.352+0000 7f4d8b7d0740 1 client.15444 shutdown 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.352+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 msgr2=0x5560cccc2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.352+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc2fd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000f8f0 tx=0x7f4d7000f920 comp rx=0 tx=0).stop 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.352+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 shutdown_connections 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc8cecd0 0x5560cccd5570 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc910350 0x5560ccccef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccc3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d62ff5640 0 client.15444 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/3840440357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc2fd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 >> 172.21.15.31:0/3840440357 conn(0x5560cc904740 msgr2=0x5560cccba5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:30.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 shutdown_connections 2024-09-21T06:11:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/3840440357 wait complete. 2024-09-21T06:11:30.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.353+0000 7f4d8b7d0740 20 client.15444 trim_cache size 0 max 16384 2024-09-21T06:11:30.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 Processor -- start 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 -- start start 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8cecd0 0x5560cccbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560ccccee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccbffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801ab570 con 0x5560cc8e7490 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801addf0 con 0x5560cc8ef670 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.370+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801abbe0 con 0x5560cc8cecd0 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560ccccee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560ccccee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44738/0 (socket says 172.21.15.31:44738) 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 -- 172.21.15.31:0/1596265232 learned_addr learned my addr 172.21.15.31:0/1596265232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:30.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccbffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8cecd0 0x5560cccbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 -- 172.21.15.31:0/1596265232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8cecd0 msgr2=0x5560cccbd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8cecd0 0x5560cccbd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 -- 172.21.15.31:0/1596265232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 msgr2=0x5560cccbffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccbffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 -- 172.21.15.31:0/1596265232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cccaf520 con 0x5560cc8e7490 2024-09-21T06:11:30.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560ccccee00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39c00117f24631b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:30.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d64ff9640 1 -- 172.21.15.31:0/1596265232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74004240 con 0x5560cc8e7490 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d64ff9640 1 -- 172.21.15.31:0/1596265232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d740043e0 con 0x5560cc8e7490 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.371+0000 7f4d64ff9640 1 -- 172.21.15.31:0/1596265232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74004db0 con 0x5560cc8e7490 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1596265232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 msgr2=0x5560ccccee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560ccccee00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7403f040 comp rx=0 tx=0).stop 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1596265232 shutdown_connections 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8ef670 0x5560cccbffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560ccccee00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1596265232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8cecd0 0x5560cccbd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1596265232 >> 172.21.15.31:0/1596265232 conn(0x5560cc904740 msgr2=0x5560cccc1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1596265232 shutdown_connections 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.372+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1596265232 wait complete. 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 -- start start 2024-09-21T06:11:30.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 Processor -- start 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8cecd0 0x5560cccc6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8ef670 0x5560cccc7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801ac2b0 con 0x5560cc8e7490 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801ad720 con 0x5560cc8cecd0 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801ac980 con 0x5560cc8ef670 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d8b7d0740 10 client.0 _ll_register_callbacks cb 0x7f4d7001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44750/0 (socket says 172.21.15.31:44750) 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8cecd0 0x5560cccc6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7e59c640 1 -- 172.21.15.31:0/1384809581 learned_addr learned my addr 172.21.15.31:0/1384809581 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7f59e640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8ef670 0x5560cccc7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/1384809581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8ef670 msgr2=0x5560cccc7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.373+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8ef670 0x5560cccc7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/1384809581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 msgr2=0x5560cccc73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d7ed9d640 1 -- 172.21.15.31:0/1384809581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ccd3efc0 con 0x5560cc8cecd0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d7ed9d640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8cecd0 0x5560cccc6e80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d70019710 tx=0x7f4d70005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25e45cc44f7bc6d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d62ff5640 1 -- 172.21.15.31:0/1384809581 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7001e040 con 0x5560cc8cecd0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d62ff5640 1 -- 172.21.15.31:0/1384809581 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4d70011270 con 0x5560cc8cecd0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d740043e0 con 0x5560cc8cecd0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d62ff5640 1 -- 172.21.15.31:0/1384809581 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4d70005bf0 con 0x5560cc8cecd0 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d8b7d0740 10 client.25166 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:30.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5560ccd40280 con 0x5560cc8cecd0 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.374+0000 7f4d62ff5640 1 -- 172.21.15.31:0/1384809581 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d740043e0 con 0x5560cc8cecd0 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d62ff5640 1 -- 172.21.15.31:0/1384809581 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4d70021030 con 0x5560cc8cecd0 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d8b7d0740 10 client.25166 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d8b7d0740 10 client.25166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d8b7d0740 20 client.25166 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc910350 0x5560ccd42990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d8b7d0740 4 client.25166 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560ccc2b860 con 0x5560cc910350 2024-09-21T06:11:30.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc910350 0x5560ccd42990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d7e59c640 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc910350 0x5560ccd42990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecfaf930ee45e1e server_cookie=5291210b21dd8978 in_seq=0 out_seq=0 2024-09-21T06:11:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.375+0000 7f4d62ff5640 10 client.25166 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:30.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d62ff5640 1 -- 172.21.15.31:0/1384809581 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560ccc2b860 con 0x5560cc910350 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d62ff5640 10 client.25166 handle_command_reply: tid=0 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 client.25166 shutdown 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8cecd0 msgr2=0x5560cccc6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8cecd0 0x5560cccc6e80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4d70019710 tx=0x7f4d70005f90 comp rx=0 tx=0).stop 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 shutdown_connections 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5560cc910350 0x5560ccd42990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5560cc8ef670 0x5560cccc7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560cc8e7490 0x5560cccc73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 --2- 172.21.15.31:0/1384809581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5560cc8cecd0 0x5560cccc6e80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 >> 172.21.15.31:0/1384809581 conn(0x5560cc904740 msgr2=0x5560cccc1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:30.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d62ff5640 0 client.25166 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 shutdown_connections 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 1 -- 172.21.15.31:0/1384809581 wait complete. 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.377+0000 7f4d8b7d0740 20 client.25166 trim_cache size 0 max 16384 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "f09db644-bf80-47e4-97a1-7c8d6f95b83b", 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:11:30.383 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4d680b65f0 msgr2=0x7f4d680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4d680b65f0 0x7f4d680b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009230 tx=0x7f4d74000bc0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 msgr2=0x7f4d8019fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 0x7f4d8019fe40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009fd0 tx=0x7f4d7800b260 comp rx=0 tx=0).stop 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 shutdown_connections 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4d680b65f0 0x7f4d680b8b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d80109e10 0x7f4d8019fe40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d801094c0 0x7f4d8019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 --2- 172.21.15.31:0/1896124276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d80103b80 0x7f4d8019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 >> 172.21.15.31:0/1896124276 conn(0x7f4d800ff610 msgr2=0x7f4d80101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 shutdown_connections 2024-09-21T06:11:30.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:30.378+0000 7f4d8525c640 1 -- 172.21.15.31:0/1896124276 wait complete. 2024-09-21T06:11:31.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:11:31.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:31 smithi031 ceph-mon[31237]: pgmap v317: 97 pgs: 97 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 709 KiB/s wr, 56 op/s 2024-09-21T06:11:31.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:31 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:11:31.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:31 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:31.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:31 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:11:31.481 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:31 smithi110 ceph-mon[34550]: pgmap v317: 97 pgs: 97 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 709 KiB/s wr, 56 op/s 2024-09-21T06:11:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:31 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:11:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:31 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:31 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:11:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:31 smithi107 ceph-mon[34380]: pgmap v317: 97 pgs: 97 active+clean; 7.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 709 KiB/s wr, 56 op/s 2024-09-21T06:11:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:31 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:11:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:31 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:31 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:11:31.741 INFO:teuthology.orchestra.run.smithi031.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-21T06:11:31.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T06:11:31.826 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/: 2024-09-21T06:11:31.826 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.826 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:11:31.826 INFO:teuthology.orchestra.run.smithi031.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 .. 2024-09-21T06:11:31.826 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 data 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data: 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:total 35 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 . 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 .. 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 base 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 06:11 global 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 log 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_commit_ts 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_dynshmem 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 06:11 pg_hba.conf 2024-09-21T06:11:31.827 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 06:11 pg_ident.conf 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 pg_logical 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 pg_multixact 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_notify 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_replslot 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_serial 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_snapshots 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_stat 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_stat_tmp 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 pg_subtrans 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_tblspc 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_twophase 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:11:31.828 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 pg_wal 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 pg_xact 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 06:11 postgresql.auto.conf 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 06:11 postgresql.conf 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 . 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 1 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 13493 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 13494 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.829 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:total 7777 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 . 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 .. 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 112 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 113 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 1247 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1247_fsm 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1247_vm 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:11 1249 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1249_fsm 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1249_vm 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:11 1255 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1255_fsm 2024-09-21T06:11:31.830 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1255_vm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:11 1259 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1259_fsm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1259_vm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:11 13344 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13344_fsm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13344_vm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13346 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13348 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13349_fsm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349_vm 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13351 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13353 2024-09-21T06:11:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13354_fsm 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354_vm 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13356 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13358 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13359_fsm 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359_vm 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13361 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13363 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1417 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1418 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 174 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 175 2024-09-21T06:11:31.832 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2187 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2224 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2328 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2336 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2337 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2579 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2600 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2600_fsm 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2600_vm 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2601_fsm 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601_vm 2024-09-21T06:11:31.833 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2602 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2602_fsm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2602_vm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2603 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2603_fsm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2603_vm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2604 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2605 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2605_fsm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2605_vm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2606_fsm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606_vm 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2607 2024-09-21T06:11:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2607_fsm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2607_vm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:11 2608 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2608_fsm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2608_vm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:11 2609 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2609_fsm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2609_vm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2610 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2610_fsm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2610_vm 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2611 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612 2024-09-21T06:11:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2612_fsm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612_vm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2613 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2615_fsm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615_vm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2616 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2616_fsm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2616_vm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:11 2617 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2617_fsm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2617_vm 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:11 2618 2024-09-21T06:11:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2618_fsm 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2618_vm 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:11 2619 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2619_fsm 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2619_vm 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2620 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2650 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2651 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2652 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2653 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2654 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2655 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2656 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2657 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:11 2658 2024-09-21T06:11:31.837 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 2659 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2660 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2661 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2662 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2663 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2664 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2665 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2666 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2667 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2668 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2669 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2670 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:11 2673 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:11 2674 2024-09-21T06:11:31.838 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:11 2675 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2678 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2679 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2680 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2681 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2682 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2683 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2684 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2685 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2686 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2687 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2688 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2689 2024-09-21T06:11:31.839 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:11 2690 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:11 2691 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2692 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2693 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2696 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2699 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2701 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2702 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2703 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2704 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2753 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2753_fsm 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2753_vm 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2754 2024-09-21T06:11:31.840 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2755 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2756 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2757 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2830 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2831 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2832 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2833 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2834 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2835 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2836 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2837 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:11 2838 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2838_fsm 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2838_vm 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2839 2024-09-21T06:11:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840_fsm 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2840_vm 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2841 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2995 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2996 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3079_fsm 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079_vm 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3080 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3081 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3085 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3118 2024-09-21T06:11:31.842 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3119 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:11 3164 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3256 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3257 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3258 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3350 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3351 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3379 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3380 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3381 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3394 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3394_fsm 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3394_vm 2024-09-21T06:11:31.843 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3395 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3429 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3430 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3431 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3433 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3439 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3440 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3455 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:11 3456 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3456_fsm 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3456_vm 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3466 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3467 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3468 2024-09-21T06:11:31.844 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3501 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3502 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3503 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3534 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3541_fsm 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541_vm 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3542 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3574 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3575 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3576 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3596 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3597 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3598 2024-09-21T06:11:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3599 2024-09-21T06:11:31.846 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600 2024-09-21T06:11:31.846 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3600_fsm 2024-09-21T06:11:31.846 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600_vm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3601_fsm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601_vm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3602_fsm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602_vm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603_fsm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3603_vm 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3604 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3605 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3606 2024-09-21T06:11:31.847 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3607 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3608 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 3609 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3712 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3764_fsm 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764_vm 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3766 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3767 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3997 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4143 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4144 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4145 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4146 2024-09-21T06:11:31.848 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4147 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4148 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4149 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4150 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4151 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4152 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4153 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4154 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4155 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4156 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4157 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4158 2024-09-21T06:11:31.849 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4159 2024-09-21T06:11:31.850 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4160 2024-09-21T06:11:31.850 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4161 2024-09-21T06:11:31.850 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4162 2024-09-21T06:11:31.850 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4163 2024-09-21T06:11:31.850 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4164 2024-09-21T06:11:31.850 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4165 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4166 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4167 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4168 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4169 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4170 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4171 2024-09-21T06:11:31.853 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4172 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4173 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4174 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 5002 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 548 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 549 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6102 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6104 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6106 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6110 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6111 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6112 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6113 2024-09-21T06:11:31.854 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6117 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 826 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 827 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 828 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:total 7777 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 . 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 .. 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 112 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 113 2024-09-21T06:11:31.855 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 1247 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1247_fsm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1247_vm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:11 1249 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1249_fsm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1249_vm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:11 1255 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1255_fsm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1255_vm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:11 1259 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1259_fsm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1259_vm 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:11 13344 2024-09-21T06:11:31.856 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13344_fsm 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13344_vm 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13346 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13348 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13349_fsm 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349_vm 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13351 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13353 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13354_fsm 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354_vm 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13356 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13358 2024-09-21T06:11:31.857 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13359_fsm 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359_vm 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13361 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13363 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1417 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1418 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 174 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 175 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2187 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2224 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2328 2024-09-21T06:11:31.858 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2336 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2337 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2579 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2600 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2600_fsm 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2600_vm 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2601_fsm 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601_vm 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2602 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2602_fsm 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2602_vm 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2603 2024-09-21T06:11:31.859 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2603_fsm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2603_vm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2604 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2605 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2605_fsm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2605_vm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2606_fsm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606_vm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2607 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2607_fsm 2024-09-21T06:11:31.860 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2607_vm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:11 2608 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2608_fsm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2608_vm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:11 2609 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2609_fsm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2609_vm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2610 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2610_fsm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2610_vm 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2611 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612 2024-09-21T06:11:31.861 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2612_fsm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612_vm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2613 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2615_fsm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615_vm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2616 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2616_fsm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2616_vm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:11 2617 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2617_fsm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2617_vm 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:11 2618 2024-09-21T06:11:31.862 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2618_fsm 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2618_vm 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:11 2619 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2619_fsm 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2619_vm 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2620 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2650 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2651 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2652 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2653 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2654 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2655 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2656 2024-09-21T06:11:31.863 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2657 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:11 2658 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 2659 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2660 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2661 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2662 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2663 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2664 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2665 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2666 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2667 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2668 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2669 2024-09-21T06:11:31.864 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2670 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:11 2673 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:11 2674 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:11 2675 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2678 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2679 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2680 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2681 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2682 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2683 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2684 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2685 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2686 2024-09-21T06:11:31.865 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2687 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2688 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2689 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:11 2690 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:11 2691 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2692 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2693 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2696 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2699 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2701 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2702 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2703 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2704 2024-09-21T06:11:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2753 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2753_fsm 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2753_vm 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2754 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2755 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2756 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2757 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2830 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2831 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2832 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2833 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2834 2024-09-21T06:11:31.867 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2835 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2836 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2837 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:11 2838 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2838_fsm 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2838_vm 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2839 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840_fsm 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2840_vm 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2841 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2995 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2996 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079 2024-09-21T06:11:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3079_fsm 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079_vm 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3080 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3081 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3085 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3118 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3119 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:11 3164 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3256 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3257 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3258 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3350 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3351 2024-09-21T06:11:31.869 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3379 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3380 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3381 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3394 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3394_fsm 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3394_vm 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3395 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3429 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3430 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3431 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3433 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3439 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3440 2024-09-21T06:11:31.870 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3455 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:11 3456 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3456_fsm 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3456_vm 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3466 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3467 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3468 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3501 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3502 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3503 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3534 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3541_fsm 2024-09-21T06:11:31.871 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541_vm 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3542 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3574 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3575 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3576 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3596 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3597 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3598 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3599 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3600_fsm 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600_vm 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3601_fsm 2024-09-21T06:11:31.872 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601_vm 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3602_fsm 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602_vm 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603_fsm 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3603_vm 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3604 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3605 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3606 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3607 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3608 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 3609 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3712 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764 2024-09-21T06:11:31.873 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3764_fsm 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764_vm 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3766 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3767 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3997 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4143 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4144 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4145 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4146 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4147 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4148 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4149 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4150 2024-09-21T06:11:31.874 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4151 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4152 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4153 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4154 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4155 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4156 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4157 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4158 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4159 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4160 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4161 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4162 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4163 2024-09-21T06:11:31.875 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4164 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4165 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4166 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4167 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4168 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4169 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4170 2024-09-21T06:11:31.876 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4171 2024-09-21T06:11:31.877 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4172 2024-09-21T06:11:31.877 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4173 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4174 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 5002 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 548 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 549 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6102 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6104 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6106 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6110 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6111 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6112 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6113 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6117 2024-09-21T06:11:31.878 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 826 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 827 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 828 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:total 7777 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 . 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 .. 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 112 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 113 2024-09-21T06:11:31.879 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 1247 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1247_fsm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1247_vm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:11 1249 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1249_fsm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1249_vm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:11 1255 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1255_fsm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1255_vm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:11 1259 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1259_fsm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1259_vm 2024-09-21T06:11:31.880 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:11 13344 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13344_fsm 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13344_vm 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13346 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13348 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13349_fsm 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349_vm 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13351 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13353 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13354_fsm 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354_vm 2024-09-21T06:11:31.881 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13356 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13358 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13359_fsm 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359_vm 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13361 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13363 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1417 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1418 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 174 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 175 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2187 2024-09-21T06:11:31.882 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2224 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2328 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2336 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2337 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2579 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2600 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2600_fsm 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2600_vm 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2601_fsm 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601_vm 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2602 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2602_fsm 2024-09-21T06:11:31.883 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2602_vm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2603 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2603_fsm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2603_vm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2604 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2605 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2605_fsm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2605_vm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2606_fsm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606_vm 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2607 2024-09-21T06:11:31.884 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2607_fsm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2607_vm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:11 2608 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2608_fsm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2608_vm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:11 2609 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2609_fsm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2609_vm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2610 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2610_fsm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2610_vm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2611 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2612_fsm 2024-09-21T06:11:31.885 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612_vm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2613 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2615_fsm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615_vm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2616 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2616_fsm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2616_vm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:11 2617 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2617_fsm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2617_vm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:11 2618 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2618_fsm 2024-09-21T06:11:31.886 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2618_vm 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:11 2619 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2619_fsm 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2619_vm 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2620 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2650 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2651 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2652 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2653 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2654 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2655 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2656 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2657 2024-09-21T06:11:31.887 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:11 2658 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 2659 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2660 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2661 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2662 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2663 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2664 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2665 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2666 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2667 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2668 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2669 2024-09-21T06:11:31.888 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2670 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:11 2673 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:11 2674 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:11 2675 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2678 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2679 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2680 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2681 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2682 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2683 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2684 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2685 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2686 2024-09-21T06:11:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2687 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2688 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2689 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:11 2690 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:11 2691 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2692 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2693 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2696 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2699 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2701 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2702 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2703 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2704 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2753 2024-09-21T06:11:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2753_fsm 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2753_vm 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2754 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2755 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2756 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2757 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2830 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2831 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2832 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2833 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2834 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2835 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2836 2024-09-21T06:11:31.891 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2837 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:11 2838 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2838_fsm 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2838_vm 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2839 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840_fsm 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2840_vm 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2841 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2995 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2996 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3079_fsm 2024-09-21T06:11:31.892 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079_vm 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3080 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3081 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3085 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3118 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3119 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:11 3164 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3256 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3257 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3258 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3350 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3351 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3379 2024-09-21T06:11:31.893 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3380 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3381 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3394 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3394_fsm 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3394_vm 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3395 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3429 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3430 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3431 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3433 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3439 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3440 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3455 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:11 3456 2024-09-21T06:11:31.894 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3456_fsm 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3456_vm 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3466 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3467 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3468 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3501 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3502 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3503 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3534 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3541_fsm 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541_vm 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3542 2024-09-21T06:11:31.895 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3574 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3575 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3576 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3596 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3597 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3598 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3599 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3600_fsm 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600_vm 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3601_fsm 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601_vm 2024-09-21T06:11:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3602_fsm 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602_vm 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603_fsm 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3603_vm 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3604 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3605 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3606 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3607 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3608 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 3609 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3712 2024-09-21T06:11:31.897 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3764_fsm 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764_vm 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3766 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3767 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3997 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4143 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4144 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4145 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4146 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4147 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4148 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4149 2024-09-21T06:11:31.898 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4150 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4151 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4152 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4153 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4154 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4155 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4156 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4157 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4158 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4159 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4160 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4161 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4162 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4163 2024-09-21T06:11:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4164 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4165 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4166 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4167 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4168 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4169 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4170 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4171 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4172 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4173 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4174 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 5002 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 548 2024-09-21T06:11:31.900 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 549 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6102 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6104 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6106 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6110 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6111 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6112 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6113 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6117 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 826 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 827 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 828 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.901 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:total 537 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 06:11 . 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1213 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1213_fsm 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1213_vm 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1214 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1214_fsm 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1214_vm 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 1232 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 1233 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1260 2024-09-21T06:11:31.902 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1260_fsm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1260_vm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1261 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1261_fsm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1261_vm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1262 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1262_fsm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1262_vm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2396 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2396_fsm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2396_vm 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2397 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2671 2024-09-21T06:11:31.903 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2672 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2676 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2677 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2694 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2695 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2697 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2698 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2846 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2847 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2964 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2965 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2966 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2967 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3592 2024-09-21T06:11:31.904 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3593 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4060 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4061 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4175 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4176 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4177 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4178 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4181 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4182 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4183 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4184 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4185 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4186 2024-09-21T06:11:31.905 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6000 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6001 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6002 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6100 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6114 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6115 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 pg_control 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:total 1 2024-09-21T06:11:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 . 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 mappings 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 06:11 replorigin_checkpoint 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 snapshots 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 .. 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.908 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 .. 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 . 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 members 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 offsets 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:total 8 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:11:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 .. 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 0000 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:total 8 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 .. 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 0000 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.911 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:total 8 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 0000 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:total 16384 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 . 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:11 000000010000000000000001 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 archive_status 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 .. 2024-09-21T06:11:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:31.915 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T06:11:31.915 INFO:teuthology.orchestra.run.smithi031.stdout:total 8 2024-09-21T06:11:31.915 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:11:31.915 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:11 .. 2024-09-21T06:11:31.915 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 0000 2024-09-21T06:11:31.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-21T06:11:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 -- 172.21.15.31:0/1134672570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c102a60 msgr2=0x7f155c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 --2- 172.21.15.31:0/1134672570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c102a60 0x7f155c102e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1558009900 tx=0x7f155802a410 comp rx=0 tx=0).stop 2024-09-21T06:11:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 -- 172.21.15.31:0/1134672570 shutdown_connections 2024-09-21T06:11:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 --2- 172.21.15.31:0/1134672570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155c108cf0 0x7f155c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 --2- 172.21.15.31:0/1134672570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155c1083a0 0x7f155c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 --2- 172.21.15.31:0/1134672570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c102a60 0x7f155c102e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 -- 172.21.15.31:0/1134672570 >> 172.21.15.31:0/1134672570 conn(0x7f155c079730 msgr2=0x7f155c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 -- 172.21.15.31:0/1134672570 shutdown_connections 2024-09-21T06:11:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 -- 172.21.15.31:0/1134672570 wait complete. 2024-09-21T06:11:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 Processor -- start 2024-09-21T06:11:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.021+0000 7f1563f0d640 1 -- start start 2024-09-21T06:11:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1563f0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155c102a60 0x7f155c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1563f0d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155c1083a0 0x7f155c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1563f0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 0x7f155c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1563f0d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f155c112960 con 0x7f155c108cf0 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1563f0d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1127e0 con 0x7f155c1083a0 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1563f0d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f155c112ae0 con 0x7f155c102a60 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 0x7f155c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 0x7f155c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44764/0 (socket says 172.21.15.31:44764) 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 -- 172.21.15.31:0/2233959580 learned_addr learned my addr 172.21.15.31:0/2233959580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f1561ca9640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155c102a60 0x7f155c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15614a8640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155c1083a0 0x7f155c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 -- 172.21.15.31:0/2233959580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155c102a60 msgr2=0x7f155c19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155c102a60 0x7f155c19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 -- 172.21.15.31:0/2233959580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155c1083a0 msgr2=0x7f155c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155c1083a0 0x7f155c19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.022+0000 7f15624aa640 1 -- 172.21.15.31:0/2233959580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c1a64c0 con 0x7f155c108cf0 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.023+0000 7f15624aa640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 0x7f155c19bcb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f155400eb10 tx=0x7f155400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.023+0000 7f154effd640 1 -- 172.21.15.31:0/2233959580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554004b20 con 0x7f155c108cf0 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.023+0000 7f154effd640 1 -- 172.21.15.31:0/2233959580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1554004cc0 con 0x7f155c108cf0 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.023+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c1a67b0 con 0x7f155c108cf0 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.023+0000 7f154effd640 1 -- 172.21.15.31:0/2233959580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554010670 con 0x7f155c108cf0 2024-09-21T06:11:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.023+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c1a6cf0 con 0x7f155c108cf0 2024-09-21T06:11:32.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.026+0000 7f156a481740 1 Processor -- start 2024-09-21T06:11:32.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.026+0000 7f154effd640 1 -- 172.21.15.31:0/2233959580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f1554005020 con 0x7f155c108cf0 2024-09-21T06:11:32.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.026+0000 7f156a481740 1 -- start start 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f156a481740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b806070 0x55ad8bbad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f156a481740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f156a481740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7e5390 0x55ad8bbae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f156a481740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ad8bb23e50 con 0x55ad8b7e5390 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f156a481740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ad8bb13280 con 0x55ad8b806070 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f156a481740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad8bb17090 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f15614a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7e5390 0x55ad8bbae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f1561ca9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f15614a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7e5390 0x55ad8bbae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44768/0 (socket says 172.21.15.31:44768) 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f15614a8640 1 -- 172.21.15.31:0/2657034111 learned_addr learned my addr 172.21.15.31:0/2657034111 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f15614a8640 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b806070 0x55ad8bbad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f1561ca9640 1 -- 172.21.15.31:0/2657034111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b806070 msgr2=0x55ad8bbad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f154effd640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15480b6620 0x7f15480b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f1561ca9640 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b806070 0x55ad8bbad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f1561ca9640 1 -- 172.21.15.31:0/2657034111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7e5390 msgr2=0x55ad8bbae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f1561ca9640 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7e5390 0x55ad8bbae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f1561ca9640 1 -- 172.21.15.31:0/2657034111 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8b871650 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.027+0000 7f15624aa640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15480b6620 0x7f15480b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.028+0000 7f1561ca9640 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbadb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f155c06b9c0 tx=0x7f1558032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567d2c878df994fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.028+0000 7f154effd640 1 -- 172.21.15.31:0/2233959580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f15540e7aa0 con 0x7f155c108cf0 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.028+0000 7f1547fff640 1 -- 172.21.15.31:0/2657034111 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558040070 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.028+0000 7f1547fff640 1 -- 172.21.15.31:0/2657034111 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1558004030 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.028+0000 7f1547fff640 1 -- 172.21.15.31:0/2657034111 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558035040 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 -- 172.21.15.31:0/2657034111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 msgr2=0x55ad8bbadb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f15624aa640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15480b6620 0x7f15480b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15540ad980 tx=0x7f155401c7b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbadb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f155c06b9c0 tx=0x7f1558032bd0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 -- 172.21.15.31:0/2657034111 shutdown_connections 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7e5390 0x55ad8bbae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbadb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 --2- 172.21.15.31:0/2657034111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b806070 0x55ad8bbad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 -- 172.21.15.31:0/2657034111 >> 172.21.15.31:0/2657034111 conn(0x55ad8b7fa460 msgr2=0x55ad8bbab540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 -- 172.21.15.31:0/2657034111 shutdown_connections 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 -- 172.21.15.31:0/2657034111 wait complete. 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 -- start start 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.029+0000 7f156a481740 1 Processor -- start 2024-09-21T06:11:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbb8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b806070 0x55ad8bbc50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ad8b974700 con 0x55ad8b806070 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ad8baae110 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad8bbbbdd0 con 0x55ad8b7e5390 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f156a481740 10 client.0 _ll_register_callbacks cb 0x55ad8bac7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f15614a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b806070 0x55ad8bbc50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41308/0 (socket says 172.21.15.31:41308) 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 -- 172.21.15.31:0/458896076 learned_addr learned my addr 172.21.15.31:0/458896076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f15614a8640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbb8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 -- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7dd1b0 msgr2=0x55ad8bbb8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbb8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 -- 172.21.15.31:0/458896076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b806070 msgr2=0x55ad8bbc50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b806070 0x55ad8bbc50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f1561ca9640 1 -- 172.21.15.31:0/458896076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8badbc40 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.030+0000 7f15614a8640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b806070 0x55ad8bbc50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f1561ca9640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb9270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1558005de0 tx=0x7f1558005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e0e5d7c0a2599e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f15457fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f15457fa640 1 -- 172.21.15.31:0/458896076 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558040070 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f15457fa640 1 -- 172.21.15.31:0/458896076 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f15580316a0 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1558004030 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f15457fa640 1 -- 172.21.15.31:0/458896076 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1558056050 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f156a481740 10 client.25189 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ad8bbc7b10 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.031+0000 7f15457fa640 1 -- 172.21.15.31:0/458896076 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1558004030 con 0x55ad8b7e5390 2024-09-21T06:11:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f15457fa640 1 -- 172.21.15.31:0/458896076 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1558004a80 con 0x55ad8b7e5390 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f156a481740 10 client.25189 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f156a481740 10 client.25189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f156a481740 20 client.25189 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f156a481740 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b7c49f0 0x55ad8bbcae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f156a481740 4 client.25189 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad8bb25da0 con 0x55ad8b7c49f0 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f15614a8640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b7c49f0 0x55ad8bbcae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f15457fa640 10 client.25189 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.032+0000 7f15614a8640 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b7c49f0 0x55ad8bbcae70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445560abf1274269 server_cookie=7de813c7a736d935 in_seq=0 out_seq=0 2024-09-21T06:11:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.033+0000 7f15457fa640 1 -- 172.21.15.31:0/458896076 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad8bb25da0 con 0x55ad8b7c49f0 2024-09-21T06:11:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.033+0000 7f15457fa640 10 client.25189 handle_command_reply: tid=0 2024-09-21T06:11:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.033+0000 7f156a481740 1 client.25189 shutdown 2024-09-21T06:11:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.033+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 msgr2=0x55ad8bbb9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.033+0000 7f156a481740 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb9270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1558005de0 tx=0x7f1558005e10 comp rx=0 tx=0).stop 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 shutdown_connections 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b7c49f0 0x55ad8bbcae70 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b806070 0x55ad8bbc50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb9270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f15457fa640 0 client.25189 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 --2- 172.21.15.31:0/458896076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbb8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 >> 172.21.15.31:0/458896076 conn(0x55ad8b7fa460 msgr2=0x55ad8bbb0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 shutdown_connections 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 1 -- 172.21.15.31:0/458896076 wait complete. 2024-09-21T06:11:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.034+0000 7f156a481740 20 client.25189 trim_cache size 0 max 16384 2024-09-21T06:11:32.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.053+0000 7f156a481740 1 Processor -- start 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 -- start start 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbb91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbc5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1ad5a0 con 0x55ad8b7c49f0 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1afe20 con 0x55ad8b7e5390 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f156a481740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1adc10 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f15614a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbb91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f15614a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbb91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44776/0 (socket says 172.21.15.31:44776) 2024-09-21T06:11:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f15614a8640 1 -- 172.21.15.31:0/3397297322 learned_addr learned my addr 172.21.15.31:0/3397297322 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f1561ca9640 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbc5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f15614a8640 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f1561ca9640 1 -- 172.21.15.31:0/3397297322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7e5390 msgr2=0x55ad8bbb5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f1561ca9640 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f1561ca9640 1 -- 172.21.15.31:0/3397297322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 msgr2=0x55ad8bbb91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f1561ca9640 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbb91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f1561ca9640 1 -- 172.21.15.31:0/3397297322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad8bbb04b0 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.054+0000 7f15614a8640 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbb91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.055+0000 7f1561ca9640 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbc5030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1558009900 tx=0x7f1558031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b493c46dd0feb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.055+0000 7f15477fe640 1 -- 172.21.15.31:0/3397297322 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558060070 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.055+0000 7f15477fe640 1 -- 172.21.15.31:0/3397297322 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f155804d070 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.055+0000 7f15477fe640 1 -- 172.21.15.31:0/3397297322 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558040040 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 -- 172.21.15.31:0/3397297322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 msgr2=0x55ad8bbc5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbc5030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1558009900 tx=0x7f1558031400 comp rx=0 tx=0).stop 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 -- 172.21.15.31:0/3397297322 shutdown_connections 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7e5390 0x55ad8bbb5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbc5030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 --2- 172.21.15.31:0/3397297322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbb91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 -- 172.21.15.31:0/3397297322 >> 172.21.15.31:0/3397297322 conn(0x55ad8b7fa460 msgr2=0x55ad8bbb8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 -- 172.21.15.31:0/3397297322 shutdown_connections 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 -- 172.21.15.31:0/3397297322 wait complete. 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.056+0000 7f156a481740 1 -- start start 2024-09-21T06:11:32.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.057+0000 7f156a481740 1 Processor -- start 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.057+0000 7f156a481740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbcbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.057+0000 7f156a481740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbcc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.057+0000 7f156a481740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.057+0000 7f156a481740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1ae2e0 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f156a481740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1af750 con 0x55ad8b7c49f0 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f15614a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbcbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbcc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f156a481740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1ae9b0 con 0x55ad8b7e5390 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbcc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44790/0 (socket says 172.21.15.31:44790) 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 -- 172.21.15.31:0/677139351 learned_addr learned my addr 172.21.15.31:0/677139351 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f15614a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbcbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37342/0 (socket says 172.21.15.31:37342) 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f156a481740 10 client.0 _ll_register_callbacks cb 0x7f155000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:32.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f15614a8640 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 -- 172.21.15.31:0/677139351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 msgr2=0x55ad8bbcc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbcc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 -- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7c49f0 msgr2=0x55ad8bbcbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbcbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 -- 172.21.15.31:0/677139351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad8bc345f0 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f1561ca9640 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbcc340 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1558005780 tx=0x7f1558005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d7bff30b9a640 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f15457fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f15457fa640 1 -- 172.21.15.31:0/677139351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558040070 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.058+0000 7f15457fa640 1 -- 172.21.15.31:0/677139351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f15580046c0 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f155804d070 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f15457fa640 1 -- 172.21.15.31:0/677139351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f155806c050 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 10 client.15468 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ad8bc357c0 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f15457fa640 1 -- 172.21.15.31:0/677139351 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f155804d070 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f15457fa640 1 -- 172.21.15.31:0/677139351 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1558031c10 con 0x55ad8b7dd1b0 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 10 client.15468 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 20 client.15468 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b806070 0x55ad8bc37e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 4 client.15468 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:11:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.059+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad8bb21850 con 0x55ad8b806070 2024-09-21T06:11:32.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f15614a8640 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b806070 0x55ad8bc37e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f15457fa640 10 client.15468 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f15614a8640 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b806070 0x55ad8bc37e40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7bf1909e710248f server_cookie=9be90f4bd434bb70 in_seq=0 out_seq=0 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f15457fa640 1 -- 172.21.15.31:0/677139351 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55ad8bb21850 con 0x55ad8b806070 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f15457fa640 10 client.15468 handle_command_reply: tid=0 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f156a481740 1 client.15468 shutdown 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 msgr2=0x55ad8bbcc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.060+0000 7f156a481740 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbcc340 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1558005780 tx=0x7f1558005e10 comp rx=0 tx=0).stop 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 shutdown_connections 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad8b806070 0x55ad8bc37e40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad8b7e5390 0x55ad8bbcc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad8b7dd1b0 0x55ad8bbcc340 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 --2- 172.21.15.31:0/677139351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad8b7c49f0 0x55ad8bbcbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 >> 172.21.15.31:0/677139351 conn(0x55ad8b7fa460 msgr2=0x55ad8bbae680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f15457fa640 0 client.15468 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 shutdown_connections 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 1 -- 172.21.15.31:0/677139351 wait complete. 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f156a481740 20 client.15468 trim_cache size 0 max 16384 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:11:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:11:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15480b6620 msgr2=0x7f15480b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f1563f0d640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15480b6620 0x7f15480b8b30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15540ad980 tx=0x7f155401c7b0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 msgr2=0x7f155c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:32.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.061+0000 7f1563f0d640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 0x7f155c19bcb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f155400eb10 tx=0x7f155400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 shutdown_connections 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f15480b6620 0x7f15480b8b30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c108cf0 0x7f155c19bcb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f155c1083a0 0x7f155c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 --2- 172.21.15.31:0/2233959580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f155c102a60 0x7f155c19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 >> 172.21.15.31:0/2233959580 conn(0x7f155c079730 msgr2=0x7f155c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 shutdown_connections 2024-09-21T06:11:32.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:32.062+0000 7f1563f0d640 1 -- 172.21.15.31:0/2233959580 wait complete. 2024-09-21T06:11:32.108 INFO:teuthology.orchestra.run.smithi031.stderr:dropping old tables... 2024-09-21T06:11:32.108 INFO:teuthology.orchestra.run.smithi031.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-21T06:11:32.108 INFO:teuthology.orchestra.run.smithi031.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-21T06:11:32.108 INFO:teuthology.orchestra.run.smithi031.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-21T06:11:32.108 INFO:teuthology.orchestra.run.smithi031.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-21T06:11:32.108 INFO:teuthology.orchestra.run.smithi031.stderr:creating tables... 2024-09-21T06:11:32.119 INFO:teuthology.orchestra.run.smithi031.stderr:generating data (client-side)... 2024-09-21T06:11:32.217 INFO:teuthology.orchestra.run.smithi031.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.08 s, remaining 2.40 s) 2024-09-21T06:11:32.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:32 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:32.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:32 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:11:32.388 INFO:teuthology.orchestra.run.smithi031.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.25 s, remaining 3.73 s) 2024-09-21T06:11:32.484 INFO:teuthology.orchestra.run.smithi031.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.34 s, remaining 3.33 s) 2024-09-21T06:11:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:32 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:32.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:32 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:11:32.538 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:32 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:32.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:32 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:11:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.50 s, remaining 3.48 s) 2024-09-21T06:11:32.802 INFO:teuthology.orchestra.run.smithi031.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.66 s, remaining 3.58 s) 2024-09-21T06:11:32.949 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:11:32.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:11:32.987 INFO:teuthology.orchestra.run.smithi031.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.85 s, remaining 3.67 s) 2024-09-21T06:11:33.044 INFO:teuthology.orchestra.run.smithi031.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.90 s, remaining 3.23 s) 2024-09-21T06:11:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.11 s, remaining 3.32 s) 2024-09-21T06:11:33.269 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:33.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:33 smithi031 ceph-mon[31237]: pgmap v318: 97 pgs: 97 active+clean; 32 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-09-21T06:11:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.28 s, remaining 3.28 s) 2024-09-21T06:11:33.484 INFO:teuthology.orchestra.run.smithi031.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.34 s, remaining 2.96 s) 2024-09-21T06:11:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:33 smithi110 ceph-mon[34550]: pgmap v318: 97 pgs: 97 active+clean; 32 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-09-21T06:11:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:33 smithi107 ceph-mon[34380]: pgmap v318: 97 pgs: 97 active+clean; 32 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 2.7 MiB/s wr, 91 op/s 2024-09-21T06:11:33.682 INFO:teuthology.orchestra.run.smithi031.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.54 s, remaining 2.94 s) 2024-09-21T06:11:33.742 INFO:teuthology.orchestra.run.smithi031.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.60 s, remaining 2.67 s) 2024-09-21T06:11:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 -- 172.21.15.31:0/3211005532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 msgr2=0x7fd480112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 --2- 172.21.15.31:0/3211005532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd480112600 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd478009ca0 tx=0x7fd47802a970 comp rx=0 tx=0).stop 2024-09-21T06:11:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 -- 172.21.15.31:0/3211005532 shutdown_connections 2024-09-21T06:11:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 --2- 172.21.15.31:0/3211005532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd480112600 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 --2- 172.21.15.31:0/3211005532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd480102f80 0x7fd48010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 --2- 172.21.15.31:0/3211005532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102630 0x7fd480102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 -- 172.21.15.31:0/3211005532 >> 172.21.15.31:0/3211005532 conn(0x7fd480079730 msgr2=0x7fd480079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 -- 172.21.15.31:0/3211005532 shutdown_connections 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.762+0000 7fd484bb4640 1 -- 172.21.15.31:0/3211005532 wait complete. 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.763+0000 7fd484bb4640 1 Processor -- start 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.763+0000 7fd484bb4640 1 -- start start 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd484bb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd480102630 0x7fd48019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd484bb4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102f80 0x7fd48019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd484bb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd4801a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd484bb4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd480115cb0 con 0x7fd4801101e0 2024-09-21T06:11:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd484bb4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd480115b30 con 0x7fd480102f80 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd484bb4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd480115e30 con 0x7fd480102630 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd4801a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd4801a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44808/0 (socket says 172.21.15.31:44808) 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 -- 172.21.15.31:0/2813371327 learned_addr learned my addr 172.21.15.31:0/2813371327 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47dd9b640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102f80 0x7fd48019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 -- 172.21.15.31:0/2813371327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd480102630 msgr2=0x7fd48019c700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd480102630 0x7fd48019c700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 -- 172.21.15.31:0/2813371327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102f80 msgr2=0x7fd48019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102f80 0x7fd48019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 -- 172.21.15.31:0/2813371327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801a25b0 con 0x7fd4801101e0 2024-09-21T06:11:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47dd9b640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102f80 0x7fd48019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd47ed9d640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd4801a1f40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd478009dc0 tx=0x7fd478003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.764+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478034030 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.765+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801a2840 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.765+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47802f870 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.765+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47802e590 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.765+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a2d20 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.766+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480102a40 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.768+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fd478037020 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.769+0000 7fd4777fe640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd46c0b6620 0x7fd46c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.769+0000 7fd47e59c640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd46c0b6620 0x7fd46c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.769+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd478108ac0 con 0x7fd4801101e0 2024-09-21T06:11:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.769+0000 7fd47e59c640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd46c0b6620 0x7fd46c0b8b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd47000a930 tx=0x7fd470005c00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:33.772+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4780cf110 con 0x7fd4801101e0 2024-09-21T06:11:33.921 INFO:teuthology.orchestra.run.smithi031.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.78 s, remaining 2.60 s) 2024-09-21T06:11:34.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.029+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4801a91f0 con 0x7fd4801101e0 2024-09-21T06:11:34.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.030+0000 7fd4777fe640 1 -- 172.21.15.31:0/2813371327 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fd4780d34f0 con 0x7fd4801101e0 2024-09-21T06:11:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:11:34.033 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:11:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:11:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd46c0b6620 msgr2=0x7fd46c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd46c0b6620 0x7fd46c0b8b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd47000a930 tx=0x7fd470005c00 comp rx=0 tx=0).stop 2024-09-21T06:11:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 msgr2=0x7fd4801a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd4801a1f40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd478009dc0 tx=0x7fd478003f10 comp rx=0 tx=0).stop 2024-09-21T06:11:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 shutdown_connections 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd46c0b6620 0x7fd46c0b8b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4801101e0 0x7fd4801a1f40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd480102f80 0x7fd48019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 --2- 172.21.15.31:0/2813371327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd480102630 0x7fd48019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 >> 172.21.15.31:0/2813371327 conn(0x7fd480079730 msgr2=0x7fd480109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 shutdown_connections 2024-09-21T06:11:34.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:34.034+0000 7fd484bb4640 1 -- 172.21.15.31:0/2813371327 wait complete. 2024-09-21T06:11:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.97 s, remaining 2.53 s) 2024-09-21T06:11:34.168 INFO:teuthology.orchestra.run.smithi031.stderr:1500000 of 3200000 tuples (46%) done (elapsed 2.03 s, remaining 2.30 s) 2024-09-21T06:11:34.342 INFO:teuthology.orchestra.run.smithi031.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.20 s, remaining 2.20 s) 2024-09-21T06:11:34.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:34 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2813371327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:34.447 INFO:teuthology.orchestra.run.smithi031.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.31 s, remaining 2.04 s) 2024-09-21T06:11:34.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:34 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2813371327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:34.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:34 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2813371327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:11:34.616 INFO:teuthology.orchestra.run.smithi031.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.48 s, remaining 1.93 s) 2024-09-21T06:11:34.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:11:34.791 INFO:teuthology.orchestra.run.smithi031.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.65 s, remaining 1.81 s) 2024-09-21T06:11:34.888 INFO:teuthology.orchestra.run.smithi031.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.75 s, remaining 1.65 s) 2024-09-21T06:11:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:35.064 INFO:teuthology.orchestra.run.smithi031.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.92 s, remaining 1.53 s) 2024-09-21T06:11:35.327 INFO:teuthology.orchestra.run.smithi031.stderr:2200000 of 3200000 tuples (68%) done (elapsed 3.19 s, remaining 1.45 s) 2024-09-21T06:11:35.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:35 smithi031 ceph-mon[31237]: pgmap v319: 97 pgs: 97 active+clean; 32 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.7 MiB/s wr, 90 op/s 2024-09-21T06:11:35.457 INFO:teuthology.orchestra.run.smithi031.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.32 s, remaining 1.30 s) 2024-09-21T06:11:35.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:35 smithi110 ceph-mon[34550]: pgmap v319: 97 pgs: 97 active+clean; 32 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.7 MiB/s wr, 90 op/s 2024-09-21T06:11:35.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:35 smithi107 ceph-mon[34380]: pgmap v319: 97 pgs: 97 active+clean; 32 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 2.7 MiB/s wr, 90 op/s 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 -- 172.21.15.31:0/2005855538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe298108cf0 msgr2=0x7fe29810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/2005855538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe298108cf0 0x7fe29810f520 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe290009ca0 tx=0x7fe29002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 -- 172.21.15.31:0/2005855538 shutdown_connections 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/2005855538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe298108cf0 0x7fe29810f520 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/2005855538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2981083a0 0x7fe2981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/2005855538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe298102a60 0x7fe298102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 -- 172.21.15.31:0/2005855538 >> 172.21.15.31:0/2005855538 conn(0x7fe298079730 msgr2=0x7fe298079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:35.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 -- 172.21.15.31:0/2005855538 shutdown_connections 2024-09-21T06:11:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.601+0000 7fe29fa8b640 1 -- 172.21.15.31:0/2005855538 wait complete. 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 Processor -- start 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 -- start start 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe298102a60 0x7fe29819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 0x7fe29819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe298108cf0 0x7fe29819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe298112960 con 0x7fe2981083a0 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981127e0 con 0x7fe298102a60 2024-09-21T06:11:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29fa8b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe298112ae0 con 0x7fe298108cf0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29d026640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 0x7fe29819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29d026640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 0x7fe29819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44824/0 (socket says 172.21.15.31:44824) 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29d026640 1 -- 172.21.15.31:0/1145789440 learned_addr learned my addr 172.21.15.31:0/1145789440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.602+0000 7fe29e028640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe298108cf0 0x7fe29819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d827640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe298102a60 0x7fe29819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d026640 1 -- 172.21.15.31:0/1145789440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe298108cf0 msgr2=0x7fe29819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d026640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe298108cf0 0x7fe29819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d026640 1 -- 172.21.15.31:0/1145789440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe298102a60 msgr2=0x7fe29819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d026640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe298102a60 0x7fe29819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d026640 1 -- 172.21.15.31:0/1145789440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2981a6520 con 0x7fe2981083a0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29d026640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 0x7fe29819b750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c970 tx=0x7fe28800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe28effd640 1 -- 172.21.15.31:0/1145789440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288007bb0 con 0x7fe2981083a0 2024-09-21T06:11:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe298199060 con 0x7fe2981083a0 2024-09-21T06:11:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe28effd640 1 -- 172.21.15.31:0/1145789440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe288007d50 con 0x7fe2981083a0 2024-09-21T06:11:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe28effd640 1 -- 172.21.15.31:0/1145789440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f450 con 0x7fe2981083a0 2024-09-21T06:11:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.603+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2981995a0 con 0x7fe2981083a0 2024-09-21T06:11:35.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.606+0000 7fe2a5fff740 1 Processor -- start 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.606+0000 7fe2a5fff740 1 -- start start 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe28effd640 1 -- 172.21.15.31:0/1145789440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fe28800f5f0 con 0x7fe2981083a0 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d493f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d493fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d4940120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c6d48b5ea0 con 0x55c6d456f1b0 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6d48a52d0 con 0x55c6d4577390 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6d48a90e0 con 0x55c6d4598070 2024-09-21T06:11:35.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe29d827640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d493fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe29d827640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d493fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44838/0 (socket says 172.21.15.31:44838) 2024-09-21T06:11:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe29d827640 1 -- 172.21.15.31:0/2861871331 learned_addr learned my addr 172.21.15.31:0/2861871331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe29e028640 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d493f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.607+0000 7fe29d026640 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d4940120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29d827640 1 -- 172.21.15.31:0/2861871331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 msgr2=0x55c6d493f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29d827640 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d493f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29d827640 1 -- 172.21.15.31:0/2861871331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 msgr2=0x55c6d4940120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29d827640 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d4940120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29d827640 1 -- 172.21.15.31:0/2861871331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d4603650 con 0x55c6d456f1b0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe28effd640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2840b65f0 0x7fe2840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29d827640 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d493fbe0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe294009320 tx=0x7fe2940097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a972f480ce5b29a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe29e028640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2840b65f0 0x7fe2840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe28effd640 1 -- 172.21.15.31:0/1145789440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe2880e7210 con 0x7fe2981083a0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe283fff640 1 -- 172.21.15.31:0/2861871331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29401a660 con 0x55c6d456f1b0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe283fff640 1 -- 172.21.15.31:0/2861871331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29401f070 con 0x55c6d456f1b0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.608+0000 7fe283fff640 1 -- 172.21.15.31:0/2861871331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294019b00 con 0x55c6d456f1b0 2024-09-21T06:11:35.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2861871331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 msgr2=0x55c6d493fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe29e028640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2840b65f0 0x7fe2840b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe290009a60 tx=0x7fe29002f450 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d493fbe0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe294009320 tx=0x7fe2940097e0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2861871331 shutdown_connections 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d4940120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d493fbe0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.609+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2861871331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d493f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.610+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2861871331 >> 172.21.15.31:0/2861871331 conn(0x55c6d458c460 msgr2=0x55c6d493d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.610+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2861871331 shutdown_connections 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.610+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2861871331 wait complete. 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.610+0000 7fe2a5fff740 1 -- start start 2024-09-21T06:11:35.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.610+0000 7fe2a5fff740 1 Processor -- start 2024-09-21T06:11:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4577390 0x55c6d494b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d4957170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c6d4706700 con 0x55c6d456f1b0 2024-09-21T06:11:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6d4840160 con 0x55c6d4598070 2024-09-21T06:11:35.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6d494de80 con 0x55c6d4577390 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2a5fff740 10 client.0 _ll_register_callbacks cb 0x55c6d48599e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d827640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d827640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44840/0 (socket says 172.21.15.31:44840) 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d827640 1 -- 172.21.15.31:0/1113009322 learned_addr learned my addr 172.21.15.31:0/1113009322 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4577390 0x55c6d494b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29e028640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d4957170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 -- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 msgr2=0x55c6d4957170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d4957170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 -- 172.21.15.31:0/1113009322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 msgr2=0x55c6d494ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 -- 172.21.15.31:0/1113009322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d486dc90 con 0x55c6d4577390 2024-09-21T06:11:35.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d827640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe29d026640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4577390 0x55c6d494b240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2880e4ae0 tx=0x7fe2880ac840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef6352b70620914 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.611+0000 7fe2817fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2817fa640 1 -- 172.21.15.31:0/1113009322 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2880d0070 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2817fa640 1 -- 172.21.15.31:0/1113009322 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe2880b2dd0 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29401f070 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2817fa640 1 -- 172.21.15.31:0/1113009322 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe28801d070 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 10 client.25207 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c6d4959b90 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2817fa640 1 -- 172.21.15.31:0/1113009322 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29401f070 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2817fa640 1 -- 172.21.15.31:0/1113009322 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe2880b2fd0 con 0x55c6d4577390 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 10 client.25207 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 10 client.25207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:35.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 20 client.25207 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d45569f0 0x55c6d495d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 4 client.25207 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.612+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d48b7df0 con 0x55c6d45569f0 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.613+0000 7fe29d827640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d45569f0 0x55c6d495d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.613+0000 7fe29d827640 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d45569f0 0x55c6d495d070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643d15023b69c7a4 server_cookie=f67a6995e36e1652 in_seq=0 out_seq=0 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.613+0000 7fe2817fa640 10 client.25207 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2817fa640 1 -- 172.21.15.31:0/1113009322 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6d48b7df0 con 0x55c6d45569f0 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2817fa640 10 client.25207 handle_command_reply: tid=0 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 client.25207 shutdown 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4577390 msgr2=0x55c6d494b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4577390 0x55c6d494b240 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe2880e4ae0 tx=0x7fe2880ac840 comp rx=0 tx=0).stop 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 shutdown_connections 2024-09-21T06:11:35.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d45569f0 0x55c6d495d070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d4957170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2817fa640 0 client.25207 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4577390 0x55c6d494b240 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/1113009322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 >> 172.21.15.31:0/1113009322 conn(0x55c6d458c460 msgr2=0x55c6d4942830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 shutdown_connections 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 1 -- 172.21.15.31:0/1113009322 wait complete. 2024-09-21T06:11:35.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.614+0000 7fe2a5fff740 20 client.25207 trim_cache size 0 max 16384 2024-09-21T06:11:35.627 INFO:teuthology.orchestra.run.smithi031.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.49 s, remaining 1.16 s) 2024-09-21T06:11:35.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.627+0000 7fe2a5fff740 1 Processor -- start 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 -- start start 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d456f1b0 0x55c6d494b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 0x55c6d4957070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d493cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981ab4a0 con 0x55c6d4577390 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981add20 con 0x55c6d4598070 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981abb10 con 0x55c6d456f1b0 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 0x55c6d4957070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 0x55c6d4957070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44844/0 (socket says 172.21.15.31:44844) 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d827640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d456f1b0 0x55c6d494b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 -- 172.21.15.31:0/2242460279 learned_addr learned my addr 172.21.15.31:0/2242460279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:35.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29e028640 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d493cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 -- 172.21.15.31:0/2242460279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d456f1b0 msgr2=0x55c6d494b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d456f1b0 0x55c6d494b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 -- 172.21.15.31:0/2242460279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 msgr2=0x55c6d493cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d493cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d026640 1 -- 172.21.15.31:0/2242460279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d4942500 con 0x55c6d4577390 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29e028640 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d493cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.628+0000 7fe29d827640 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d456f1b0 0x55c6d494b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.629+0000 7fe29d026640 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 0x55c6d4957070 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe2880ad1a0 tx=0x7fe28800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77e3c0103359ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.629+0000 7fe2837fe640 1 -- 172.21.15.31:0/2242460279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2880d0070 con 0x55c6d4577390 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.629+0000 7fe2837fe640 1 -- 172.21.15.31:0/2242460279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2880acd20 con 0x55c6d4577390 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.629+0000 7fe2837fe640 1 -- 172.21.15.31:0/2242460279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2880acfe0 con 0x55c6d4577390 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2242460279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 msgr2=0x55c6d4957070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 0x55c6d4957070 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe2880ad1a0 tx=0x7fe28800cf60 comp rx=0 tx=0).stop 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2242460279 shutdown_connections 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4598070 0x55c6d493cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d4577390 0x55c6d4957070 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2242460279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d456f1b0 0x55c6d494b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2242460279 >> 172.21.15.31:0/2242460279 conn(0x55c6d458c460 msgr2=0x55c6d494aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2242460279 shutdown_connections 2024-09-21T06:11:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2242460279 wait complete. 2024-09-21T06:11:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.630+0000 7fe2a5fff740 1 -- start start 2024-09-21T06:11:35.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 Processor -- start 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d494f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d494faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981ac1e0 con 0x55c6d456f1b0 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981ad650 con 0x55c6d4577390 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981ac8b0 con 0x55c6d4598070 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe2a5fff740 10 client.0 _ll_register_callbacks cb 0x7fe28800c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe29d827640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe29d827640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44860/0 (socket says 172.21.15.31:44860) 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe29d827640 1 -- 172.21.15.31:0/2388320631 learned_addr learned my addr 172.21.15.31:0/2388320631 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:35.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe29e028640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d494faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.631+0000 7fe29d026640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d494f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29d827640 1 -- 172.21.15.31:0/2388320631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 msgr2=0x55c6d494faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29d827640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d494faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29d827640 1 -- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 msgr2=0x55c6d494f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29d827640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d494f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29d827640 1 -- 172.21.15.31:0/2388320631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d49c6a30 con 0x55c6d456f1b0 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29e028640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d494faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe29d827640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494f070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe294012010 tx=0x7fe294009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcfc7c0a54a00ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2817fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2817fa640 1 -- 172.21.15.31:0/2388320631 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294019910 con 0x55c6d456f1b0 2024-09-21T06:11:35.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2817fa640 1 -- 172.21.15.31:0/2388320631 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe294019ab0 con 0x55c6d456f1b0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2880acd20 con 0x55c6d456f1b0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2817fa640 1 -- 172.21.15.31:0/2388320631 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe29403e050 con 0x55c6d456f1b0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2a5fff740 10 client.15492 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.632+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c6d49c7bf0 con 0x55c6d456f1b0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2817fa640 1 -- 172.21.15.31:0/2388320631 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2880acd20 con 0x55c6d456f1b0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2817fa640 1 -- 172.21.15.31:0/2388320631 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe294003be0 con 0x55c6d456f1b0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2a5fff740 10 client.15492 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2a5fff740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2a5fff740 20 client.15492 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d452bda0 0x55c6d49ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2a5fff740 4 client.15492 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6d48b38a0 con 0x55c6d452bda0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.633+0000 7fe29d026640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d452bda0 0x55c6d49ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:35.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2817fa640 10 client.15492 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe29d026640 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d452bda0 0x55c6d49ca0a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf225471d12cbe6 server_cookie=b51dfdd933947458 in_seq=0 out_seq=0 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2817fa640 1 -- 172.21.15.31:0/2388320631 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6d48b38a0 con 0x55c6d452bda0 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2817fa640 10 client.15492 handle_command_reply: tid=0 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 client.15492 shutdown 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 msgr2=0x55c6d494f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494f070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe294012010 tx=0x7fe294009d70 comp rx=0 tx=0).stop 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 shutdown_connections 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6d452bda0 0x55c6d49ca0a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6d4598070 0x55c6d494faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6d4577390 0x55c6d494f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 --2- 172.21.15.31:0/2388320631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6d456f1b0 0x55c6d494f070 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2817fa640 0 client.15492 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 >> 172.21.15.31:0/2388320631 conn(0x55c6d458c460 msgr2=0x55c6d4940660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:35.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 shutdown_connections 2024-09-21T06:11:35.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.634+0000 7fe2a5fff740 1 -- 172.21.15.31:0/2388320631 wait complete. 2024-09-21T06:11:35.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe2a5fff740 20 client.15492 trim_cache size 0 max 16384 2024-09-21T06:11:35.641 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2840b65f0 msgr2=0x7fe2840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2840b65f0 0x7fe2840b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe290009a60 tx=0x7fe29002f450 comp rx=0 tx=0).stop 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 msgr2=0x7fe29819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 0x7fe29819b750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c970 tx=0x7fe28800ce30 comp rx=0 tx=0).stop 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 shutdown_connections 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe298108cf0 0x7fe29819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2840b65f0 0x7fe2840b8b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2981083a0 0x7fe29819b750 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 --2- 172.21.15.31:0/1145789440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe298102a60 0x7fe29819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:35.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 >> 172.21.15.31:0/1145789440 conn(0x7fe298079730 msgr2=0x7fe2981043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:35.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 shutdown_connections 2024-09-21T06:11:35.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:35.635+0000 7fe29fa8b640 1 -- 172.21.15.31:0/1145789440 wait complete. 2024-09-21T06:11:35.803 INFO:teuthology.orchestra.run.smithi031.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.66 s, remaining 1.03 s) 2024-09-21T06:11:35.906 INFO:teuthology.orchestra.run.smithi031.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.77 s, remaining 0.87 s) 2024-09-21T06:11:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 -- 172.21.15.31:0/3160261784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c108340 msgr2=0x7fdc0c108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3160261784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c108340 0x7fdc0c108750 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009900 tx=0x7fdc0002a410 comp rx=0 tx=0).stop 2024-09-21T06:11:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 -- 172.21.15.31:0/3160261784 shutdown_connections 2024-09-21T06:11:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3160261784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108c90 0x7fdc0c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3160261784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c108340 0x7fdc0c108750 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3160261784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c102a10 0x7fdc0c102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 -- 172.21.15.31:0/3160261784 >> 172.21.15.31:0/3160261784 conn(0x7fdc0c0fe410 msgr2=0x7fdc0c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 -- 172.21.15.31:0/3160261784 shutdown_connections 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.067+0000 7fdc125cc640 1 -- 172.21.15.31:0/3160261784 wait complete. 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 Processor -- start 2024-09-21T06:11:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 -- start start 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 0x7fdc0c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108340 0x7fdc0c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c108c90 0x7fdc0c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c113a70 con 0x7fdc0c102a10 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1138f0 con 0x7fdc0c108340 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc125cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c113bf0 con 0x7fdc0c108c90 2024-09-21T06:11:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 0x7fdc0c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 0x7fdc0c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44876/0 (socket says 172.21.15.31:44876) 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc0bfff640 1 -- 172.21.15.31:0/3796163960 learned_addr learned my addr 172.21.15.31:0/3796163960 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108340 0x7fdc0c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.068+0000 7fdc10b69640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c108c90 0x7fdc0c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0bfff640 1 -- 172.21.15.31:0/3796163960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c108c90 msgr2=0x7fdc0c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c108c90 0x7fdc0c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0bfff640 1 -- 172.21.15.31:0/3796163960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108340 msgr2=0x7fdc0c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108340 0x7fdc0c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0bfff640 1 -- 172.21.15.31:0/3796163960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c1a0400 con 0x7fdc0c102a10 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc10b69640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c108c90 0x7fdc0c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108340 0x7fdc0c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 0x7fdc0c19a4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e9f0 tx=0x7fdbfc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc097fa640 1 -- 172.21.15.31:0/3796163960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00cd80 con 0x7fdc0c102a10 2024-09-21T06:11:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c1a06f0 con 0x7fdc0c102a10 2024-09-21T06:11:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc097fa640 1 -- 172.21.15.31:0/3796163960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbfc0054d0 con 0x7fdc0c102a10 2024-09-21T06:11:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc097fa640 1 -- 172.21.15.31:0/3796163960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc010690 con 0x7fdc0c102a10 2024-09-21T06:11:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.069+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c1a6e80 con 0x7fdc0c102a10 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 Processor -- start 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 -- start start 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a65071b0 0x55d7a68ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64e64d0 0x55d7a68ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d7a68216f0 con 0x55d7a64de2f0 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7a6810b20 con 0x55d7a65071b0 2024-09-21T06:11:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.072+0000 7fdc19614740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d7a6814930 con 0x55d7a64e64d0 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc10b69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a65071b0 0x55d7a68ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc10b69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a65071b0 0x55d7a68ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37464/0 (socket says 172.21.15.31:37464) 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc10b69640 1 -- 172.21.15.31:0/1110111713 learned_addr learned my addr 172.21.15.31:0/1110111713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64e64d0 0x55d7a68ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/1110111713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a65071b0 msgr2=0x55d7a68ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a65071b0 0x55d7a68ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/1110111713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 msgr2=0x55d7a68acef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68acef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/1110111713 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a67d6530 con 0x55d7a64e64d0 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc097fa640 1 -- 172.21.15.31:0/3796163960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fdbfc004e90 con 0x7fdc0c102a10 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64e64d0 0x55d7a68ad430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc0002a930 tx=0x7fdc00032bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca28d6c555cbc12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.073+0000 7fdbf27fc640 1 -- 172.21.15.31:0/1110111713 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0003f070 con 0x55d7a64e64d0 2024-09-21T06:11:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdbf27fc640 1 -- 172.21.15.31:0/1110111713 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc000316a0 con 0x55d7a64e64d0 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdbf27fc640 1 -- 172.21.15.31:0/1110111713 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00035070 con 0x55d7a64e64d0 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdc097fa640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdbf40b65f0 0x7fdbf40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdc10b69640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdbf40b65f0 0x7fdbf40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdc097fa640 1 -- 172.21.15.31:0/3796163960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdbfc0e7b70 con 0x7fdc0c102a10 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdc19614740 1 -- 172.21.15.31:0/1110111713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64e64d0 msgr2=0x55d7a68ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.074+0000 7fdc19614740 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64e64d0 0x55d7a68ad430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc0002a930 tx=0x7fdc00032bf0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 -- 172.21.15.31:0/1110111713 shutdown_connections 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64e64d0 0x55d7a68ad430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 --2- 172.21.15.31:0/1110111713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a65071b0 0x55d7a68ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 -- 172.21.15.31:0/1110111713 >> 172.21.15.31:0/1110111713 conn(0x55d7a64fb300 msgr2=0x55d7a68aa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 -- 172.21.15.31:0/1110111713 shutdown_connections 2024-09-21T06:11:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 -- 172.21.15.31:0/1110111713 wait complete. 2024-09-21T06:11:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc10b69640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdbf40b65f0 0x7fdbf40b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdc040099a0 tx=0x7fdc04009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 -- start start 2024-09-21T06:11:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.075+0000 7fdc19614740 1 Processor -- start 2024-09-21T06:11:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d7a6748ab0 con 0x55d7a64de2f0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7a67b6730 con 0x55d7a64e64d0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d7a68bf4b0 con 0x55d7a65071b0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc19614740 10 client.0 _ll_register_callbacks cb 0x55d7a680a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44882/0 (socket says 172.21.15.31:44882) 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 -- 172.21.15.31:0/134506810 learned_addr learned my addr 172.21.15.31:0/134506810 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc10b69640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 -- 172.21.15.31:0/134506810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 msgr2=0x55d7a68c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 -- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 msgr2=0x55d7a68bc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 -- 172.21.15.31:0/134506810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a6572790 con 0x55d7a64de2f0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc10b69640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0e5470 tx=0x7fdbfc0e58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214f9e605f0f8d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.076+0000 7fdbeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/134506810 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0b2380 con 0x55d7a64de2f0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/134506810 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdbfc016bb0 con 0x55d7a64de2f0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc000316a0 con 0x55d7a64de2f0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/134506810 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdbfc0ad5d0 con 0x55d7a64de2f0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 10 client.15501 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d7a65eb620 con 0x55d7a64de2f0 2024-09-21T06:11:36.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/134506810 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc000316a0 con 0x55d7a64de2f0 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/134506810 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdbfc0a9410 con 0x55d7a64de2f0 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 10 client.15501 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 20 client.15501 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a64c5d80 0x55d7a68ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 4 client.15501 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.077+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a6823640 con 0x55d7a64c5d80 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.078+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a64c5d80 0x55d7a68ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.078+0000 7fdbeb7fe640 10 client.15501 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.078+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a64c5d80 0x55d7a68ce550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cd40d957872208 server_cookie=b79808f7de8c4933 in_seq=0 out_seq=0 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/134506810 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7a6823640 con 0x55d7a64c5d80 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdbeb7fe640 10 client.15501 handle_command_reply: tid=0 2024-09-21T06:11:36.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 client.15501 shutdown 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 msgr2=0x55d7a68bc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc440 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0e5470 tx=0x7fdbfc0e58c0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 shutdown_connections 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a64c5d80 0x55d7a68ce550 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 --2- 172.21.15.31:0/134506810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc440 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 >> 172.21.15.31:0/134506810 conn(0x55d7a64fb300 msgr2=0x55d7a68a8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdbeb7fe640 0 client.15501 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 shutdown_connections 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 1 -- 172.21.15.31:0/134506810 wait complete. 2024-09-21T06:11:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.079+0000 7fdc19614740 20 client.15501 trim_cache size 0 max 16384 2024-09-21T06:11:36.088 INFO:teuthology.orchestra.run.smithi031.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.94 s, remaining 0.73 s) 2024-09-21T06:11:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 Processor -- start 2024-09-21T06:11:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 -- start start 2024-09-21T06:11:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 0x55d7a68aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1ab390 con 0x55d7a64e64d0 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1adc10 con 0x55d7a64de2f0 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc19614740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1aba00 con 0x55d7a65071b0 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 0x55d7a68aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 0x55d7a68aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44898/0 (socket says 172.21.15.31:44898) 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/2091314466 learned_addr learned my addr 172.21.15.31:0/2091314466 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc10b69640 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/2091314466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 msgr2=0x55d7a68c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/2091314466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64de2f0 msgr2=0x55d7a68bc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.105+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.106+0000 7fdc0b7fe640 1 -- 172.21.15.31:0/2091314466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a64c5fb0 con 0x55d7a64e64d0 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.106+0000 7fdc10b69640 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.106+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 0x55d7a68aa310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc00005940 tx=0x7fdc000317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1730e2eab87aebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.106+0000 7fdbf1ffb640 1 -- 172.21.15.31:0/2091314466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc00035040 con 0x55d7a64e64d0 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.106+0000 7fdbf1ffb640 1 -- 172.21.15.31:0/2091314466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc00031820 con 0x55d7a64e64d0 2024-09-21T06:11:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.106+0000 7fdbf1ffb640 1 -- 172.21.15.31:0/2091314466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0003f040 con 0x55d7a64e64d0 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 -- 172.21.15.31:0/2091314466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 msgr2=0x55d7a68aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 0x55d7a68aa310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdc00005940 tx=0x7fdc000317b0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 -- 172.21.15.31:0/2091314466 shutdown_connections 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a65071b0 0x55d7a68c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a64e64d0 0x55d7a68aa310 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 --2- 172.21.15.31:0/2091314466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64de2f0 0x55d7a68bc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 -- 172.21.15.31:0/2091314466 >> 172.21.15.31:0/2091314466 conn(0x55d7a64fb300 msgr2=0x55d7a68a9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 -- 172.21.15.31:0/2091314466 shutdown_connections 2024-09-21T06:11:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.107+0000 7fdc19614740 1 -- 172.21.15.31:0/2091314466 wait complete. 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 -- start start 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 Processor -- start 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 0x55d7a68bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a65071b0 0x55d7a68c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1ac0d0 con 0x55d7a65071b0 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1ad540 con 0x55d7a64e64d0 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.108+0000 7fdc19614740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c1ac7a0 con 0x55d7a64de2f0 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc19614740 10 client.0 _ll_register_callbacks cb 0x7fdbfc009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 0x55d7a68bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc10b69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a65071b0 0x55d7a68c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 0x55d7a68bf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41474/0 (socket says 172.21.15.31:41474) 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37498/0 (socket says 172.21.15.31:37498) 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc10b69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a65071b0 0x55d7a68c0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44900/0 (socket says 172.21.15.31:44900) 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 -- 172.21.15.31:0/771874193 learned_addr learned my addr 172.21.15.31:0/771874193 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 -- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 msgr2=0x55d7a68bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 -- 172.21.15.31:0/771874193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a65071b0 msgr2=0x55d7a68c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a65071b0 0x55d7a68c0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 -- 172.21.15.31:0/771874193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a69370b0 con 0x55d7a64de2f0 2024-09-21T06:11:36.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdc0bfff640 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 0x55d7a68bf900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ada50 tx=0x7fdbfc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76bfd97e7ff72fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.109+0000 7fdbeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.110+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/771874193 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0a9960 con 0x55d7a64de2f0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.110+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/771874193 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdbfc016db0 con 0x55d7a64de2f0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.110+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc00031820 con 0x55d7a64de2f0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/771874193 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdbfc0b2390 con 0x55d7a64de2f0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 10 client.25231 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d7a6938350 con 0x55d7a64de2f0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/771874193 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc00031820 con 0x55d7a64de2f0 2024-09-21T06:11:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/771874193 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdbfc0ac790 con 0x55d7a64de2f0 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 10 client.25231 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 10 client.25231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 20 client.25231 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a649b1c0 0x55d7a693a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 4 client.25231 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.111+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7a681f0f0 con 0x55d7a649b1c0 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.112+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a649b1c0 0x55d7a693a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.112+0000 7fdbeb7fe640 10 client.25231 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.112+0000 7fdc0b7fe640 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a649b1c0 0x55d7a693a980 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961179ccd26381bc server_cookie=bcc6679c98621936 in_seq=0 out_seq=0 2024-09-21T06:11:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdbeb7fe640 1 -- 172.21.15.31:0/771874193 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7a681f0f0 con 0x55d7a649b1c0 2024-09-21T06:11:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdbeb7fe640 10 client.25231 handle_command_reply: tid=0 2024-09-21T06:11:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 client.25231 shutdown 2024-09-21T06:11:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 msgr2=0x55d7a68bf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 0x55d7a68bf900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ada50 tx=0x7fdbfc009310 comp rx=0 tx=0).stop 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 shutdown_connections 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7a649b1c0 0x55d7a693a980 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7a65071b0 0x55d7a68c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7a64e64d0 0x55d7a68bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 --2- 172.21.15.31:0/771874193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7a64de2f0 0x55d7a68bf900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdbeb7fe640 0 client.25231 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 >> 172.21.15.31:0/771874193 conn(0x55d7a64fb300 msgr2=0x55d7a68ae090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 shutdown_connections 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 1 -- 172.21.15.31:0/771874193 wait complete. 2024-09-21T06:11:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.284+0000 7fdc19614740 20 client.25231 trim_cache size 0 max 16384 2024-09-21T06:11:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:11:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:11:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:11:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdbf40b65f0 msgr2=0x7fdbf40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdbf40b65f0 0x7fdbf40b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdc040099a0 tx=0x7fdc04009500 comp rx=0 tx=0).stop 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 msgr2=0x7fdc0c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 0x7fdc0c19a4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e9f0 tx=0x7fdbfc00edf0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 shutdown_connections 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc0c108c90 0x7fdc0c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc0c108340 0x7fdc0c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdbf40b65f0 0x7fdbf40b8b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 --2- 172.21.15.31:0/3796163960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc0c102a10 0x7fdc0c19a4d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 >> 172.21.15.31:0/3796163960 conn(0x7fdc0c0fe410 msgr2=0x7fdc0c1059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 shutdown_connections 2024-09-21T06:11:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:36.285+0000 7fdc125cc640 1 -- 172.21.15.31:0/3796163960 wait complete. 2024-09-21T06:11:36.361 INFO:teuthology.orchestra.run.smithi031.stderr:2800000 of 3200000 tuples (87%) done (elapsed 4.22 s, remaining 0.60 s) 2024-09-21T06:11:36.422 INFO:teuthology.orchestra.run.smithi031.stderr:2900000 of 3200000 tuples (90%) done (elapsed 4.28 s, remaining 0.44 s) 2024-09-21T06:11:36.631 INFO:teuthology.orchestra.run.smithi031.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.49 s, remaining 0.30 s) 2024-09-21T06:11:36.802 INFO:teuthology.orchestra.run.smithi031.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.66 s, remaining 0.15 s) 2024-09-21T06:11:36.865 INFO:teuthology.orchestra.run.smithi031.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.73 s, remaining 0.00 s) 2024-09-21T06:11:36.907 INFO:teuthology.orchestra.run.smithi031.stderr:vacuuming... 2024-09-21T06:11:37.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:37 smithi031 ceph-mon[31237]: pgmap v320: 97 pgs: 97 active+clean; 165 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 25 MiB/s wr, 164 op/s 2024-09-21T06:11:37.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:37 smithi110 ceph-mon[34550]: pgmap v320: 97 pgs: 97 active+clean; 165 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 25 MiB/s wr, 164 op/s 2024-09-21T06:11:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:37 smithi107 ceph-mon[34380]: pgmap v320: 97 pgs: 97 active+clean; 165 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 25 MiB/s wr, 164 op/s 2024-09-21T06:11:37.552 INFO:teuthology.orchestra.run.smithi031.stderr:creating primary keys... 2024-09-21T06:11:37.567 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:11:37.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:11:37.851 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 -- 172.21.15.31:0/914756040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7810de10 msgr2=0x7f1b78110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 --2- 172.21.15.31:0/914756040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7810de10 0x7f1b78110300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1b70009ca0 tx=0x7f1b7002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 -- 172.21.15.31:0/914756040 shutdown_connections 2024-09-21T06:11:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 --2- 172.21.15.31:0/914756040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b7810de10 0x7f1b78110300 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 --2- 172.21.15.31:0/914756040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78105300 0x7f1b7810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 --2- 172.21.15.31:0/914756040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b781049b0 0x7f1b78104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 -- 172.21.15.31:0/914756040 >> 172.21.15.31:0/914756040 conn(0x7f1b78079730 msgr2=0x7f1b78079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 -- 172.21.15.31:0/914756040 shutdown_connections 2024-09-21T06:11:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.306+0000 7f1b80348640 1 -- 172.21.15.31:0/914756040 wait complete. 2024-09-21T06:11:38.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.307+0000 7f1b80348640 1 Processor -- start 2024-09-21T06:11:38.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.307+0000 7f1b80348640 1 -- start start 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.307+0000 7f1b80348640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b781049b0 0x7f1b7819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b80348640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 0x7f1b7819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b80348640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b7810de10 0x7f1b7819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b80348640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78113ae0 con 0x7f1b781049b0 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b80348640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78113960 con 0x7f1b78105300 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b80348640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b78113c60 con 0x7f1b7810de10 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7e0e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b781049b0 0x7f1b7819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 0x7f1b7819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 0x7f1b7819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37510/0 (socket says 172.21.15.31:37510) 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7e0e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b781049b0 0x7f1b7819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44906/0 (socket says 172.21.15.31:44906) 2024-09-21T06:11:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/1718890516 learned_addr learned my addr 172.21.15.31:0/1718890516 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b7810de10 0x7f1b7819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/1718890516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b7810de10 msgr2=0x7f1b7819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b7810de10 0x7f1b7819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b781049b0 msgr2=0x7f1b7819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b781049b0 0x7f1b7819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/1718890516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b781a03e0 con 0x7f1b78105300 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.308+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 0x7f1b7819aa10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9c0 tx=0x7f1b6800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.309+0000 7f1b6f7fe640 1 -- 172.21.15.31:0/1718890516 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800ccc0 con 0x7f1b78105300 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.309+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b781a06d0 con 0x7f1b78105300 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.309+0000 7f1b6f7fe640 1 -- 172.21.15.31:0/1718890516 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b680054d0 con 0x7f1b78105300 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.309+0000 7f1b6f7fe640 1 -- 172.21.15.31:0/1718890516 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68010430 con 0x7f1b78105300 2024-09-21T06:11:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.309+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b781a6e30 con 0x7f1b78105300 2024-09-21T06:11:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.311+0000 7f1b868bc740 1 Processor -- start 2024-09-21T06:11:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 -- start start 2024-09-21T06:11:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b6f7fe640 1 -- 172.21.15.31:0/1718890516 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f1b680105d0 con 0x7f1b78105300 2024-09-21T06:11:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d6067350 0x5568d640e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 0x5568d640e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d640eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5568d6384e60 con 0x5568d6046670 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568d6374290 con 0x5568d6067350 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b868bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5568d63780a0 con 0x5568d603e490 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b7e0e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 0x5568d640e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b7e8e5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d6067350 0x5568d640e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b7e0e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 0x5568d640e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41494/0 (socket says 172.21.15.31:41494) 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.312+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/220567599 learned_addr learned my addr 172.21.15.31:0/220567599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d640eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/220567599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d6067350 msgr2=0x5568d640e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d6067350 0x5568d640e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/220567599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 msgr2=0x5568d640eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d640eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/220567599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d60d2930 con 0x5568d603e490 2024-09-21T06:11:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d640eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 0x5568d640e970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400bce0 tx=0x7f1b740107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d640a57983606de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.313+0000 7f1b6cd9a640 1 -- 172.21.15.31:0/220567599 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004020 con 0x5568d603e490 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.314+0000 7f1b6cd9a640 1 -- 172.21.15.31:0/220567599 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b740054d0 con 0x5568d603e490 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.314+0000 7f1b6cd9a640 1 -- 172.21.15.31:0/220567599 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004cb0 con 0x5568d603e490 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.314+0000 7f1b6f7fe640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b640b65f0 0x7f1b640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.314+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b640b65f0 0x7f1b640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.314+0000 7f1b6f7fe640 1 -- 172.21.15.31:0/1718890516 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1b680e7960 con 0x7f1b78105300 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b640b65f0 0x7f1b640b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006010 tx=0x7f1b70005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 -- 172.21.15.31:0/220567599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 msgr2=0x5568d640e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 0x5568d640e970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400bce0 tx=0x7f1b740107a0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 -- 172.21.15.31:0/220567599 shutdown_connections 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d640eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d603e490 0x5568d640e970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 --2- 172.21.15.31:0/220567599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d6067350 0x5568d640e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 -- 172.21.15.31:0/220567599 >> 172.21.15.31:0/220567599 conn(0x5568d605b740 msgr2=0x5568d640c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 -- 172.21.15.31:0/220567599 shutdown_connections 2024-09-21T06:11:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.315+0000 7f1b868bc740 1 -- 172.21.15.31:0/220567599 wait complete. 2024-09-21T06:11:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.316+0000 7f1b868bc740 1 -- start start 2024-09-21T06:11:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.316+0000 7f1b868bc740 1 Processor -- start 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.316+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d641e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.316+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 0x5568d642a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b868bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5568d61d5890 con 0x5568d6067350 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b868bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568d630ed40 con 0x5568d603e490 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b868bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5568d6420f40 con 0x5568d6046670 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 0x5568d642a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b868bc740 10 client.0 _ll_register_callbacks cb 0x5568d6372c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 0x5568d642a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44924/0 (socket says 172.21.15.31:44924) 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 -- 172.21.15.31:0/1901190266 learned_addr learned my addr 172.21.15.31:0/1901190266 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d641e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 -- 172.21.15.31:0/1901190266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 msgr2=0x5568d641e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 -- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 msgr2=0x5568d641e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d641e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e8e5640 1 -- 172.21.15.31:0/1901190266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d642a800 con 0x5568d6067350 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.317+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d641e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 0x5568d642a270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b7002f540 tx=0x7f1b7002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22980bcdcdfefdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:38.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b61ffb640 1 -- 172.21.15.31:0/1901190266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7002f610 con 0x5568d6067350 2024-09-21T06:11:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b61ffb640 1 -- 172.21.15.31:0/1901190266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1b7002f7b0 con 0x5568d6067350 2024-09-21T06:11:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b740054d0 con 0x5568d6067350 2024-09-21T06:11:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b61ffb640 1 -- 172.21.15.31:0/1901190266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1b7002f9b0 con 0x5568d6067350 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.318+0000 7f1b868bc740 10 client.15519 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5568d642d110 con 0x5568d6067350 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b61ffb640 1 -- 172.21.15.31:0/1901190266 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b70036050 con 0x5568d6067350 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b61ffb640 1 -- 172.21.15.31:0/1901190266 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1b70051070 con 0x5568d6067350 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 10 client.15519 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 20 client.15519 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6025cd0 0x5568d6430600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 4 client.15519 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:38.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.319+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b740054d0 con 0x5568d6025cd0 2024-09-21T06:11:38.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.320+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6025cd0 0x5568d6430600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.321+0000 7f1b61ffb640 10 client.15519 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:38.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.321+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6025cd0 0x5568d6430600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee4a31aed8f5ebd server_cookie=af38706207d56340 in_seq=0 out_seq=0 2024-09-21T06:11:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b61ffb640 1 -- 172.21.15.31:0/1901190266 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1b740054d0 con 0x5568d6025cd0 2024-09-21T06:11:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b61ffb640 10 client.15519 handle_command_reply: tid=0 2024-09-21T06:11:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 client.15519 shutdown 2024-09-21T06:11:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 msgr2=0x5568d642a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 0x5568d642a270 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1b7002f540 tx=0x7f1b7002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 shutdown_connections 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6025cd0 0x5568d6430600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b61ffb640 0 client.15519 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6067350 0x5568d642a270 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 --2- 172.21.15.31:0/1901190266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d641e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.322+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 >> 172.21.15.31:0/1901190266 conn(0x5568d605b740 msgr2=0x5568d64115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.323+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 shutdown_connections 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.323+0000 7f1b868bc740 1 -- 172.21.15.31:0/1901190266 wait complete. 2024-09-21T06:11:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.323+0000 7f1b868bc740 20 client.15519 trim_cache size 0 max 16384 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 Processor -- start 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 -- start start 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d605b740 0x5568d642a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 0x5568d641b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ab460 con 0x5568d603e490 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781adce0 con 0x5568d605b740 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b868bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781abad0 con 0x5568d6046670 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b7d8e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 0x5568d641b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b7d8e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 0x5568d641b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44926/0 (socket says 172.21.15.31:44926) 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/3908889661 learned_addr learned my addr 172.21.15.31:0/3908889661 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d605b740 0x5568d642a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.346+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/3908889661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 msgr2=0x5568d641b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/3908889661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d605b740 msgr2=0x5568d642a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d605b740 0x5568d642a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b7d8e3640 1 -- 172.21.15.31:0/3908889661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d6411230 con 0x5568d603e490 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 0x5568d641b3a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1b680a9e80 tx=0x7f1b680a9750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fd692f4a141bc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b63fff640 1 -- 172.21.15.31:0/3908889661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b680ad020 con 0x5568d603e490 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b63fff640 1 -- 172.21.15.31:0/3908889661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b680ad3c0 con 0x5568d603e490 2024-09-21T06:11:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.347+0000 7f1b63fff640 1 -- 172.21.15.31:0/3908889661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800ce60 con 0x5568d603e490 2024-09-21T06:11:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 -- 172.21.15.31:0/3908889661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 msgr2=0x5568d641b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 0x5568d641b3a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1b680a9e80 tx=0x7f1b680a9750 comp rx=0 tx=0).stop 2024-09-21T06:11:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 -- 172.21.15.31:0/3908889661 shutdown_connections 2024-09-21T06:11:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d6046670 0x5568d641b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d603e490 0x5568d641b3a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3908889661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d605b740 0x5568d642a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 -- 172.21.15.31:0/3908889661 >> 172.21.15.31:0/3908889661 conn(0x5568d6419b00 msgr2=0x5568d641ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 -- 172.21.15.31:0/3908889661 shutdown_connections 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.348+0000 7f1b868bc740 1 -- 172.21.15.31:0/3908889661 wait complete. 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 -- start start 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 Processor -- start 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d6426aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d6422f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d605b740 0x5568d64235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ac1a0 con 0x5568d6046670 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ad610 con 0x5568d603e490 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b781ac870 con 0x5568d605b740 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b868bc740 10 client.0 _ll_register_callbacks cb 0x7f1b70009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b7d8e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d6422f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b7e0e4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d6426aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b7e0e4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d6426aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37544/0 (socket says 172.21.15.31:37544) 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b7e8e5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d605b740 0x5568d64235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/3095939029 learned_addr learned my addr 172.21.15.31:0/3095939029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.349+0000 7f1b7d8e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d6422f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44942/0 (socket says 172.21.15.31:44942) 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/3095939029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d605b740 msgr2=0x5568d64235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d605b740 0x5568d64235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/3095939029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 msgr2=0x5568d6422f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d6422f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e0e4640 1 -- 172.21.15.31:0/3095939029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d6423d20 con 0x5568d603e490 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d6422f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e8e5640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d605b740 0x5568d64235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b7e0e4640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d6426aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400ba40 tx=0x7f1b7400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7c580ed10485a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b61ffb640 1 -- 172.21.15.31:0/3095939029 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74033060 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b61ffb640 1 -- 172.21.15.31:0/3095939029 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1b74011640 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b680ad3c0 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b61ffb640 1 -- 172.21.15.31:0/3095939029 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1b74011840 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b868bc740 10 client.25235 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.350+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5568d649c090 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b61ffb640 1 -- 172.21.15.31:0/3095939029 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b680ad3c0 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b61ffb640 1 -- 172.21.15.31:0/3095939029 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1b74038970 con 0x5568d603e490 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b868bc740 10 client.25235 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b868bc740 10 client.25235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b868bc740 20 client.25235 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6067350 0x5568d649e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b868bc740 4 client.25235 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:11:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5568d6386db0 con 0x5568d6067350 2024-09-21T06:11:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.351+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6067350 0x5568d649e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.352+0000 7f1b7d8e3640 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6067350 0x5568d649e5a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0565c2c8a241c05 server_cookie=cfab920a5356fc0b in_seq=0 out_seq=0 2024-09-21T06:11:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.352+0000 7f1b61ffb640 10 client.25235 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b61ffb640 1 -- 172.21.15.31:0/3095939029 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5568d6386db0 con 0x5568d6067350 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b61ffb640 10 client.25235 handle_command_reply: tid=0 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 client.25235 shutdown 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 msgr2=0x5568d6426aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d6426aa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400ba40 tx=0x7f1b7400b8e0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 shutdown_connections 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568d6067350 0x5568d649e5a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568d605b740 0x5568d64235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b61ffb640 0 client.25235 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568d6046670 0x5568d6422f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 --2- 172.21.15.31:0/3095939029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568d603e490 0x5568d6426aa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.353+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 >> 172.21.15.31:0/3095939029 conn(0x5568d6419b00 msgr2=0x5568d6411580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 shutdown_connections 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b868bc740 1 -- 172.21.15.31:0/3095939029 wait complete. 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b868bc740 20 client.25235 trim_cache size 0 max 16384 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46", 2024-09-21T06:11:38.359 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b640b65f0 msgr2=0x7f1b640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b640b65f0 0x7f1b640b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006010 tx=0x7f1b70005d80 comp rx=0 tx=0).stop 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 msgr2=0x7f1b7819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 0x7f1b7819aa10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9c0 tx=0x7f1b6800ee80 comp rx=0 tx=0).stop 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 shutdown_connections 2024-09-21T06:11:38.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b7810de10 0x7f1b7819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b78105300 0x7f1b7819aa10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b640b65f0 0x7f1b640b8b00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 --2- 172.21.15.31:0/1718890516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b781049b0 0x7f1b7819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 >> 172.21.15.31:0/1718890516 conn(0x7f1b78079730 msgr2=0x7f1b780ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 shutdown_connections 2024-09-21T06:11:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:38.354+0000 7f1b80348640 1 -- 172.21.15.31:0/1718890516 wait complete. 2024-09-21T06:11:39.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:11:39.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:39 smithi031 ceph-mon[31237]: pgmap v321: 97 pgs: 97 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 36 MiB/s wr, 128 op/s 2024-09-21T06:11:39.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:39 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:11:39.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:39 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:39.242 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:39 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:11:39.491 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:11:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:39 smithi110 ceph-mon[34550]: pgmap v321: 97 pgs: 97 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 36 MiB/s wr, 128 op/s 2024-09-21T06:11:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:39 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:11:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:39 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:39 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:11:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:39 smithi107 ceph-mon[34380]: pgmap v321: 97 pgs: 97 active+clean; 237 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 36 MiB/s wr, 128 op/s 2024-09-21T06:11:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:39 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:11:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:39 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:39 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:11:39.724 INFO:teuthology.orchestra.run.smithi031.stderr:done in 7.62 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.79 s, vacuum 0.64 s, primary keys 2.17 s). 2024-09-21T06:11:39.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-21T06:11:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 -- 172.21.15.31:0/470846737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4103b60 msgr2=0x7f2fc4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 --2- 172.21.15.31:0/470846737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4103b60 0x7f2fc4103f70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009900 tx=0x7f2fb802a410 comp rx=0 tx=0).stop 2024-09-21T06:11:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 -- 172.21.15.31:0/470846737 shutdown_connections 2024-09-21T06:11:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 --2- 172.21.15.31:0/470846737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4109df0 0x7f2fc4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 --2- 172.21.15.31:0/470846737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 0x7f2fc41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 --2- 172.21.15.31:0/470846737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4103b60 0x7f2fc4103f70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 -- 172.21.15.31:0/470846737 >> 172.21.15.31:0/470846737 conn(0x7f2fc407b850 msgr2=0x7f2fc407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 -- 172.21.15.31:0/470846737 shutdown_connections 2024-09-21T06:11:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.983+0000 7f2fcacac640 1 -- 172.21.15.31:0/470846737 wait complete. 2024-09-21T06:11:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.984+0000 7f2fcacac640 1 Processor -- start 2024-09-21T06:11:40.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.984+0000 7f2fcacac640 1 -- start start 2024-09-21T06:11:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fcacac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4103b60 0x7f2fc419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fcacac640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 0x7f2fc419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fcacac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 0x7f2fc419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fcacac640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc4113ab0 con 0x7f2fc4109df0 2024-09-21T06:11:40.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fcacac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc4113930 con 0x7f2fc4103b60 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fcacac640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc4113c30 con 0x7f2fc41094a0 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc9249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 0x7f2fc419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4103b60 0x7f2fc419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 0x7f2fc419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc9249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 0x7f2fc419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53226/0 (socket says 172.21.15.31:53226) 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc9249640 1 -- 172.21.15.31:0/2985109906 learned_addr learned my addr 172.21.15.31:0/2985109906 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 -- 172.21.15.31:0/2985109906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 msgr2=0x7f2fc419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 0x7f2fc419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 -- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 msgr2=0x7f2fc419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 0x7f2fc419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 -- 172.21.15.31:0/2985109906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc41a0410 con 0x7f2fc4103b60 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc9249640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 0x7f2fc419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 0x7f2fc419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.985+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4103b60 0x7f2fc419a4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fb802a930 tx=0x7f2fb8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.986+0000 7f2fc1ffb640 1 -- 172.21.15.31:0/2985109906 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb803f030 con 0x7f2fc4103b60 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.986+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc41a06a0 con 0x7f2fc4103b60 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.986+0000 7f2fc1ffb640 1 -- 172.21.15.31:0/2985109906 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb8004490 con 0x7f2fc4103b60 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.986+0000 7f2fc1ffb640 1 -- 172.21.15.31:0/2985109906 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8035070 con 0x7f2fc4103b60 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.986+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc41a6e60 con 0x7f2fc4103b60 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.988+0000 7f2fd1220740 1 Processor -- start 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 -- start start 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 0x55e918cb6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188e61b0 0x55e918cb6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188ee390 0x55e918cb70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc1ffb640 1 -- 172.21.15.31:0/2985109906 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f2fb8042030 con 0x7f2fc4103b60 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e918c2ce50 con 0x55e91890f070 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e918c1c280 con 0x55e9188ee390 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fd1220740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e918c20090 con 0x55e9188e61b0 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc9249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 0x55e918cb6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc9249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 0x55e918cb6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53228/0 (socket says 172.21.15.31:53228) 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc9249640 1 -- 172.21.15.31:0/3212168848 learned_addr learned my addr 172.21.15.31:0/3212168848 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188ee390 0x55e918cb70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188e61b0 0x55e918cb6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc9249640 1 -- 172.21.15.31:0/3212168848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188e61b0 msgr2=0x55e918cb6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.989+0000 7f2fc9249640 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188e61b0 0x55e918cb6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc9249640 1 -- 172.21.15.31:0/3212168848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188ee390 msgr2=0x55e918cb70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc9249640 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188ee390 0x55e918cb70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc9249640 1 -- 172.21.15.31:0/3212168848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91897a650 con 0x55e91890f070 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188e61b0 0x55e918cb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc9249640 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 0x55e918cb6650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0118b0 tx=0x7f2fbc011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86700ac4b1e7ffbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2faeffd640 1 -- 172.21.15.31:0/3212168848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc01e070 con 0x55e91890f070 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2faeffd640 1 -- 172.21.15.31:0/3212168848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fbc00fe10 con 0x55e91890f070 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2faeffd640 1 -- 172.21.15.31:0/3212168848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc019d40 con 0x55e91890f070 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc1ffb640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2fb40b65f0 0x7f2fb40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2fb40b65f0 0x7f2fb40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.990+0000 7f2fc1ffb640 1 -- 172.21.15.31:0/2985109906 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2fb8107880 con 0x7f2fc4103b60 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 -- 172.21.15.31:0/3212168848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 msgr2=0x55e918cb6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fc8a48640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2fb40b65f0 0x7f2fb40b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80caa90 tx=0x7f2fb8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 0x55e918cb6650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0118b0 tx=0x7f2fbc011d70 comp rx=0 tx=0).stop 2024-09-21T06:11:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 -- 172.21.15.31:0/3212168848 shutdown_connections 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188ee390 0x55e918cb70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188e61b0 0x55e918cb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 --2- 172.21.15.31:0/3212168848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e91890f070 0x55e918cb6650 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 -- 172.21.15.31:0/3212168848 >> 172.21.15.31:0/3212168848 conn(0x55e918903460 msgr2=0x55e918cb4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 -- 172.21.15.31:0/3212168848 shutdown_connections 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 -- 172.21.15.31:0/3212168848 wait complete. 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.991+0000 7f2fd1220740 1 -- start start 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 Processor -- start 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918cc5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cc6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e918a7d700 con 0x55e9188ee390 2024-09-21T06:11:40.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e918bb7110 con 0x55e9188e61b0 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e918cc9190 con 0x55e91890f070 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fd1220740 10 client.0 _ll_register_callbacks cb 0x55e918bd0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cc6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53238/0 (socket says 172.21.15.31:53238) 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc3fff640 1 -- 172.21.15.31:0/8179513 learned_addr learned my addr 172.21.15.31:0/8179513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc9249640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918cc5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc9249640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cc6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc3fff640 1 -- 172.21.15.31:0/8179513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 msgr2=0x55e918cc6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.992+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cc6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fc3fff640 1 -- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 msgr2=0x55e918cc5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918cc5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fc3fff640 1 -- 172.21.15.31:0/8179513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e918be4c40 con 0x55e9188ee390 2024-09-21T06:11:40.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cc6530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012d20 tx=0x7f2fb000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7f47071225dd34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fa3fff640 1 -- 172.21.15.31:0/8179513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0012110 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fa3fff640 1 -- 172.21.15.31:0/8179513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2fb0004050 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fbc00fe10 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fa3fff640 1 -- 172.21.15.31:0/8179513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2fb00122b0 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fd1220740 10 client.15537 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e918cd4e10 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.993+0000 7f2fa3fff640 1 -- 172.21.15.31:0/8179513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fbc00fe10 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fa3fff640 1 -- 172.21.15.31:0/8179513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2fb0025440 con 0x55e9188ee390 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fd1220740 10 client.15537 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fd1220740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fd1220740 20 client.15537 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:40.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fd1220740 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188cd9f0 0x55e918cd82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fd1220740 4 client.15537 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e918c2eda0 con 0x55e9188cd9f0 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fc9249640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188cd9f0 0x55e918cd82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fa3fff640 10 client.15537 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.994+0000 7f2fc9249640 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188cd9f0 0x55e918cd82e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8783bee796ca8c8 server_cookie=249066687625bc83 in_seq=0 out_seq=0 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fa3fff640 1 -- 172.21.15.31:0/8179513 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e918c2eda0 con 0x55e9188cd9f0 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fa3fff640 10 client.15537 handle_command_reply: tid=0 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fd1220740 1 client.15537 shutdown 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 msgr2=0x55e918cc6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fd1220740 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cc6530 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012d20 tx=0x7f2fb000f790 comp rx=0 tx=0).stop 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 shutdown_connections 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fd1220740 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188cd9f0 0x55e918cd82e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.997+0000 7f2fd1220740 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cc6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fd1220740 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cc6530 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fd1220740 1 --2- 172.21.15.31:0/8179513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918cc5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 >> 172.21.15.31:0/8179513 conn(0x55e918903460 msgr2=0x55e918cb8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fa3fff640 0 client.15537 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 shutdown_connections 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fd1220740 1 -- 172.21.15.31:0/8179513 wait complete. 2024-09-21T06:11:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:39.998+0000 7f2fd1220740 20 client.15537 trim_cache size 0 max 16384 2024-09-21T06:11:40.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 Processor -- start 2024-09-21T06:11:40.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 -- start start 2024-09-21T06:11:40.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 0x55e918cc17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188ee390 0x55e918cd1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e91890f070 0x55e918cc31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41ab3e0 con 0x55e9188e61b0 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41adc60 con 0x55e91890f070 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fd1220740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41aba50 con 0x55e9188ee390 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fc9249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 0x55e918cc17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fc3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188ee390 0x55e918cd1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fc9249640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 0x55e918cc17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53244/0 (socket says 172.21.15.31:53244) 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.017+0000 7f2fc9249640 1 -- 172.21.15.31:0/4138937263 learned_addr learned my addr 172.21.15.31:0/4138937263 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e91890f070 0x55e918cc31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 -- 172.21.15.31:0/4138937263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188ee390 msgr2=0x55e918cd1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188ee390 0x55e918cd1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 -- 172.21.15.31:0/4138937263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e91890f070 msgr2=0x55e918cc31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e91890f070 0x55e918cc31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 -- 172.21.15.31:0/4138937263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e918cc7090 con 0x55e9188e61b0 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fc9249640 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 0x55e918cc17a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc019950 tx=0x7f2fbc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df1ce26c465a7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fae7fc640 1 -- 172.21.15.31:0/4138937263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc035070 con 0x55e9188e61b0 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fae7fc640 1 -- 172.21.15.31:0/4138937263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fbc00f7b0 con 0x55e9188e61b0 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.018+0000 7f2fae7fc640 1 -- 172.21.15.31:0/4138937263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc01e070 con 0x55e9188e61b0 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 -- 172.21.15.31:0/4138937263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 msgr2=0x55e918cc17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 0x55e918cc17a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc019950 tx=0x7f2fbc009500 comp rx=0 tx=0).stop 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 -- 172.21.15.31:0/4138937263 shutdown_connections 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e91890f070 0x55e918cc31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9188ee390 0x55e918cd1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 --2- 172.21.15.31:0/4138937263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188e61b0 0x55e918cc17a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 -- 172.21.15.31:0/4138937263 >> 172.21.15.31:0/4138937263 conn(0x55e918903460 msgr2=0x55e918cd13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 -- 172.21.15.31:0/4138937263 shutdown_connections 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.019+0000 7f2fd1220740 1 -- 172.21.15.31:0/4138937263 wait complete. 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 -- start start 2024-09-21T06:11:40.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 Processor -- start 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918ccee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41ac120 con 0x55e9188ee390 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41ad590 con 0x55e9188e61b0 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc41ac7f0 con 0x55e91890f070 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fd1220740 10 client.0 _ll_register_callbacks cb 0x7f2fb0012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fc3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fc3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53256/0 (socket says 172.21.15.31:53256) 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fc3fff640 1 -- 172.21.15.31:0/1761074351 learned_addr learned my addr 172.21.15.31:0/1761074351 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.020+0000 7f2fc9249640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918ccee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc9249640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc3fff640 1 -- 172.21.15.31:0/1761074351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 msgr2=0x55e918cca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc3fff640 1 -- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 msgr2=0x55e918ccee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918ccee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc3fff640 1 -- 172.21.15.31:0/1761074351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e918d41ca0 con 0x55e9188ee390 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fc3fff640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cca2c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012cf0 tx=0x7f2fb0025c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e98e897f73b61c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fa3fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fa3fff640 1 -- 172.21.15.31:0/1761074351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb002b040 con 0x55e9188ee390 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fa3fff640 1 -- 172.21.15.31:0/1761074351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2fb0021030 con 0x55e9188ee390 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fbc00f7b0 con 0x55e9188ee390 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fa3fff640 1 -- 172.21.15.31:0/1761074351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2fb0012110 con 0x55e9188ee390 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fd1220740 10 client.15543 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.021+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e918d42e00 con 0x55e9188ee390 2024-09-21T06:11:40.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fa3fff640 1 -- 172.21.15.31:0/1761074351 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fbc00f7b0 con 0x55e9188ee390 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fa3fff640 1 -- 172.21.15.31:0/1761074351 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2fb00038c0 con 0x55e9188ee390 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fd1220740 10 client.15543 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fd1220740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fd1220740 20 client.15543 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fd1220740 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188a2da0 0x55e918d455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fd1220740 4 client.15543 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e918c2a850 con 0x55e9188a2da0 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fc9249640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188a2da0 0x55e918d455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fa3fff640 10 client.15543 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:40.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.022+0000 7f2fc9249640 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188a2da0 0x55e918d455e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5206f8a0c73f9c47 server_cookie=a440f60a38b0f290 in_seq=0 out_seq=0 2024-09-21T06:11:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fa3fff640 1 -- 172.21.15.31:0/1761074351 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1145 (crc 0 0 0) 0x55e918c2a850 con 0x55e9188a2da0 2024-09-21T06:11:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fa3fff640 10 client.15543 handle_command_reply: tid=0 2024-09-21T06:11:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 client.15543 shutdown 2024-09-21T06:11:40.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 msgr2=0x55e918cca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cca2c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0012cf0 tx=0x7f2fb0025c90 comp rx=0 tx=0).stop 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 shutdown_connections 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9188a2da0 0x55e918d455e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e91890f070 0x55e918cca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9188ee390 0x55e918cca2c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 --2- 172.21.15.31:0/1761074351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9188e61b0 0x55e918ccee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 >> 172.21.15.31:0/1761074351 conn(0x55e918903460 msgr2=0x55e918cb7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fa3fff640 0 client.15543 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 shutdown_connections 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 1 -- 172.21.15.31:0/1761074351 wait complete. 2024-09-21T06:11:40.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.026+0000 7f2fd1220740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (31 inodes in the stack)", 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46", 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:11:40.034 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:11:40.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:11:40.036 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2fb40b65f0 msgr2=0x7f2fb40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2fb40b65f0 0x7f2fb40b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80caa90 tx=0x7f2fb8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4103b60 msgr2=0x7f2fc419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4103b60 0x7f2fc419a4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fb802a930 tx=0x7f2fb8031a20 comp rx=0 tx=0).stop 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 shutdown_connections 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2fb40b65f0 0x7f2fb40b8b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fc4109df0 0x7f2fc419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2fc41094a0 0x7f2fc419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 --2- 172.21.15.31:0/2985109906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc4103b60 0x7f2fc419a4a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:40.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 >> 172.21.15.31:0/2985109906 conn(0x7f2fc407b850 msgr2=0x7f2fc4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:40.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 shutdown_connections 2024-09-21T06:11:40.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:11:40.027+0000 7f2fcacac640 1 -- 172.21.15.31:0/2985109906 wait complete. 2024-09-21T06:11:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:starting vacuum...end. 2024-09-21T06:11:40.935 INFO:tasks.cephfs.filesystem:scrub status for tag:1b0e6866-bf74-4daf-b0bc-b4a3ee098d46 - {'path': '/', 'tag': '1b0e6866-bf74-4daf-b0bc-b4a3ee098d46', 'options': 'recursive,force'} 2024-09-21T06:11:41.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:41 smithi031 ceph-mon[31237]: pgmap v322: 97 pgs: 97 active+clean; 314 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 48 MiB/s wr, 234 op/s 2024-09-21T06:11:41.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:41 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:41.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:41 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:41 smithi110 ceph-mon[34550]: pgmap v322: 97 pgs: 97 active+clean; 314 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 48 MiB/s wr, 234 op/s 2024-09-21T06:11:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:41 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:41 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:41 smithi107 ceph-mon[34380]: pgmap v322: 97 pgs: 97 active+clean; 314 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 48 MiB/s wr, 234 op/s 2024-09-21T06:11:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:41 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:11:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:41 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:11:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:42 smithi110 ceph-mon[34550]: pgmap v323: 97 pgs: 97 active+clean; 487 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 73 MiB/s wr, 460 op/s 2024-09-21T06:11:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:42 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:11:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:42 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:42 smithi107 ceph-mon[34380]: pgmap v323: 97 pgs: 97 active+clean; 487 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 73 MiB/s wr, 460 op/s 2024-09-21T06:11:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:42 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:11:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:42 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:42 smithi031 ceph-mon[31237]: pgmap v323: 97 pgs: 97 active+clean; 487 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 73 MiB/s wr, 460 op/s 2024-09-21T06:11:43.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:42 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:11:43.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:43.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:42 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 5.0 s, 5307.9 tps, lat 18.496 ms stddev 21.426 2024-09-21T06:11:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:45 smithi110 ceph-mon[34550]: pgmap v324: 97 pgs: 97 active+clean; 487 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 71 MiB/s wr, 425 op/s 2024-09-21T06:11:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:45.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:45 smithi107 ceph-mon[34380]: pgmap v324: 97 pgs: 97 active+clean; 487 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 71 MiB/s wr, 425 op/s 2024-09-21T06:11:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:45 smithi031 ceph-mon[31237]: pgmap v324: 97 pgs: 97 active+clean; 487 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 71 MiB/s wr, 425 op/s 2024-09-21T06:11:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:11:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:11:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:11:47.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:47 smithi110 ceph-mon[34550]: pgmap v325: 97 pgs: 97 active+clean; 809 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 76 MiB/s wr, 698 op/s 2024-09-21T06:11:47.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:47 smithi031 ceph-mon[31237]: pgmap v325: 97 pgs: 97 active+clean; 809 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 76 MiB/s wr, 698 op/s 2024-09-21T06:11:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:47 smithi107 ceph-mon[34380]: pgmap v325: 97 pgs: 97 active+clean; 809 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 76 MiB/s wr, 698 op/s 2024-09-21T06:11:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.458+0000 7f25df007640 1 -- 172.21.15.31:0/3660011712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d810a0e0 msgr2=0x7f25d810a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.458+0000 7f25df007640 1 --2- 172.21.15.31:0/3660011712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d810a0e0 0x7f25d810a4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009900 tx=0x7f25cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.459+0000 7f25df007640 1 -- 172.21.15.31:0/3660011712 shutdown_connections 2024-09-21T06:11:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.459+0000 7f25df007640 1 --2- 172.21.15.31:0/3660011712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d8110360 0x7f25d8116b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.459+0000 7f25df007640 1 --2- 172.21.15.31:0/3660011712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 0x7f25d810fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.459+0000 7f25df007640 1 --2- 172.21.15.31:0/3660011712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d810a0e0 0x7f25d810a4f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.459+0000 7f25df007640 1 -- 172.21.15.31:0/3660011712 >> 172.21.15.31:0/3660011712 conn(0x7f25d8079630 msgr2=0x7f25d81081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.460+0000 7f25df007640 1 -- 172.21.15.31:0/3660011712 shutdown_connections 2024-09-21T06:11:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.460+0000 7f25df007640 1 -- 172.21.15.31:0/3660011712 wait complete. 2024-09-21T06:11:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.461+0000 7f25df007640 1 Processor -- start 2024-09-21T06:11:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.461+0000 7f25df007640 1 -- start start 2024-09-21T06:11:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25df007640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 0x7f25d8083b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25dcda3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 0x7f25d8083b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25dcda3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 0x7f25d8083b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51208/0 (socket says 172.21.15.31:51208) 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25df007640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 0x7f25d807ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25dcda3640 1 -- 172.21.15.31:0/1344099464 learned_addr learned my addr 172.21.15.31:0/1344099464 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25df007640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8110360 0x7f25d807d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25df007640 1 -- 172.21.15.31:0/1344099464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25d811a020 con 0x7f25d810fa10 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25df007640 1 -- 172.21.15.31:0/1344099464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25d8119ea0 con 0x7f25d810a0e0 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25df007640 1 -- 172.21.15.31:0/1344099464 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25d811a1a0 con 0x7f25d8110360 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.462+0000 7f25d7fff640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 0x7f25d807ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8110360 0x7f25d807d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dcda3640 1 -- 172.21.15.31:0/1344099464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8110360 msgr2=0x7f25d807d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dcda3640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8110360 0x7f25d807d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dcda3640 1 -- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 msgr2=0x7f25d807ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dcda3640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 0x7f25d807ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dcda3640 1 -- 172.21.15.31:0/1344099464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d807da70 con 0x7f25d810a0e0 2024-09-21T06:11:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8110360 0x7f25d807d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25d7fff640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 0x7f25d807ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:11:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.463+0000 7f25dcda3640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 0x7f25d8083b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009850 tx=0x7f25cc030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.464+0000 7f25d5ffb640 1 -- 172.21.15.31:0/1344099464 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc03f070 con 0x7f25d810a0e0 2024-09-21T06:11:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.464+0000 7f25d5ffb640 1 -- 172.21.15.31:0/1344099464 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25cc004460 con 0x7f25d810a0e0 2024-09-21T06:11:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.464+0000 7f25d5ffb640 1 -- 172.21.15.31:0/1344099464 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25cc035030 con 0x7f25d810a0e0 2024-09-21T06:11:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.464+0000 7f25df007640 1 -- 172.21.15.31:0/1344099464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d80827a0 con 0x7f25d810a0e0 2024-09-21T06:11:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.464+0000 7f25df007640 1 -- 172.21.15.31:0/1344099464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d8082c40 con 0x7f25d810a0e0 2024-09-21T06:11:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.468+0000 7f25e604d740 1 Processor -- start 2024-09-21T06:11:49.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.468+0000 7f25e604d740 1 -- start start 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.468+0000 7f25e604d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc973246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25e604d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc97321ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25e604d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc97322260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25e604d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dc972966f0 con 0x55dc96f5b4d0 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc973246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25e604d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc97285b20 con 0x55dc96f7c1b0 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc973246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51222/0 (socket says 172.21.15.31:51222) 2024-09-21T06:11:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25dd5a4640 1 -- 172.21.15.31:0/618634933 learned_addr learned my addr 172.21.15.31:0/618634933 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25e604d740 1 -- 172.21.15.31:0/618634933 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc97289930 con 0x55dc96f532f0 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.469+0000 7f25d7fff640 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc97322260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.470+0000 7f25dd5a4640 1 -- 172.21.15.31:0/618634933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 msgr2=0x55dc97321ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.470+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc97321ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.470+0000 7f25dd5a4640 1 -- 172.21.15.31:0/618634933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 msgr2=0x55dc97322260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.470+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc97322260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.470+0000 7f25dd5a4640 1 -- 172.21.15.31:0/618634933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9724b530 con 0x55dc96f7c1b0 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.473+0000 7f25d5ffb640 1 -- 172.21.15.31:0/1344099464 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f25cc042070 con 0x7f25d810a0e0 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.473+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc973246a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009fd0 tx=0x7f25d0032870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e14c69253a1d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.475+0000 7f25df007640 1 -- 172.21.15.31:0/618634933 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0033710 con 0x55dc96f7c1b0 2024-09-21T06:11:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.476+0000 7f25df007640 1 -- 172.21.15.31:0/618634933 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0033d60 con 0x55dc96f7c1b0 2024-09-21T06:11:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.476+0000 7f25d5ffb640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f25c80b66f0 0x7f25c80b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25d5ffb640 1 -- 172.21.15.31:0/1344099464 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f25cc107f60 con 0x7f25d810a0e0 2024-09-21T06:11:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 -- 172.21.15.31:0/618634933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 msgr2=0x55dc973246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc973246a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009fd0 tx=0x7f25d0032870 comp rx=0 tx=0).stop 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 -- 172.21.15.31:0/618634933 shutdown_connections 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc97322260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc97321ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 --2- 172.21.15.31:0/618634933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc973246a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.477+0000 7f25e604d740 1 -- 172.21.15.31:0/618634933 >> 172.21.15.31:0/618634933 conn(0x55dc96f70300 msgr2=0x55dc9731fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.478+0000 7f25dcda3640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f25c80b66f0 0x7f25c80b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.480+0000 7f25dcda3640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f25c80b66f0 0x7f25c80b8c00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0cdde0 tx=0x7f25cc02a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.485+0000 7f25e604d740 1 -- 172.21.15.31:0/618634933 shutdown_connections 2024-09-21T06:11:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.485+0000 7f25e604d740 1 -- 172.21.15.31:0/618634933 wait complete. 2024-09-21T06:11:49.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.485+0000 7f25e604d740 1 -- start start 2024-09-21T06:11:49.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.486+0000 7f25e604d740 1 Processor -- start 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.490+0000 7f25e604d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc9732d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.490+0000 7f25e604d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9732d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.490+0000 7f25e604d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9732dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.490+0000 7f25e604d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dc971bdab0 con 0x55dc96f5b4d0 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.490+0000 7f25e604d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc9722b730 con 0x55dc96f7c1b0 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.490+0000 7f25e604d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc97330190 con 0x55dc96f532f0 2024-09-21T06:11:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.491+0000 7f25e604d740 10 client.0 _ll_register_callbacks cb 0x55dc9727f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.492+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc9732d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.492+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9732dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.492+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc9732d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:53700/0 (socket says 172.21.15.31:53700) 2024-09-21T06:11:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.492+0000 7f25dd5a4640 1 -- 172.21.15.31:0/246880032 learned_addr learned my addr 172.21.15.31:0/246880032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:49.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.492+0000 7f25d7fff640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9732d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25dd5a4640 1 -- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 msgr2=0x55dc9732dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9732dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25dd5a4640 1 -- 172.21.15.31:0/246880032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 msgr2=0x55dc9732d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9732d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25dd5a4640 1 -- 172.21.15.31:0/246880032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc96fe7790 con 0x55dc96f532f0 2024-09-21T06:11:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25d7fff640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9732d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:11:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc9732d2d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005bf0 tx=0x7f25d0005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=297f00637be9b520 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.493+0000 7f25b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:11:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.494+0000 7f25b7fff640 1 -- 172.21.15.31:0/246880032 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0045030 con 0x55dc96f532f0 2024-09-21T06:11:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.494+0000 7f25b7fff640 1 -- 172.21.15.31:0/246880032 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f25d0030a60 con 0x55dc96f532f0 2024-09-21T06:11:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.494+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc97298640 con 0x55dc96f532f0 2024-09-21T06:11:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.495+0000 7f25b7fff640 1 -- 172.21.15.31:0/246880032 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f25d0054050 con 0x55dc96f532f0 2024-09-21T06:11:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.495+0000 7f25e604d740 10 client.25276 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.495+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dc97060620 con 0x55dc96f532f0 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25b7fff640 1 -- 172.21.15.31:0/246880032 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dc97298640 con 0x55dc96f532f0 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25b7fff640 1 -- 172.21.15.31:0/246880032 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f25d00339e0 con 0x55dc96f532f0 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25e604d740 10 client.25276 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25e604d740 10 client.25276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25e604d740 20 client.25276 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25e604d740 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f3ad80 0x55dc9733f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25e604d740 4 client.25276 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:11:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.496+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc972940f0 con 0x55dc96f3ad80 2024-09-21T06:11:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.497+0000 7f25d7fff640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f3ad80 0x55dc9733f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.498+0000 7f25b7fff640 10 client.25276 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.498+0000 7f25d7fff640 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f3ad80 0x55dc9733f1a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f019b3108e1f4bd server_cookie=c1114427a4e14072 in_seq=0 out_seq=0 2024-09-21T06:11:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.499+0000 7f25b7fff640 1 -- 172.21.15.31:0/246880032 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc972940f0 con 0x55dc96f3ad80 2024-09-21T06:11:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.499+0000 7f25b7fff640 10 client.25276 handle_command_reply: tid=0 2024-09-21T06:11:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.499+0000 7f25e604d740 1 client.25276 shutdown 2024-09-21T06:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 msgr2=0x55dc9732d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc9732d2d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005bf0 tx=0x7f25d0005cf0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 shutdown_connections 2024-09-21T06:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f3ad80 0x55dc9733f1a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9732dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9732d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 --2- 172.21.15.31:0/246880032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f532f0 0x55dc9732d2d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 >> 172.21.15.31:0/246880032 conn(0x55dc96f70300 msgr2=0x55dc97323690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25b7fff640 0 client.25276 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 shutdown_connections 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.500+0000 7f25e604d740 1 -- 172.21.15.31:0/246880032 wait complete. 2024-09-21T06:11:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.501+0000 7f25e604d740 20 client.25276 trim_cache size 0 max 16384 2024-09-21T06:11:49.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:49 smithi110 ceph-mon[34550]: pgmap v326: 97 pgs: 97 active+clean; 884 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 55 MiB/s wr, 701 op/s 2024-09-21T06:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.534+0000 7f25e604d740 1 Processor -- start 2024-09-21T06:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.535+0000 7f25e604d740 1 -- start start 2024-09-21T06:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.535+0000 7f25e604d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 0x55dc9731f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.535+0000 7f25e604d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9731f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 0x55dc9731f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 0x55dc9731f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51242/0 (socket says 172.21.15.31:51242) 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25dd5a4640 1 -- 172.21.15.31:0/740567260 learned_addr learned my addr 172.21.15.31:0/740567260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25d7fff640 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9731f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25e604d740 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f7c1b0 0x55dc973421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81a55b0 con 0x55dc96f5b4d0 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81a7e30 con 0x55dc96f532f0 2024-09-21T06:11:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.536+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81a5c20 con 0x55dc96f7c1b0 2024-09-21T06:11:49.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:49 smithi107 ceph-mon[34380]: pgmap v326: 97 pgs: 97 active+clean; 884 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 55 MiB/s wr, 701 op/s 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f7c1b0 0x55dc973421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 -- 172.21.15.31:0/740567260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f7c1b0 msgr2=0x55dc973421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f7c1b0 0x55dc973421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 -- 172.21.15.31:0/740567260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 msgr2=0x55dc9731f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9731f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 -- 172.21.15.31:0/740567260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9732ffc0 con 0x55dc96f532f0 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 0x55dc9731f160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25d0002230 tx=0x7f25d0035040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1800ebe76a6d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.537+0000 7f25c27fc640 1 -- 172.21.15.31:0/740567260 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0045070 con 0x55dc96f532f0 2024-09-21T06:11:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.538+0000 7f25c27fc640 1 -- 172.21.15.31:0/740567260 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0060070 con 0x55dc96f532f0 2024-09-21T06:11:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.538+0000 7f25c27fc640 1 -- 172.21.15.31:0/740567260 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0048040 con 0x55dc96f532f0 2024-09-21T06:11:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.539+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 msgr2=0x55dc9731f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.539+0000 7f25e604d740 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 0x55dc9731f160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25d0002230 tx=0x7f25d0035040 comp rx=0 tx=0).stop 2024-09-21T06:11:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 shutdown_connections 2024-09-21T06:11:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f7c1b0 0x55dc973421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f5b4d0 0x55dc9731f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 --2- 172.21.15.31:0/740567260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f532f0 0x55dc9731f160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 >> 172.21.15.31:0/740567260 conn(0x55dc96f70300 msgr2=0x55dc9731a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 shutdown_connections 2024-09-21T06:11:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.540+0000 7f25e604d740 1 -- 172.21.15.31:0/740567260 wait complete. 2024-09-21T06:11:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.541+0000 7f25e604d740 1 -- start start 2024-09-21T06:11:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.541+0000 7f25e604d740 1 Processor -- start 2024-09-21T06:11:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.542+0000 7f25e604d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f532f0 0x55dc9733f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.542+0000 7f25e604d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f5b4d0 0x55dc9733f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.542+0000 7f25e604d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9733fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.542+0000 7f25e604d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81a62f0 con 0x55dc96f532f0 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.542+0000 7f25e604d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81a7760 con 0x55dc96f7c1b0 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.542+0000 7f25e604d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25d81a69c0 con 0x55dc96f5b4d0 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.543+0000 7f25e604d740 10 client.0 _ll_register_callbacks cb 0x7f25c4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.543+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f532f0 0x55dc9733f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.543+0000 7f25dd5a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f532f0 0x55dc9733f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39148/0 (socket says 172.21.15.31:39148) 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.543+0000 7f25dd5a4640 1 -- 172.21.15.31:0/1787945443 learned_addr learned my addr 172.21.15.31:0/1787945443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:11:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.543+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9733fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25dd5a4640 1 -- 172.21.15.31:0/1787945443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f5b4d0 msgr2=0x55dc9733f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f5b4d0 0x55dc9733f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25dd5a4640 1 -- 172.21.15.31:0/1787945443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f532f0 msgr2=0x55dc9733f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f532f0 0x55dc9733f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25dd5a4640 1 -- 172.21.15.31:0/1787945443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc973a7a40 con 0x55dc96f7c1b0 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25dd5a4640 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9733fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55dc97289990 tx=0x7f25d0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4934f90144e760a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:11:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.544+0000 7f25b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.545+0000 7f25b7fff640 1 -- 172.21.15.31:0/1787945443 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0045070 con 0x55dc96f7c1b0 2024-09-21T06:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.545+0000 7f25b7fff640 1 -- 172.21.15.31:0/1787945443 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f25d0058800 con 0x55dc96f7c1b0 2024-09-21T06:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.545+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d0060070 con 0x55dc96f7c1b0 2024-09-21T06:11:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.545+0000 7f25b7fff640 1 -- 172.21.15.31:0/1787945443 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f25d0005bf0 con 0x55dc96f7c1b0 2024-09-21T06:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.546+0000 7f25e604d740 10 client.25271 fetch_fsmap learned FSMap version 19 2024-09-21T06:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.546+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dc973a8cd0 con 0x55dc96f7c1b0 2024-09-21T06:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.546+0000 7f25b7fff640 1 -- 172.21.15.31:0/1787945443 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0032d70 con 0x55dc96f7c1b0 2024-09-21T06:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.546+0000 7f25b7fff640 1 -- 172.21.15.31:0/1787945443 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f25d0033ac0 con 0x55dc96f7c1b0 2024-09-21T06:11:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.547+0000 7f25e604d740 10 client.25271 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.547+0000 7f25e604d740 10 client.25271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.547+0000 7f25e604d740 20 client.25271 populate_metadata read hostname 'smithi031' 2024-09-21T06:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.547+0000 7f25e604d740 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f101c0 0x55dc973ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.547+0000 7f25e604d740 4 client.25271 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:11:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.547+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f25d0060070 con 0x55dc96f101c0 2024-09-21T06:11:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.549+0000 7f25d7fff640 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f101c0 0x55dc973ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:11:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.549+0000 7f25d7fff640 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f101c0 0x55dc973ab300 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e64fa26d534a1e1 server_cookie=3f222e3f3d8ca0ea in_seq=0 out_seq=0 2024-09-21T06:11:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.550+0000 7f25b7fff640 10 client.25271 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:49.611 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.609+0000 7f25b7fff640 1 -- 172.21.15.31:0/1787945443 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f25d0060070 con 0x55dc96f101c0 2024-09-21T06:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.609+0000 7f25b7fff640 10 client.25271 handle_command_reply: tid=0 2024-09-21T06:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.609+0000 7f25e604d740 1 client.25271 shutdown 2024-09-21T06:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.610+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 msgr2=0x55dc9733fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.612 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.610+0000 7f25e604d740 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9733fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55dc97289990 tx=0x7f25d0000f40 comp rx=0 tx=0).stop 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.610+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 shutdown_connections 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.610+0000 7f25e604d740 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc96f101c0 0x55dc973ab300 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.611+0000 7f25e604d740 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc96f7c1b0 0x55dc9733fd50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.611+0000 7f25b7fff640 0 client.25271 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.611+0000 7f25e604d740 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc96f5b4d0 0x55dc9733f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.611+0000 7f25e604d740 1 --2- 172.21.15.31:0/1787945443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc96f532f0 0x55dc9733f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.613 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.611+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 >> 172.21.15.31:0/1787945443 conn(0x55dc96f70300 msgr2=0x55dc973384d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.612+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 shutdown_connections 2024-09-21T06:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.612+0000 7f25e604d740 1 -- 172.21.15.31:0/1787945443 wait complete. 2024-09-21T06:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.612+0000 7f25e604d740 20 client.25271 trim_cache size 0 max 16384 2024-09-21T06:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:11:49.614 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:11:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.614+0000 7f25c37fe640 1 -- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f25c80b66f0 msgr2=0x7f25c80b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.614+0000 7f25c37fe640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f25c80b66f0 0x7f25c80b8c00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0cdde0 tx=0x7f25cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.614+0000 7f25c37fe640 1 -- 172.21.15.31:0/1344099464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 msgr2=0x7f25d8083b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:11:49.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.614+0000 7f25c37fe640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 0x7f25d8083b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009850 tx=0x7f25cc030650 comp rx=0 tx=0).stop 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 -- 172.21.15.31:0/1344099464 shutdown_connections 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25d8110360 0x7f25d807d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f25c80b66f0 0x7f25c80b8c00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25d810fa10 0x7f25d807ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 --2- 172.21.15.31:0/1344099464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25d810a0e0 0x7f25d8083b40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 -- 172.21.15.31:0/1344099464 >> 172.21.15.31:0/1344099464 conn(0x7f25d8079630 msgr2=0x7f25d8110a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:11:49.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 -- 172.21.15.31:0/1344099464 shutdown_connections 2024-09-21T06:11:49.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:11:49.615+0000 7f25c37fe640 1 -- 172.21.15.31:0/1344099464 wait complete. 2024-09-21T06:11:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:49 smithi031 ceph-mon[31237]: pgmap v326: 97 pgs: 97 active+clean; 884 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 55 MiB/s wr, 701 op/s 2024-09-21T06:11:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 10.0 s, 5790.1 tps, lat 17.271 ms stddev 18.025 2024-09-21T06:11:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:51 smithi110 ceph-mon[34550]: pgmap v327: 97 pgs: 97 active+clean; 895 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 45 MiB/s wr, 775 op/s 2024-09-21T06:11:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:51 smithi107 ceph-mon[34380]: pgmap v327: 97 pgs: 97 active+clean; 895 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 45 MiB/s wr, 775 op/s 2024-09-21T06:11:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:51 smithi031 ceph-mon[31237]: pgmap v327: 97 pgs: 97 active+clean; 895 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 45 MiB/s wr, 775 op/s 2024-09-21T06:11:53.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:53 smithi110 ceph-mon[34550]: pgmap v328: 97 pgs: 97 active+clean; 928 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 37 MiB/s wr, 990 op/s 2024-09-21T06:11:53.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:53 smithi107 ceph-mon[34380]: pgmap v328: 97 pgs: 97 active+clean; 928 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 37 MiB/s wr, 990 op/s 2024-09-21T06:11:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:53 smithi031 ceph-mon[31237]: pgmap v328: 97 pgs: 97 active+clean; 928 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 37 MiB/s wr, 990 op/s 2024-09-21T06:11:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 15.0 s, 5688.8 tps, lat 17.611 ms stddev 18.330 2024-09-21T06:11:55.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:55 smithi110 ceph-mon[34550]: pgmap v329: 97 pgs: 97 active+clean; 928 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 12 MiB/s wr, 757 op/s 2024-09-21T06:11:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:55 smithi107 ceph-mon[34380]: pgmap v329: 97 pgs: 97 active+clean; 928 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 12 MiB/s wr, 757 op/s 2024-09-21T06:11:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:55 smithi031 ceph-mon[31237]: pgmap v329: 97 pgs: 97 active+clean; 928 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 12 MiB/s wr, 757 op/s 2024-09-21T06:11:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:57 smithi110 ceph-mon[34550]: pgmap v330: 97 pgs: 97 active+clean; 949 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-21T06:11:57.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:57 smithi107 ceph-mon[34380]: pgmap v330: 97 pgs: 97 active+clean; 949 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-21T06:11:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:57 smithi031 ceph-mon[31237]: pgmap v330: 97 pgs: 97 active+clean; 949 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-21T06:11:59.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:11:59 smithi110 ceph-mon[34550]: pgmap v331: 97 pgs: 97 active+clean; 952 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 12 MiB/s wr, 849 op/s 2024-09-21T06:11:59.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:11:59 smithi107 ceph-mon[34380]: pgmap v331: 97 pgs: 97 active+clean; 952 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 12 MiB/s wr, 849 op/s 2024-09-21T06:11:59.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:11:59 smithi031 ceph-mon[31237]: pgmap v331: 97 pgs: 97 active+clean; 952 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 12 MiB/s wr, 849 op/s 2024-09-21T06:12:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 20.0 s, 5909.3 tps, lat 16.629 ms stddev 17.256 2024-09-21T06:12:00.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:12:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:12:00.059+0000 7f24b931f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:00.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:12:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:12:00.059+0000 7f24b931f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:00.386 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:12:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:12:00.059+0000 7f24b931f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:00.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:12:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:12:00.059+0000 7f24b931f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:00.387 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:12:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:12:00.059+0000 7f24b931f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:12:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:01 smithi110 ceph-mon[34550]: pgmap v332: 97 pgs: 97 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 858 op/s 2024-09-21T06:12:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:01 smithi107 ceph-mon[34380]: pgmap v332: 97 pgs: 97 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 858 op/s 2024-09-21T06:12:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:01 smithi031 ceph-mon[31237]: pgmap v332: 97 pgs: 97 active+clean; 958 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 858 op/s 2024-09-21T06:12:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 -- 172.21.15.31:0/3951733089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06a350 msgr2=0x7fec9c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 --2- 172.21.15.31:0/3951733089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06a350 0x7fec9c102180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a410 comp rx=0 tx=0).stop 2024-09-21T06:12:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 -- 172.21.15.31:0/3951733089 shutdown_connections 2024-09-21T06:12:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 --2- 172.21.15.31:0/3951733089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec9c102ad0 0x7fec9c107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 --2- 172.21.15.31:0/3951733089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c1026c0 0x7fec9c107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 --2- 172.21.15.31:0/3951733089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c06a350 0x7fec9c102180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.750+0000 7feca38f2640 1 -- 172.21.15.31:0/3951733089 >> 172.21.15.31:0/3951733089 conn(0x7fec9c079e90 msgr2=0x7fec9c07a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.751+0000 7feca38f2640 1 -- 172.21.15.31:0/3951733089 shutdown_connections 2024-09-21T06:12:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.751+0000 7feca38f2640 1 -- 172.21.15.31:0/3951733089 wait complete. 2024-09-21T06:12:02.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.752+0000 7feca38f2640 1 Processor -- start 2024-09-21T06:12:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 -- start start 2024-09-21T06:12:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 0x7fec9c074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c1026c0 0x7fec9c074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec9c06dd20 0x7fec9c06e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10f8f0 con 0x7fec9c06dd20 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10f770 con 0x7fec9c06a350 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca38f2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec9c10fa70 con 0x7fec9c1026c0 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca168e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 0x7fec9c074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.753+0000 7feca168e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 0x7fec9c074a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33232/0 (socket says 172.21.15.31:33232) 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca168e640 1 -- 172.21.15.31:0/3454311377 learned_addr learned my addr 172.21.15.31:0/3454311377 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c1026c0 0x7fec9c074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca168e640 1 -- 172.21.15.31:0/3454311377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c1026c0 msgr2=0x7fec9c074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca168e640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c1026c0 0x7fec9c074fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca168e640 1 -- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec9c06dd20 msgr2=0x7fec9c06e220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:12:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca168e640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec9c06dd20 0x7fec9c06e220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.754+0000 7feca168e640 1 -- 172.21.15.31:0/3454311377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9c06ea40 con 0x7fec9c06a350 2024-09-21T06:12:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.756+0000 7feca168e640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 0x7fec9c074a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec98005ab0 tx=0x7fec98003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.757+0000 7fec927fc640 1 -- 172.21.15.31:0/3454311377 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9803f070 con 0x7fec9c06a350 2024-09-21T06:12:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.757+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9c073670 con 0x7fec9c06a350 2024-09-21T06:12:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.757+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec9c073bc0 con 0x7fec9c06a350 2024-09-21T06:12:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.757+0000 7fec927fc640 1 -- 172.21.15.31:0/3454311377 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec98031b50 con 0x7fec9c06a350 2024-09-21T06:12:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.757+0000 7fec927fc640 1 -- 172.21.15.31:0/3454311377 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98035030 con 0x7fec9c06a350 2024-09-21T06:12:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.762+0000 7fecaa861740 1 Processor -- start 2024-09-21T06:12:02.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.762+0000 7fecaa861740 1 -- start start 2024-09-21T06:12:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7fecaa861740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7fecaa861740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7feca1e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7feca1e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33244/0 (socket says 172.21.15.31:33244) 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7feca1e8f640 1 -- 172.21.15.31:0/2148997640 learned_addr learned my addr 172.21.15.31:0/2148997640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7fecaa861740 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555fc11176f0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555fc1106b20 con 0x555fc0dfd1b0 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.763+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555fc110a930 con 0x555fc0dd42f0 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.764+0000 7feca168e640 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.764+0000 7feca1e8f640 1 -- 172.21.15.31:0/2148997640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 msgr2=0x555fc11a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.764+0000 7feca1e8f640 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.764+0000 7feca1e8f640 1 -- 172.21.15.31:0/2148997640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 msgr2=0x555fc11a3430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:12:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.764+0000 7feca1e8f640 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11a3430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.764+0000 7feca1e8f640 1 -- 172.21.15.31:0/2148997640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc10cc530 con 0x555fc0dfd1b0 2024-09-21T06:12:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.765+0000 7fec927fc640 1 -- 172.21.15.31:0/3454311377 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fec98042070 con 0x7fec9c06a350 2024-09-21T06:12:02.768 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.765+0000 7feca1e8f640 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11a29b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec94009d30 tx=0x7fec9400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e48fcfe1accc62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.765+0000 7fec877fe640 1 -- 172.21.15.31:0/2148997640 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400bd40 con 0x555fc0dfd1b0 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.765+0000 7fec877fe640 1 -- 172.21.15.31:0/2148997640 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec94017070 con 0x555fc0dfd1b0 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.766+0000 7fec877fe640 1 -- 172.21.15.31:0/2148997640 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940115d0 con 0x555fc0dfd1b0 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.766+0000 7fec927fc640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec880b68c0 0x7fec880b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7feca168e640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec880b68c0 0x7fec880b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 msgr2=0x555fc11a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.769 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11a29b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec94009d30 tx=0x7fec9400b420 comp rx=0 tx=0).stop 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fec927fc640 1 -- 172.21.15.31:0/3454311377 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fec98107bf0 con 0x7fec9c06a350 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7feca168e640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec880b68c0 0x7fec880b8dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x555fc11a1630 tx=0x7fec9810b7c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 shutdown_connections 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 --2- 172.21.15.31:0/2148997640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11a29b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 >> 172.21.15.31:0/2148997640 conn(0x555fc0df1300 msgr2=0x555fc11a08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 shutdown_connections 2024-09-21T06:12:02.770 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.767+0000 7fecaa861740 1 -- 172.21.15.31:0/2148997640 wait complete. 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.768+0000 7fecaa861740 1 -- start start 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.768+0000 7fecaa861740 1 Processor -- start 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555fc103eab0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555fc10ac730 con 0x555fc0dfd1b0 2024-09-21T06:12:02.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555fc11b1130 con 0x555fc0dd42f0 2024-09-21T06:12:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7fecaa861740 10 client.0 _ll_register_callbacks cb 0x555fc11006b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7feca1e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7feca1e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11ba4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33258/0 (socket says 172.21.15.31:33258) 2024-09-21T06:12:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7feca1e8f640 1 -- 172.21.15.31:0/3510994242 learned_addr learned my addr 172.21.15.31:0/3510994242 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.769+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.772 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.770+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.770+0000 7feca0e8d640 1 -- 172.21.15.31:0/3510994242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 msgr2=0x555fc11ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.770+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.770+0000 7feca0e8d640 1 -- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 msgr2=0x555fc11ba4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.770+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11ba4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.770+0000 7feca0e8d640 1 -- 172.21.15.31:0/3510994242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc0e68790 con 0x555fc0ddc4d0 2024-09-21T06:12:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.771+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae640 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0756a0 tx=0x7fec8c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bd73a986f47794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.771+0000 7fec84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:12:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.772+0000 7fec84ff9640 1 -- 172.21.15.31:0/3510994242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c011640 con 0x555fc0ddc4d0 2024-09-21T06:12:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.772+0000 7fec84ff9640 1 -- 172.21.15.31:0/3510994242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fec8c0117e0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.772+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec94017070 con 0x555fc0ddc4d0 2024-09-21T06:12:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.773+0000 7fec84ff9640 1 -- 172.21.15.31:0/3510994242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fec8c0119e0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.773+0000 7fecaa861740 10 client.15567 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.773+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555fc0ee1620 con 0x555fc0ddc4d0 2024-09-21T06:12:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.773+0000 7fec84ff9640 1 -- 172.21.15.31:0/3510994242 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec94017070 con 0x555fc0ddc4d0 2024-09-21T06:12:02.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fec84ff9640 1 -- 172.21.15.31:0/3510994242 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fec8c023cc0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fecaa861740 10 client.15567 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fecaa861740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fecaa861740 20 client.15567 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fecaa861740 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0dbbd80 0x555fc11c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fecaa861740 4 client.15567 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fc1119640 con 0x555fc0dbbd80 2024-09-21T06:12:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.774+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0dbbd80 0x555fc11c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.775+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0dbbd80 0x555fc11c0370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95d0be3f1fe8b5b server_cookie=da44eb495c561020 in_seq=0 out_seq=0 2024-09-21T06:12:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.775+0000 7fec84ff9640 10 client.15567 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.776+0000 7fec84ff9640 1 -- 172.21.15.31:0/3510994242 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fc1119640 con 0x555fc0dbbd80 2024-09-21T06:12:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.776+0000 7fec84ff9640 10 client.15567 handle_command_reply: tid=0 2024-09-21T06:12:02.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.776+0000 7fecaa861740 1 client.15567 shutdown 2024-09-21T06:12:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.777+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 msgr2=0x555fc11ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.777+0000 7fecaa861740 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae640 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fec9c0756a0 tx=0x7fec8c00f730 comp rx=0 tx=0).stop 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 shutdown_connections 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fecaa861740 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0dbbd80 0x555fc11c0370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fecaa861740 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11ba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fecaa861740 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae640 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fec84ff9640 0 client.15567 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fecaa861740 1 --2- 172.21.15.31:0/3510994242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.778+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 >> 172.21.15.31:0/3510994242 conn(0x555fc0df1300 msgr2=0x555fc11a39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.779+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 shutdown_connections 2024-09-21T06:12:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.779+0000 7fecaa861740 1 -- 172.21.15.31:0/3510994242 wait complete. 2024-09-21T06:12:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.779+0000 7fecaa861740 20 client.15567 trim_cache size 0 max 16384 2024-09-21T06:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.809+0000 7fecaa861740 1 Processor -- start 2024-09-21T06:12:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.809+0000 7fecaa861740 1 -- start start 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7fecaa861740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7fecaa861740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7fecaa861740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7fecaa861740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec8000c6f0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7fecaa861740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec8000ef70 con 0x555fc0dfd1b0 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7fecaa861740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec8000cd60 con 0x555fc0dd42f0 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7feca1e8f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7feca1e8f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59772/0 (socket says 172.21.15.31:59772) 2024-09-21T06:12:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7feca1e8f640 1 -- 172.21.15.31:0/764713163 learned_addr learned my addr 172.21.15.31:0/764713163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.810+0000 7feca1e8f640 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7feca1e8f640 1 -- 172.21.15.31:0/764713163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 msgr2=0x555fc11afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7feca1e8f640 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7feca1e8f640 1 -- 172.21.15.31:0/764713163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 msgr2=0x555fc11ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7feca1e8f640 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7feca1e8f640 1 -- 172.21.15.31:0/764713163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc11a57c0 con 0x555fc0dd42f0 2024-09-21T06:12:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7feca1e8f640 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11acfd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec94012e80 tx=0x7fec94012dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05c5fdc1fc501de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7fec86ffd640 1 -- 172.21.15.31:0/764713163 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94009990 con 0x555fc0dd42f0 2024-09-21T06:12:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7fec86ffd640 1 -- 172.21.15.31:0/764713163 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9402c070 con 0x555fc0dd42f0 2024-09-21T06:12:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.811+0000 7fec86ffd640 1 -- 172.21.15.31:0/764713163 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94011e10 con 0x555fc0dd42f0 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 -- 172.21.15.31:0/764713163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 msgr2=0x555fc11acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11acfd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec94012e80 tx=0x7fec94012dd0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 -- 172.21.15.31:0/764713163 shutdown_connections 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0ddc4d0 0x555fc11ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 --2- 172.21.15.31:0/764713163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0dd42f0 0x555fc11acfd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 -- 172.21.15.31:0/764713163 >> 172.21.15.31:0/764713163 conn(0x555fc0df1300 msgr2=0x555fc11af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 -- 172.21.15.31:0/764713163 shutdown_connections 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.812+0000 7fecaa861740 1 -- 172.21.15.31:0/764713163 wait complete. 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.813+0000 7fecaa861740 1 -- start start 2024-09-21T06:12:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.813+0000 7fecaa861740 1 Processor -- start 2024-09-21T06:12:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.814+0000 7fecaa861740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0dd42f0 0x555fc1222f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.814+0000 7fecaa861740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 0x555fc1223520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.814+0000 7fecaa861740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.814+0000 7feca0e8d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 0x555fc1223520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 0x555fc1223520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59776/0 (socket says 172.21.15.31:59776) 2024-09-21T06:12:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 -- 172.21.15.31:0/3718291968 learned_addr learned my addr 172.21.15.31:0/3718291968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0dd42f0 0x555fc1222f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec8000d430 con 0x555fc0dd42f0 2024-09-21T06:12:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec8000e8a0 con 0x555fc0dfd1b0 2024-09-21T06:12:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec8000db00 con 0x555fc0ddc4d0 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7fecaa861740 10 client.0 _ll_register_callbacks cb 0x7fec8c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 -- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 msgr2=0x555fc11b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 -- 172.21.15.31:0/3718291968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0dd42f0 msgr2=0x555fc1222f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0dd42f0 0x555fc1222f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca0e8d640 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc11b6850 con 0x555fc0ddc4d0 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.815+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0dd42f0 0x555fc1222f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.816+0000 7feca0e8d640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 0x555fc1223520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec8c005bc0 tx=0x7fec8c019e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c75c7aecf3a62c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:02.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.816+0000 7fec84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.816+0000 7fec84ff9640 1 -- 172.21.15.31:0/3718291968 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c011d90 con 0x555fc0ddc4d0 2024-09-21T06:12:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.816+0000 7fec84ff9640 1 -- 172.21.15.31:0/3718291968 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fec8c011440 con 0x555fc0ddc4d0 2024-09-21T06:12:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.816+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec9402c070 con 0x555fc0ddc4d0 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.816+0000 7fec84ff9640 1 -- 172.21.15.31:0/3718291968 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fec8c011640 con 0x555fc0ddc4d0 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.817+0000 7fecaa861740 10 client.25300 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.817+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555fc122a6f0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.817+0000 7fec84ff9640 1 -- 172.21.15.31:0/3718291968 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec8c0117e0 con 0x555fc0ddc4d0 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fec84ff9640 1 -- 172.21.15.31:0/3718291968 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fec8c003c00 con 0x555fc0ddc4d0 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fecaa861740 10 client.25300 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fecaa861740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fecaa861740 20 client.25300 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fecaa861740 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0d911c0 0x555fc122ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fecaa861740 4 client.25300 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:12:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0d911c0 0x555fc122ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.818+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fec9402c070 con 0x555fc0d911c0 2024-09-21T06:12:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.819+0000 7fec84ff9640 10 client.25300 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.819+0000 7feca1e8f640 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0d911c0 0x555fc122ce90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27283194e2ff463 server_cookie=5c8e994ab9ab1638 in_seq=0 out_seq=0 2024-09-21T06:12:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fec84ff9640 1 -- 172.21.15.31:0/3718291968 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fec9402c070 con 0x555fc0d911c0 2024-09-21T06:12:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fec84ff9640 10 client.25300 handle_command_reply: tid=0 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 client.25300 shutdown 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 msgr2=0x555fc1223520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 0x555fc1223520 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec8c005bc0 tx=0x7fec8c019e00 comp rx=0 tx=0).stop 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 shutdown_connections 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555fc0d911c0 0x555fc122ce90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fec84ff9640 0 client.25300 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fc0dfd1b0 0x555fc11b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555fc0ddc4d0 0x555fc1223520 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 --2- 172.21.15.31:0/3718291968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555fc0dd42f0 0x555fc1222f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.841+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 >> 172.21.15.31:0/3718291968 conn(0x555fc0df1300 msgr2=0x555fc11a4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 shutdown_connections 2024-09-21T06:12:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7fecaa861740 1 -- 172.21.15.31:0/3718291968 wait complete. 2024-09-21T06:12:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7fecaa861740 20 client.25300 trim_cache size 0 max 16384 2024-09-21T06:12:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:12:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:12:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:12:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:12:02.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec880b68c0 msgr2=0x7fec880b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7feca38f2640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec880b68c0 0x7fec880b8dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x555fc11a1630 tx=0x7fec9810b7c0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 msgr2=0x7fec9c074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.842+0000 7feca38f2640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 0x7fec9c074a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec98005ab0 tx=0x7fec98003c20 comp rx=0 tx=0).stop 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 shutdown_connections 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec880b68c0 0x7fec880b8dd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec9c06dd20 0x7fec9c06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec9c1026c0 0x7fec9c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 --2- 172.21.15.31:0/3454311377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06a350 0x7fec9c074a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 >> 172.21.15.31:0/3454311377 conn(0x7fec9c079e90 msgr2=0x7fec9c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 shutdown_connections 2024-09-21T06:12:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:02.843+0000 7feca38f2640 1 -- 172.21.15.31:0/3454311377 wait complete. 2024-09-21T06:12:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:03 smithi031 ceph-mon[31237]: pgmap v333: 97 pgs: 97 active+clean; 973 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T06:12:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:03 smithi031 ceph-mon[31237]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T06:12:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:03 smithi110 ceph-mon[34550]: pgmap v333: 97 pgs: 97 active+clean; 973 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T06:12:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:03 smithi110 ceph-mon[34550]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T06:12:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:03 smithi107 ceph-mon[34380]: pgmap v333: 97 pgs: 97 active+clean; 973 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T06:12:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:03 smithi107 ceph-mon[34380]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T06:12:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 25.0 s, 1791.0 tps, lat 56.789 ms stddev 135.685 2024-09-21T06:12:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:05 smithi031 ceph-mon[31237]: pgmap v334: 97 pgs: 97 active+clean; 973 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 783 op/s 2024-09-21T06:12:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:05 smithi110 ceph-mon[34550]: pgmap v334: 97 pgs: 97 active+clean; 973 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 783 op/s 2024-09-21T06:12:05.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:05 smithi107 ceph-mon[34380]: pgmap v334: 97 pgs: 97 active+clean; 973 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 783 op/s 2024-09-21T06:12:07.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:07 smithi031 ceph-mon[31237]: pgmap v335: 97 pgs: 97 active+clean; 997 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 1.43k op/s 2024-09-21T06:12:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:07 smithi110 ceph-mon[34550]: pgmap v335: 97 pgs: 97 active+clean; 997 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 1.43k op/s 2024-09-21T06:12:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:07 smithi107 ceph-mon[34380]: pgmap v335: 97 pgs: 97 active+clean; 997 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 1.43k op/s 2024-09-21T06:12:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:09 smithi031 ceph-mon[31237]: pgmap v336: 97 pgs: 97 active+clean; 1009 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 MiB/s wr, 1.33k op/s 2024-09-21T06:12:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:09 smithi110 ceph-mon[34550]: pgmap v336: 97 pgs: 97 active+clean; 1009 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 MiB/s wr, 1.33k op/s 2024-09-21T06:12:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:09 smithi107 ceph-mon[34380]: pgmap v336: 97 pgs: 97 active+clean; 1009 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 MiB/s wr, 1.33k op/s 2024-09-21T06:12:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 30.0 s, 5200.6 tps, lat 19.221 ms stddev 22.411 2024-09-21T06:12:10.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:12:11.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:11 smithi031 ceph-mon[31237]: pgmap v337: 97 pgs: 97 active+clean; 1016 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 47 MiB/s wr, 1.34k op/s 2024-09-21T06:12:11.375 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:11 smithi110 ceph-mon[34550]: pgmap v337: 97 pgs: 97 active+clean; 1016 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 47 MiB/s wr, 1.34k op/s 2024-09-21T06:12:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:11 smithi107 ceph-mon[34380]: pgmap v337: 97 pgs: 97 active+clean; 1016 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 47 MiB/s wr, 1.34k op/s 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.971+0000 7fdac7cdc640 1 -- 172.21.15.31:0/2496140171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac011cc30 msgr2=0x7fdac011f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.971+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/2496140171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac011cc30 0x7fdac011f140 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009ca0 tx=0x7fdab802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.973+0000 7fdac7cdc640 1 -- 172.21.15.31:0/2496140171 shutdown_connections 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.973+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/2496140171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac011cc30 0x7fdac011f140 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.973+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/2496140171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac010a9e0 0x7fdac010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.973+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/2496140171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac0111f40 0x7fdac010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.973+0000 7fdac7cdc640 1 -- 172.21.15.31:0/2496140171 >> 172.21.15.31:0/2496140171 conn(0x7fdac0070ec0 msgr2=0x7fdac00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.973+0000 7fdac7cdc640 1 -- 172.21.15.31:0/2496140171 shutdown_connections 2024-09-21T06:12:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.974+0000 7fdac7cdc640 1 -- 172.21.15.31:0/2496140171 wait complete. 2024-09-21T06:12:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.975+0000 7fdac7cdc640 1 Processor -- start 2024-09-21T06:12:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.975+0000 7fdac7cdc640 1 -- start start 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.975+0000 7fdac7cdc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 0x7fdac01a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.975+0000 7fdac7cdc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac0111f40 0x7fdac01a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.975+0000 7fdac7cdc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac011cc30 0x7fdac01aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac7cdc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0122900 con 0x7fdac0111f40 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 0x7fdac01a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 0x7fdac01a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60918/0 (socket says 172.21.15.31:60918) 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 -- 172.21.15.31:0/1280688628 learned_addr learned my addr 172.21.15.31:0/1280688628 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5277640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac0111f40 0x7fdac01a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac6279640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac011cc30 0x7fdac01aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0122780 con 0x7fdac011cc30 2024-09-21T06:12:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0122a80 con 0x7fdac010a9e0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 -- 172.21.15.31:0/1280688628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac011cc30 msgr2=0x7fdac01aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac011cc30 0x7fdac01aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 -- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac0111f40 msgr2=0x7fdac01a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac0111f40 0x7fdac01a97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.976+0000 7fdac5a78640 1 -- 172.21.15.31:0/1280688628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac01af1a0 con 0x7fdac010a9e0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.977+0000 7fdac5a78640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 0x7fdac01a9230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00e9f0 tx=0x7fdabc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.977+0000 7fdab6ffd640 1 -- 172.21.15.31:0/1280688628 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00cd80 con 0x7fdac010a9e0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.977+0000 7fdab6ffd640 1 -- 172.21.15.31:0/1280688628 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdabc0108e0 con 0x7fdac010a9e0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.977+0000 7fdab6ffd640 1 -- 172.21.15.31:0/1280688628 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc018860 con 0x7fdac010a9e0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.977+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac01af490 con 0x7fdac010a9e0 2024-09-21T06:12:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.977+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac01b5bf0 con 0x7fdac010a9e0 2024-09-21T06:12:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.981+0000 7fdace250740 1 Processor -- start 2024-09-21T06:12:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.982+0000 7fdace250740 1 -- start start 2024-09-21T06:12:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.982+0000 7fdace250740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 0x56173f292650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.982+0000 7fdace250740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eec21b0 0x56173f292b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdac6279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 0x56173f292650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdac6279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 0x56173f292650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37174/0 (socket says 172.21.15.31:37174) 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdac6279640 1 -- 172.21.15.31:0/3719750301 learned_addr learned my addr 172.21.15.31:0/3719750301 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdab6ffd640 1 -- 172.21.15.31:0/1280688628 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdabc018a00 con 0x7fdac010a9e0 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdace250740 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeca390 0x56173f2930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56173f208e50 con 0x56173eeeb070 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56173f1f8280 con 0x56173eeca390 2024-09-21T06:12:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56173f1fc090 con 0x56173eec21b0 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdac5277640 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeca390 0x56173f2930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.983+0000 7fdac5a78640 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eec21b0 0x56173f292b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.984+0000 7fdac6279640 1 -- 172.21.15.31:0/3719750301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eec21b0 msgr2=0x56173f292b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.984+0000 7fdac6279640 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eec21b0 0x56173f292b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.984+0000 7fdac6279640 1 -- 172.21.15.31:0/3719750301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeca390 msgr2=0x56173f2930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.984+0000 7fdac6279640 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeca390 0x56173f2930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.984+0000 7fdac6279640 1 -- 172.21.15.31:0/3719750301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173ef56650 con 0x56173eeeb070 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.984+0000 7fdab6ffd640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdaac0b6850 0x7fdaac0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.985+0000 7fdab6ffd640 1 -- 172.21.15.31:0/1280688628 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdabc0e7980 con 0x7fdac010a9e0 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.985+0000 7fdac5a78640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdaac0b6850 0x7fdaac0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.985+0000 7fdac6279640 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 0x56173f292650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009ca0 tx=0x7fdab8009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd6f119698418db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.985+0000 7fdac5a78640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdaac0b6850 0x7fdaac0b8d60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56173f2912d0 tx=0x7fdabc009700 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.986+0000 7fdaabfff640 1 -- 172.21.15.31:0/3719750301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8031cf0 con 0x56173eeeb070 2024-09-21T06:12:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.986+0000 7fdaabfff640 1 -- 172.21.15.31:0/3719750301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab8036030 con 0x56173eeeb070 2024-09-21T06:12:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.986+0000 7fdaabfff640 1 -- 172.21.15.31:0/3719750301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80042a0 con 0x56173eeeb070 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 msgr2=0x56173f292650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 0x56173f292650 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009ca0 tx=0x7fdab8009430 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 shutdown_connections 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeca390 0x56173f2930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eec21b0 0x56173f292b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 --2- 172.21.15.31:0/3719750301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeeb070 0x56173f292650 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.987+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 >> 172.21.15.31:0/3719750301 conn(0x56173eedf460 msgr2=0x56173f290540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.988+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 shutdown_connections 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.988+0000 7fdace250740 1 -- 172.21.15.31:0/3719750301 wait complete. 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.988+0000 7fdace250740 1 -- start start 2024-09-21T06:12:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.989+0000 7fdace250740 1 Processor -- start 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f29ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f29e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56173f059700 con 0x56173eeca390 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56173f193110 con 0x56173eec21b0 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56173f2a0c50 con 0x56173eeeb070 2024-09-21T06:12:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdace250740 10 client.0 _ll_register_callbacks cb 0x56173f1ac990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac5277640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f29e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f29ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f29ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37270/0 (socket says 172.21.15.31:37270) 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 -- 172.21.15.31:0/2921690781 learned_addr learned my addr 172.21.15.31:0/2921690781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 -- 172.21.15.31:0/2921690781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 msgr2=0x56173f2a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 -- 172.21.15.31:0/2921690781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 msgr2=0x56173f29e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f29e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.990+0000 7fdac6279640 1 -- 172.21.15.31:0/2921690781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173f1c0c40 con 0x56173eec21b0 2024-09-21T06:12:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.991+0000 7fdac6279640 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f29ddd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdab8003c80 tx=0x7fdab8003d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b0a8542933de6e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.991+0000 7fdaa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.991+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2921690781 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8004640 con 0x56173eec21b0 2024-09-21T06:12:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.991+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab8036030 con 0x56173eec21b0 2024-09-21T06:12:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.992+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2921690781 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdab80047e0 con 0x56173eec21b0 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2921690781 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdab8004b40 con 0x56173eec21b0 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdace250740 10 client.25304 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56173f2ac9f0 con 0x56173eec21b0 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2921690781 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab804b040 con 0x56173eec21b0 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2921690781 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdab8043070 con 0x56173eec21b0 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdace250740 10 client.25304 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdace250740 10 client.25304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.993+0000 7fdace250740 20 client.25304 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.994+0000 7fdace250740 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173eea99f0 0x56173f2afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.994+0000 7fdace250740 4 client.25304 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.994+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdab8036030 con 0x56173eea99f0 2024-09-21T06:12:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.994+0000 7fdac5277640 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173eea99f0 0x56173f2afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.995+0000 7fdac5277640 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173eea99f0 0x56173f2afd90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcaa92c05292c3a server_cookie=ad0c98ce77d895e3 in_seq=0 out_seq=0 2024-09-21T06:12:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.995+0000 7fdaa97fa640 10 client.25304 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2921690781 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdab8036030 con 0x56173eea99f0 2024-09-21T06:12:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdaa97fa640 10 client.25304 handle_command_reply: tid=0 2024-09-21T06:12:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdace250740 1 client.25304 shutdown 2024-09-21T06:12:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 msgr2=0x56173f29ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdace250740 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f29ddd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdab8003c80 tx=0x7fdab8003d80 comp rx=0 tx=0).stop 2024-09-21T06:12:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 shutdown_connections 2024-09-21T06:12:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdace250740 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173eea99f0 0x56173f2afd90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.996+0000 7fdace250740 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.997+0000 7fdace250740 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f29e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.997+0000 7fdace250740 1 --2- 172.21.15.31:0/2921690781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f29ddd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.997+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 >> 172.21.15.31:0/2921690781 conn(0x56173eedf460 msgr2=0x56173f295830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.997+0000 7fdaa97fa640 0 client.25304 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.998+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 shutdown_connections 2024-09-21T06:12:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.998+0000 7fdace250740 1 -- 172.21.15.31:0/2921690781 wait complete. 2024-09-21T06:12:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:11.998+0000 7fdace250740 20 client.25304 trim_cache size 0 max 16384 2024-09-21T06:12:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.030+0000 7fdace250740 1 Processor -- start 2024-09-21T06:12:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.030+0000 7fdace250740 1 -- start start 2024-09-21T06:12:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.031+0000 7fdace250740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eec21b0 0x56173f293d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.031+0000 7fdac6279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eec21b0 0x56173f293d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.031+0000 7fdac6279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eec21b0 0x56173f293d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37192/0 (socket says 172.21.15.31:37192) 2024-09-21T06:12:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.031+0000 7fdac6279640 1 -- 172.21.15.31:0/1808348974 learned_addr learned my addr 172.21.15.31:0/1808348974 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.031+0000 7fdace250740 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeca390 0x56173f29afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.031+0000 7fdace250740 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeeb070 0x56173f29b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01bc4a0 con 0x56173eec21b0 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01bed20 con 0x56173eeeb070 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01bcb10 con 0x56173eeca390 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdac5277640 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeca390 0x56173f29afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdac5277640 1 -- 172.21.15.31:0/1808348974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeeb070 msgr2=0x56173f29b500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdac5277640 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeeb070 0x56173f29b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdac5277640 1 -- 172.21.15.31:0/1808348974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eec21b0 msgr2=0x56173f293d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdac5277640 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eec21b0 0x56173f293d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.032+0000 7fdac5277640 1 -- 172.21.15.31:0/1808348974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173f2b02b0 con 0x56173eeca390 2024-09-21T06:12:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.033+0000 7fdac5277640 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeca390 0x56173f29afc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x56173f2b0490 tx=0x7fdab0012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8804cc764605d32c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.034+0000 7fdaab7fe640 1 -- 172.21.15.31:0/1808348974 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0009a80 con 0x56173eeca390 2024-09-21T06:12:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.034+0000 7fdaab7fe640 1 -- 172.21.15.31:0/1808348974 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab00054d0 con 0x56173eeca390 2024-09-21T06:12:12.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.034+0000 7fdaab7fe640 1 -- 172.21.15.31:0/1808348974 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0010430 con 0x56173eeca390 2024-09-21T06:12:12.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeca390 msgr2=0x56173f29afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeca390 0x56173f29afc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x56173f2b0490 tx=0x7fdab0012820 comp rx=0 tx=0).stop 2024-09-21T06:12:12.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 shutdown_connections 2024-09-21T06:12:12.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eeeb070 0x56173f29b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeca390 0x56173f29afc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 --2- 172.21.15.31:0/1808348974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eec21b0 0x56173f293d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 >> 172.21.15.31:0/1808348974 conn(0x56173eedf460 msgr2=0x56173f293680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 shutdown_connections 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.035+0000 7fdace250740 1 -- 172.21.15.31:0/1808348974 wait complete. 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.036+0000 7fdace250740 1 -- start start 2024-09-21T06:12:12.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.036+0000 7fdace250740 1 Processor -- start 2024-09-21T06:12:12.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.037+0000 7fdace250740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f2a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.037+0000 7fdace250740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f2a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.037+0000 7fdace250740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdace250740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01bd1e0 con 0x56173eeca390 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f2a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f2a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37292/0 (socket says 172.21.15.31:37292) 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 -- 172.21.15.31:0/2058378553 learned_addr learned my addr 172.21.15.31:0/2058378553 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01be650 con 0x56173eec21b0 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01bd8b0 con 0x56173eeeb070 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac5277640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f2a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdace250740 10 client.0 _ll_register_callbacks cb 0x7fdab0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:12.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 -- 172.21.15.31:0/2058378553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 msgr2=0x56173f2a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:12.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 -- 172.21.15.31:0/2058378553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 msgr2=0x56173f2a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:12.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f2a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.038+0000 7fdac6279640 1 -- 172.21.15.31:0/2058378553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173f2a4460 con 0x56173eec21b0 2024-09-21T06:12:12.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.040+0000 7fdac6279640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f2a64c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdab80094e0 tx=0x7fdab8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83db08acb2c31a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:12.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.040+0000 7fdaa97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:12.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.041+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2058378553 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8004350 con 0x56173eec21b0 2024-09-21T06:12:12.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.041+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2058378553 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdab80044f0 con 0x56173eec21b0 2024-09-21T06:12:12.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.041+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab00054d0 con 0x56173eec21b0 2024-09-21T06:12:12.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.041+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2058378553 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdab80046f0 con 0x56173eec21b0 2024-09-21T06:12:12.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.041+0000 7fdace250740 10 client.25307 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:12.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.041+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56173f31a470 con 0x56173eec21b0 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2058378553 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab00054d0 con 0x56173eec21b0 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2058378553 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdab8043040 con 0x56173eec21b0 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdace250740 10 client.25307 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdace250740 10 client.25307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdace250740 20 client.25307 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdace250740 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173ee7eda0 0x56173f31cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdace250740 4 client.25307 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:12:12.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56173f20ada0 con 0x56173ee7eda0 2024-09-21T06:12:12.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.042+0000 7fdac5277640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173ee7eda0 0x56173f31cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:12.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.043+0000 7fdaa97fa640 10 client.25307 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:12.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.043+0000 7fdac5277640 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173ee7eda0 0x56173f31cb40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f942a22726e37d server_cookie=17963cbd3282e120 in_seq=0 out_seq=0 2024-09-21T06:12:12.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.043+0000 7fdaa97fa640 1 -- 172.21.15.31:0/2058378553 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x56173f20ada0 con 0x56173ee7eda0 2024-09-21T06:12:12.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.043+0000 7fdaa97fa640 10 client.25307 handle_command_reply: tid=0 2024-09-21T06:12:12.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.043+0000 7fdace250740 1 client.25307 shutdown 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.044+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 msgr2=0x56173f2a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.044+0000 7fdace250740 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f2a64c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdab80094e0 tx=0x7fdab8000f40 comp rx=0 tx=0).stop 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.044+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 shutdown_connections 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.044+0000 7fdace250740 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56173ee7eda0 0x56173f31cb40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56173eeeb070 0x56173f2a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56173eeca390 0x56173f2a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 1 --2- 172.21.15.31:0/2058378553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56173eec21b0 0x56173f2a64c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 >> 172.21.15.31:0/2058378553 conn(0x56173eedf460 msgr2=0x56173f294040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdaa97fa640 0 client.25307 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:12.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 shutdown_connections 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 1 -- 172.21.15.31:0/2058378553 wait complete. 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.045+0000 7fdace250740 20 client.25307 trim_cache size 0 max 16384 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:12.050 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.046+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdaac0b6850 msgr2=0x7fdaac0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.046+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdaac0b6850 0x7fdaac0b8d60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x56173f2912d0 tx=0x7fdabc009700 comp rx=0 tx=0).stop 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 msgr2=0x7fdac01a9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 0x7fdac01a9230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00e9f0 tx=0x7fdabc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 shutdown_connections 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdac011cc30 0x7fdac01aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdaac0b6850 0x7fdaac0b8d60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac0111f40 0x7fdac01a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 --2- 172.21.15.31:0/1280688628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdac010a9e0 0x7fdac01a9230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 >> 172.21.15.31:0/1280688628 conn(0x7fdac0070ec0 msgr2=0x7fdac0071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:12.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 shutdown_connections 2024-09-21T06:12:12.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:12.047+0000 7fdac7cdc640 1 -- 172.21.15.31:0/1280688628 wait complete. 2024-09-21T06:12:13.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:13 smithi031 ceph-mon[31237]: pgmap v338: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 MiB/s wr, 1.72k op/s 2024-09-21T06:12:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:13 smithi110 ceph-mon[34550]: pgmap v338: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 MiB/s wr, 1.72k op/s 2024-09-21T06:12:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:13 smithi107 ceph-mon[34380]: pgmap v338: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 56 MiB/s wr, 1.72k op/s 2024-09-21T06:12:13.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:12:13.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:14.379 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:14.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.984+0000 7fc9d9787640 1 -- 172.21.15.31:0/1404220840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 msgr2=0x7fc9d406b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.984+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1404220840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d406b5d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009900 tx=0x7fc9c402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.985+0000 7fc9d9787640 1 -- 172.21.15.31:0/1404220840 shutdown_connections 2024-09-21T06:12:14.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.985+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1404220840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109c60 0x7fc9d4110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.985+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1404220840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109310 0x7fc9d4109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.985+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1404220840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d406b5d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.985+0000 7fc9d9787640 1 -- 172.21.15.31:0/1404220840 >> 172.21.15.31:0/1404220840 conn(0x7fc9d4079560 msgr2=0x7fc9d4079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:14.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.986+0000 7fc9d9787640 1 -- 172.21.15.31:0/1404220840 shutdown_connections 2024-09-21T06:12:14.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.986+0000 7fc9d9787640 1 -- 172.21.15.31:0/1404220840 wait complete. 2024-09-21T06:12:14.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.987+0000 7fc9d9787640 1 Processor -- start 2024-09-21T06:12:14.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.987+0000 7fc9d9787640 1 -- start start 2024-09-21T06:12:14.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.988+0000 7fc9d9787640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.988+0000 7fc9d9787640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109310 0x7fc9d419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.988+0000 7fc9d9787640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 0x7fc9d419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d9787640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113970 con 0x7fc9d406b1c0 2024-09-21T06:12:14.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 0x7fc9d419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 0x7fc9d419fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37302/0 (socket says 172.21.15.31:37302) 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d3fff640 1 -- 172.21.15.31:0/1042764123 learned_addr learned my addr 172.21.15.31:0/1042764123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109310 0x7fc9d419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41137f0 con 0x7fc9d4109c60 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4113af0 con 0x7fc9d4109310 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.989+0000 7fc9d2ffd640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d2ffd640 1 -- 172.21.15.31:0/1042764123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109310 msgr2=0x7fc9d419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d2ffd640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109310 0x7fc9d419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d2ffd640 1 -- 172.21.15.31:0/1042764123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 msgr2=0x7fc9d419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d2ffd640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 0x7fc9d419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d2ffd640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d41a0230 con 0x7fc9d406b1c0 2024-09-21T06:12:14.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d3fff640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 0x7fc9d419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d27fc640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109310 0x7fc9d419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.990+0000 7fc9d2ffd640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d419a2c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc9c402a9e0 tx=0x7fc9c403ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.992+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4035070 con 0x7fc9d406b1c0 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.992+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c4003cb0 con 0x7fc9d406b1c0 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.992+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031da0 con 0x7fc9d406b1c0 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.992+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d41a04c0 con 0x7fc9d406b1c0 2024-09-21T06:12:14.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.992+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d41a0920 con 0x7fc9d406b1c0 2024-09-21T06:12:14.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.993+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d406b5d0 con 0x7fc9d406b1c0 2024-09-21T06:12:14.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.996+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc9c4031530 con 0x7fc9d406b1c0 2024-09-21T06:12:14.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.997+0000 7fc9bbfff640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9bc0b6800 0x7fc9bc0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:14.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.998+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc9c410bd90 con 0x7fc9d406b1c0 2024-09-21T06:12:15.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.998+0000 7fc9d27fc640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9bc0b6800 0x7fc9bc0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:15.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:14.999+0000 7fc9d27fc640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9bc0b6800 0x7fc9bc0b8d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80058c0 tx=0x7fc9c8005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:15.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.002+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9c40d23d0 con 0x7fc9d406b1c0 2024-09-21T06:12:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 35.0 s, 4018.6 tps, lat 24.872 ms stddev 27.602 2024-09-21T06:12:15.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.430+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9d419b3f0 con 0x7fc9d406b1c0 2024-09-21T06:12:15.432 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:15 smithi031 ceph-mon[31237]: pgmap v339: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 52 MiB/s wr, 1.39k op/s 2024-09-21T06:12:15.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.432+0000 7fc9bbfff640 1 -- 172.21.15.31:0/1042764123 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fc9d419b3f0 con 0x7fc9d406b1c0 2024-09-21T06:12:15.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:12:15.435 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:12:15.435 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9bc0b6800 msgr2=0x7fc9bc0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9bc0b6800 0x7fc9bc0b8d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80058c0 tx=0x7fc9c8005850 comp rx=0 tx=0).stop 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 msgr2=0x7fc9d419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d419a2c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc9c402a9e0 tx=0x7fc9c403ac80 comp rx=0 tx=0).stop 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 shutdown_connections 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d4109c60 0x7fc9d419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9d4109310 0x7fc9d419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc9bc0b6800 0x7fc9bc0b8d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 --2- 172.21.15.31:0/1042764123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9d406b1c0 0x7fc9d419a2c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:15.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.436+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 >> 172.21.15.31:0/1042764123 conn(0x7fc9d4079560 msgr2=0x7fc9d4107320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:15.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.437+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 shutdown_connections 2024-09-21T06:12:15.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:15.437+0000 7fc9d9787640 1 -- 172.21.15.31:0/1042764123 wait complete. 2024-09-21T06:12:15.579 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:15 smithi110 ceph-mon[34550]: pgmap v339: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 52 MiB/s wr, 1.39k op/s 2024-09-21T06:12:15.589 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:15 smithi107 ceph-mon[34380]: pgmap v339: 97 pgs: 97 active+clean; 1.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 52 MiB/s wr, 1.39k op/s 2024-09-21T06:12:16.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.001+0000 7f43b6656640 1 -- 172.21.15.31:0/2814909122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b0110660 msgr2=0x7f43b0110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.001+0000 7f43b6656640 1 --2- 172.21.15.31:0/2814909122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b0110660 0x7f43b0110a70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009ca0 tx=0x7f43a002a940 comp rx=0 tx=0).stop 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.001+0000 7f43b6656640 1 -- 172.21.15.31:0/2814909122 shutdown_connections 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.002+0000 7f43b6656640 1 --2- 172.21.15.31:0/2814909122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b010b5a0 0x7f43b01173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.002+0000 7f43b6656640 1 --2- 172.21.15.31:0/2814909122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b010b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.002+0000 7f43b6656640 1 --2- 172.21.15.31:0/2814909122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b0110660 0x7f43b0110a70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.002+0000 7f43b6656640 1 -- 172.21.15.31:0/2814909122 >> 172.21.15.31:0/2814909122 conn(0x7f43b0071220 msgr2=0x7f43b0073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.003+0000 7f43b6656640 1 -- 172.21.15.31:0/2814909122 shutdown_connections 2024-09-21T06:12:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.003+0000 7f43b6656640 1 -- 172.21.15.31:0/2814909122 wait complete. 2024-09-21T06:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.004+0000 7f43b6656640 1 Processor -- start 2024-09-21T06:12:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.004+0000 7f43b6656640 1 -- start start 2024-09-21T06:12:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43b6656640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b01a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43b6656640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b010b5a0 0x7f43b01a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43b6656640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b0110660 0x7f43b01aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43b6656640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43b011a050 con 0x7f43b0110660 2024-09-21T06:12:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b01a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43affff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b01a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:32788/0 (socket says 172.21.15.31:32788) 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43affff640 1 -- 172.21.15.31:0/2219865708 learned_addr learned my addr 172.21.15.31:0/2219865708 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43a77fe640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b010b5a0 0x7f43b01a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.005+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0119ed0 con 0x7f43b010b5a0 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43b011a1d0 con 0x7f43b010ac50 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b0110660 0x7f43b01aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43affff640 1 -- 172.21.15.31:0/2219865708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b010b5a0 msgr2=0x7f43b01a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43affff640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b010b5a0 0x7f43b01a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43affff640 1 -- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b0110660 msgr2=0x7f43b01aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43affff640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b0110660 0x7f43b01aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.006+0000 7f43affff640 1 -- 172.21.15.31:0/2219865708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b01af190 con 0x7f43b010ac50 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.007+0000 7f43affff640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b01a9260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43a0005e30 tx=0x7f43a00309b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.008+0000 7f43adffb640 1 -- 172.21.15.31:0/2219865708 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0034040 con 0x7f43b010ac50 2024-09-21T06:12:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.008+0000 7f43adffb640 1 -- 172.21.15.31:0/2219865708 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a002fd10 con 0x7f43b010ac50 2024-09-21T06:12:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.008+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b01af420 con 0x7f43b010ac50 2024-09-21T06:12:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.008+0000 7f43adffb640 1 -- 172.21.15.31:0/2219865708 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0004670 con 0x7f43b010ac50 2024-09-21T06:12:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.008+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b01b5c10 con 0x7f43b010ac50 2024-09-21T06:12:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.012+0000 7f43bd5c8740 1 Processor -- start 2024-09-21T06:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.012+0000 7f43bd5c8740 1 -- start start 2024-09-21T06:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.012+0000 7f43adffb640 1 -- 172.21.15.31:0/2219865708 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f43a0037020 con 0x7f43b010ac50 2024-09-21T06:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.012+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.012+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5595313226f0 con 0x559530fdf2f0 2024-09-21T06:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559531311b20 con 0x559530fe74d0 2024-09-21T06:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559531315930 con 0x5595310081b0 2024-09-21T06:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:32794/0 (socket says 172.21.15.31:32794) 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 -- 172.21.15.31:0/3443106266 learned_addr learned my addr 172.21.15.31:0/3443106266 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43affff640 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43a77fe640 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 -- 172.21.15.31:0/3443106266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 msgr2=0x5595313ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 -- 172.21.15.31:0/3443106266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 msgr2=0x5595313adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43b4bf3640 1 -- 172.21.15.31:0/3443106266 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595312d7530 con 0x5595310081b0 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.013+0000 7f43affff640 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.014+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313ad9b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b6b0 tx=0x7f43a800bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606593b2d7f2b438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.015+0000 7f43a67fc640 1 -- 172.21.15.31:0/3443106266 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8003bd0 con 0x5595310081b0 2024-09-21T06:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.015+0000 7f43a67fc640 1 -- 172.21.15.31:0/3443106266 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8015070 con 0x5595310081b0 2024-09-21T06:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.015+0000 7f43a67fc640 1 -- 172.21.15.31:0/3443106266 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80106a0 con 0x5595310081b0 2024-09-21T06:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.015+0000 7f43adffb640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f439c0b67d0 0x7f439c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.015+0000 7f43adffb640 1 -- 172.21.15.31:0/2219865708 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f43a01084a0 con 0x7f43b010ac50 2024-09-21T06:12:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.015+0000 7f43affff640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f439c0b67d0 0x7f439c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3443106266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 msgr2=0x5595313ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313ad9b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b6b0 tx=0x7f43a800bb70 comp rx=0 tx=0).stop 2024-09-21T06:12:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3443106266 shutdown_connections 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3443106266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313ad9b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3443106266 >> 172.21.15.31:0/3443106266 conn(0x559530ffc300 msgr2=0x5595313ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.017+0000 7f43affff640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f439c0b67d0 0x7f439c0b8ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43a0105ce0 tx=0x7f43a002a940 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.018+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3443106266 shutdown_connections 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.018+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3443106266 wait complete. 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.018+0000 7f43bd5c8740 1 -- start start 2024-09-21T06:12:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.019+0000 7f43bd5c8740 1 Processor -- start 2024-09-21T06:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.019+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 0x5595313b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 0x5595313b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 0x5595313b9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:32796/0 (socket says 172.21.15.31:32796) 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559531249ab0 con 0x5595310081b0 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5595312b7730 con 0x559530fe74d0 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5595313bc130 con 0x559530fdf2f0 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 -- 172.21.15.31:0/2541397710 learned_addr learned my addr 172.21.15.31:0/2541397710 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 10 client.0 _ll_register_callbacks cb 0x55953130b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43a77fe640 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 -- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 msgr2=0x5595313b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 -- 172.21.15.31:0/2541397710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 msgr2=0x5595313c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43b4bf3640 1 -- 172.21.15.31:0/2541397710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559531073790 con 0x559530fdf2f0 2024-09-21T06:12:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.020+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8015070 con 0x559530fdf2f0 2024-09-21T06:12:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 0x5595313b9130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43a800bff0 tx=0x7f43a8003e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67ce9390b764d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f438f7fe640 10 client.25336 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f438f7fe640 1 -- 172.21.15.31:0/2541397710 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011850 con 0x559530fdf2f0 2024-09-21T06:12:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f438f7fe640 1 -- 172.21.15.31:0/2541397710 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f43a8005030 con 0x559530fdf2f0 2024-09-21T06:12:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f438f7fe640 1 -- 172.21.15.31:0/2541397710 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f43a80052b0 con 0x559530fdf2f0 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f43bd5c8740 10 client.25336 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.021+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5595310ec620 con 0x559530fdf2f0 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f438f7fe640 1 -- 172.21.15.31:0/2541397710 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f43a802c030 con 0x559530fdf2f0 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f43bd5c8740 10 client.25336 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f43bd5c8740 10 client.25336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f43bd5c8740 20 client.25336 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530fc6d80 0x5595313cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f43bd5c8740 4 client.25336 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.022+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559531324640 con 0x559530fc6d80 2024-09-21T06:12:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.023+0000 7f43a77fe640 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530fc6d80 0x5595313cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.023+0000 7f43a77fe640 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530fc6d80 0x5595313cafa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dcbadbb96fb372 server_cookie=eccdab3a3215f1cd in_seq=0 out_seq=0 2024-09-21T06:12:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.023+0000 7f438f7fe640 10 client.25336 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.024+0000 7f438f7fe640 1 -- 172.21.15.31:0/2541397710 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559531324640 con 0x559530fc6d80 2024-09-21T06:12:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.024+0000 7f438f7fe640 10 client.25336 handle_command_reply: tid=0 2024-09-21T06:12:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.024+0000 7f43bd5c8740 1 client.25336 shutdown 2024-09-21T06:12:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.024+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 msgr2=0x5595313b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.024+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 0x5595313b9130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43a800bff0 tx=0x7f43a8003e70 comp rx=0 tx=0).stop 2024-09-21T06:12:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 shutdown_connections 2024-09-21T06:12:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530fc6d80 0x5595313cafa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/2541397710 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fdf2f0 0x5595313b9130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 >> 172.21.15.31:0/2541397710 conn(0x559530ffc300 msgr2=0x5595313ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f438f7fe640 0 client.25336 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 shutdown_connections 2024-09-21T06:12:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.026+0000 7f43bd5c8740 1 -- 172.21.15.31:0/2541397710 wait complete. 2024-09-21T06:12:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.027+0000 7f43bd5c8740 20 client.25336 trim_cache size 0 max 16384 2024-09-21T06:12:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.056+0000 7f43bd5c8740 1 Processor -- start 2024-09-21T06:12:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.056+0000 7f43bd5c8740 1 -- start start 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.056+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 0x5595313b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fe74d0 0x5595313bad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01bc400 con 0x5595310081b0 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01bec60 con 0x559530fdf2f0 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01bca50 con 0x559530fe74d0 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 0x5595313b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 0x5595313b9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37350/0 (socket says 172.21.15.31:37350) 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 -- 172.21.15.31:0/4105548742 learned_addr learned my addr 172.21.15.31:0/4105548742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43a77fe640 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fe74d0 0x5595313bad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 -- 172.21.15.31:0/4105548742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fe74d0 msgr2=0x5595313bad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fe74d0 0x5595313bad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 -- 172.21.15.31:0/4105548742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 msgr2=0x5595313c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 -- 172.21.15.31:0/4105548742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595313b7610 con 0x559530fdf2f0 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.057+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 0x5595313b9610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005680 tx=0x7f43a8023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38166717ad3bd1c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.058+0000 7f43a5ffb640 1 -- 172.21.15.31:0/4105548742 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011850 con 0x559530fdf2f0 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.058+0000 7f43a5ffb640 1 -- 172.21.15.31:0/4105548742 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8015070 con 0x559530fdf2f0 2024-09-21T06:12:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.058+0000 7f43a5ffb640 1 -- 172.21.15.31:0/4105548742 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8024680 con 0x559530fdf2f0 2024-09-21T06:12:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.059+0000 7f43bd5c8740 1 -- 172.21.15.31:0/4105548742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 msgr2=0x5595313b9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.059+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 0x5595313b9610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005680 tx=0x7f43a8023000 comp rx=0 tx=0).stop 2024-09-21T06:12:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.059+0000 7f43bd5c8740 1 -- 172.21.15.31:0/4105548742 shutdown_connections 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.059+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5595310081b0 0x5595313c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.059+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559530fe74d0 0x5595313bad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.059+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/4105548742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fdf2f0 0x5595313b9610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.060+0000 7f43bd5c8740 1 -- 172.21.15.31:0/4105548742 >> 172.21.15.31:0/4105548742 conn(0x559530ffc300 msgr2=0x5595313ba650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.060+0000 7f43bd5c8740 1 -- 172.21.15.31:0/4105548742 shutdown_connections 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.060+0000 7f43bd5c8740 1 -- 172.21.15.31:0/4105548742 wait complete. 2024-09-21T06:12:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.061+0000 7f43bd5c8740 1 -- start start 2024-09-21T06:12:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.061+0000 7f43bd5c8740 1 Processor -- start 2024-09-21T06:12:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01bd120 con 0x559530fdf2f0 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01be590 con 0x559530fe74d0 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01bd7f0 con 0x5595310081b0 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43bd5c8740 10 client.0 _ll_register_callbacks cb 0x7f43a80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.063+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37358/0 (socket says 172.21.15.31:37358) 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 -- 172.21.15.31:0/3548165924 learned_addr learned my addr 172.21.15.31:0/3548165924 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 -- 172.21.15.31:0/3548165924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 msgr2=0x5595313cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 -- 172.21.15.31:0/3548165924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 msgr2=0x5595313caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 -- 172.21.15.31:0/3548165924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559531433710 con 0x559530fe74d0 2024-09-21T06:12:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.064+0000 7f43a77fe640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313cb470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f439801b600 tx=0x7f439801bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769b2936aef2d0ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.065+0000 7f438f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.065+0000 7f438f7fe640 1 -- 172.21.15.31:0/3548165924 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398004220 con 0x559530fe74d0 2024-09-21T06:12:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.065+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8015070 con 0x559530fe74d0 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.065+0000 7f438f7fe640 1 -- 172.21.15.31:0/3548165924 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f43980043c0 con 0x559530fe74d0 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.065+0000 7f438f7fe640 1 -- 172.21.15.31:0/3548165924 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f439801a8e0 con 0x559530fe74d0 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.065+0000 7f43bd5c8740 10 client.25331 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.066+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559531434990 con 0x559530fe74d0 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.066+0000 7f438f7fe640 1 -- 172.21.15.31:0/3548165924 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8015070 con 0x559530fe74d0 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f438f7fe640 1 -- 172.21.15.31:0/3548165924 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4398025070 con 0x559530fe74d0 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43bd5c8740 10 client.25331 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43bd5c8740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43bd5c8740 20 client.25331 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530f9c1c0 0x559531437120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43bd5c8740 4 client.25331 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:12:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595313200f0 con 0x559530f9c1c0 2024-09-21T06:12:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530f9c1c0 0x559531437120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f438f7fe640 10 client.25331 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:16.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.067+0000 7f43b4bf3640 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530f9c1c0 0x559531437120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5827941fbc73e93 server_cookie=16b7611467afe661 in_seq=0 out_seq=0 2024-09-21T06:12:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.091+0000 7f438f7fe640 1 -- 172.21.15.31:0/3548165924 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595313200f0 con 0x559530f9c1c0 2024-09-21T06:12:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.091+0000 7f438f7fe640 10 client.25331 handle_command_reply: tid=0 2024-09-21T06:12:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.091+0000 7f43bd5c8740 1 client.25331 shutdown 2024-09-21T06:12:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 msgr2=0x5595313cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313cb470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f439801b600 tx=0x7f439801bac0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 shutdown_connections 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559530f9c1c0 0x559531437120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5595310081b0 0x5595313cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559530fe74d0 0x5595313cb470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 --2- 172.21.15.31:0/3548165924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559530fdf2f0 0x5595313caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 >> 172.21.15.31:0/3548165924 conn(0x559530ffc300 msgr2=0x5595313bad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f438f7fe640 0 client.25331 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 shutdown_connections 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.092+0000 7f43bd5c8740 1 -- 172.21.15.31:0/3548165924 wait complete. 2024-09-21T06:12:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.093+0000 7f43bd5c8740 20 client.25331 trim_cache size 0 max 16384 2024-09-21T06:12:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:12:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:12:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:12:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f439c0b67d0 msgr2=0x7f439c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f439c0b67d0 0x7f439c0b8ce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43a0105ce0 tx=0x7f43a002a940 comp rx=0 tx=0).stop 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 msgr2=0x7f43b01a9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b01a9260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43a0005e30 tx=0x7f43a00309b0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 shutdown_connections 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f439c0b67d0 0x7f439c0b8ce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b0110660 0x7f43b01aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43b010b5a0 0x7f43b01a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 --2- 172.21.15.31:0/2219865708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b010ac50 0x7f43b01a9260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 >> 172.21.15.31:0/2219865708 conn(0x7f43b0071220 msgr2=0x7f43b010f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 shutdown_connections 2024-09-21T06:12:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:16.094+0000 7f43b6656640 1 -- 172.21.15.31:0/2219865708 wait complete. 2024-09-21T06:12:16.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:12:16.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:16 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1042764123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:16.616 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:16.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:16 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1042764123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:16.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:16 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1042764123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 -- 172.21.15.31:0/1558962757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1030f0 msgr2=0x7fe73c107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1558962757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1030f0 0x7fe73c107620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009ca0 tx=0x7fe72c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 -- 172.21.15.31:0/1558962757 shutdown_connections 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1558962757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 0x7fe73c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1558962757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1030f0 0x7fe73c107620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1558962757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1027a0 0x7fe73c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 -- 172.21.15.31:0/1558962757 >> 172.21.15.31:0/1558962757 conn(0x7fe73c079730 msgr2=0x7fe73c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 -- 172.21.15.31:0/1558962757 shutdown_connections 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.236+0000 7fe7412b1640 1 -- 172.21.15.31:0/1558962757 wait complete. 2024-09-21T06:12:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.237+0000 7fe7412b1640 1 Processor -- start 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.238+0000 7fe7412b1640 1 -- start start 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.238+0000 7fe7412b1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1027a0 0x7fe73c116d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe7412b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 0x7fe73c10ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe7412b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 0x7fe73c1105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe7412b1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c11a040 con 0x7fe73c1030f0 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe7412b1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c119ec0 con 0x7fe73c107be0 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe7412b1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c11a1c0 con 0x7fe73c1027a0 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 0x7fe73c10ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 0x7fe73c10ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37298/0 (socket says 172.21.15.31:37298) 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73a7fc640 1 -- 172.21.15.31:0/1582671990 learned_addr learned my addr 172.21.15.31:0/1582671990 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73affd640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1027a0 0x7fe73c116d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 0x7fe73c1105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1582671990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1027a0 msgr2=0x7fe73c116d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1027a0 0x7fe73c116d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 msgr2=0x7fe73c10ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 0x7fe73c10ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.239+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1582671990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c110df0 con 0x7fe73c107be0 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.240+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 0x7fe73c10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.240+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 0x7fe73c1105d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe72400e970 tx=0x7fe72400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.240+0000 7fe733fff640 1 -- 172.21.15.31:0/1582671990 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72400ccc0 con 0x7fe73c107be0 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.240+0000 7fe733fff640 1 -- 172.21.15.31:0/1582671990 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe724005020 con 0x7fe73c107be0 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.240+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c115aa0 con 0x7fe73c107be0 2024-09-21T06:12:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.241+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c115f50 con 0x7fe73c107be0 2024-09-21T06:12:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.241+0000 7fe733fff640 1 -- 172.21.15.31:0/1582671990 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724004a00 con 0x7fe73c107be0 2024-09-21T06:12:17.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.245+0000 7fe747825740 1 Processor -- start 2024-09-21T06:12:17.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.246+0000 7fe733fff640 1 -- 172.21.15.31:0/1582671990 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe724004ba0 con 0x7fe73c107be0 2024-09-21T06:12:17.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.246+0000 7fe747825740 1 -- start start 2024-09-21T06:12:17.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe747825740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 0x556842ae86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 0x556842ae86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 0x556842ae86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37310/0 (socket says 172.21.15.31:37310) 2024-09-21T06:12:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe747825740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842ae8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe747825740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842ae9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe747825740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556842a5eea0 con 0x556842741070 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe747825740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556842a4e2d0 con 0x556842720390 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe747825740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556842a520e0 con 0x5568427181b0 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe73a7fc640 1 -- 172.21.15.31:0/1969733348 learned_addr learned my addr 172.21.15.31:0/1969733348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.247+0000 7fe73affd640 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842ae8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842ae9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 -- 172.21.15.31:0/1969733348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 msgr2=0x556842ae8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842ae8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 -- 172.21.15.31:0/1969733348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 msgr2=0x556842ae9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842ae9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 -- 172.21.15.31:0/1969733348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568427ac650 con 0x556842741070 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 0x556842ae86a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe72c02ade0 tx=0x7fe72c009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1da63ceca6025f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.248+0000 7fe733fff640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe7280b67d0 0x7fe7280b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.249+0000 7fe733fff640 1 -- 172.21.15.31:0/1582671990 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe7240e7600 con 0x7fe73c107be0 2024-09-21T06:12:17.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.249+0000 7fe73affd640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe7280b67d0 0x7fe7280b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.249+0000 7fe7317fa640 1 -- 172.21.15.31:0/1969733348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c046020 con 0x556842741070 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.249+0000 7fe7317fa640 1 -- 172.21.15.31:0/1969733348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c036040 con 0x556842741070 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.249+0000 7fe7317fa640 1 -- 172.21.15.31:0/1969733348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c004620 con 0x556842741070 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.250+0000 7fe747825740 1 -- 172.21.15.31:0/1969733348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 msgr2=0x556842ae86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.250+0000 7fe747825740 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 0x556842ae86a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe72c02ade0 tx=0x7fe72c009410 comp rx=0 tx=0).stop 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe73affd640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe7280b67d0 0x7fe7280b8ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x556842ae7320 tx=0x7fe734005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 -- 172.21.15.31:0/1969733348 shutdown_connections 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842ae9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842ae8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 --2- 172.21.15.31:0/1969733348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842741070 0x556842ae86a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 -- 172.21.15.31:0/1969733348 >> 172.21.15.31:0/1969733348 conn(0x556842735460 msgr2=0x556842ae6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 -- 172.21.15.31:0/1969733348 shutdown_connections 2024-09-21T06:12:17.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.251+0000 7fe747825740 1 -- 172.21.15.31:0/1969733348 wait complete. 2024-09-21T06:12:17.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.252+0000 7fe747825740 1 -- start start 2024-09-21T06:12:17.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.252+0000 7fe747825740 1 Processor -- start 2024-09-21T06:12:17.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe747825740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 0x556842af3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe747825740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842af4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe747825740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842afffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842af4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842af4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37400/0 (socket says 172.21.15.31:37400) 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe73affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 0x556842af3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe73affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 0x556842af3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37312/0 (socket says 172.21.15.31:37312) 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.253+0000 7fe73a7fc640 1 -- 172.21.15.31:0/1405058231 learned_addr learned my addr 172.21.15.31:0/1405058231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842afffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5568428af700 con 0x5568427181b0 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5568429e9160 con 0x556842720390 2024-09-21T06:12:17.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556842af6ca0 con 0x556842741070 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 msgr2=0x556842af4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842af4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1405058231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 msgr2=0x556842af3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 0x556842af3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe747825740 10 client.0 _ll_register_callbacks cb 0x556842a029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556842a16c90 con 0x556842741070 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c036040 con 0x556842741070 2024-09-21T06:12:17.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.254+0000 7fe73affd640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 0x556842af3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842afffc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7240e4d90 tx=0x7fe7240acc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0fdd2024c009f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe7227fc640 10 client.25354 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe7227fc640 1 -- 172.21.15.31:0/1405058231 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72401c040 con 0x556842741070 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe7227fc640 1 -- 172.21.15.31:0/1405058231 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe7240a7b90 con 0x556842741070 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe7227fc640 1 -- 172.21.15.31:0/1405058231 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe7240a7df0 con 0x556842741070 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe747825740 10 client.25354 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.255+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x556842b02960 con 0x556842741070 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe7227fc640 1 -- 172.21.15.31:0/1405058231 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe7240ac6f0 con 0x556842741070 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe747825740 10 client.25354 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe747825740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe747825740 20 client.25354 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe747825740 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426ff9f0 0x556842b05cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe747825740 4 client.25354 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:17.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556842a60df0 con 0x5568426ff9f0 2024-09-21T06:12:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.256+0000 7fe73affd640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426ff9f0 0x556842b05cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.257+0000 7fe73affd640 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426ff9f0 0x556842b05cc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fce9ded4c8b879d server_cookie=cdd8886fce6c2403 in_seq=0 out_seq=0 2024-09-21T06:12:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.257+0000 7fe7227fc640 10 client.25354 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.258+0000 7fe7227fc640 1 -- 172.21.15.31:0/1405058231 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556842a60df0 con 0x5568426ff9f0 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.258+0000 7fe7227fc640 10 client.25354 handle_command_reply: tid=0 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.258+0000 7fe747825740 1 client.25354 shutdown 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.258+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 msgr2=0x556842afffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.258+0000 7fe747825740 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842afffc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe7240e4d90 tx=0x7fe7240acc00 comp rx=0 tx=0).stop 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 shutdown_connections 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426ff9f0 0x556842b05cc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842afffc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842720390 0x556842af4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 --2- 172.21.15.31:0/1405058231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568427181b0 0x556842af3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 >> 172.21.15.31:0/1405058231 conn(0x556842735460 msgr2=0x556842aeb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe7227fc640 0 client.25354 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 shutdown_connections 2024-09-21T06:12:17.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.259+0000 7fe747825740 1 -- 172.21.15.31:0/1405058231 wait complete. 2024-09-21T06:12:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.260+0000 7fe747825740 20 client.25354 trim_cache size 0 max 16384 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.286+0000 7fe747825740 1 Processor -- start 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.286+0000 7fe747825740 1 -- start start 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe747825740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842af5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe747825740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842ae9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe73affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842af5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe747825740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842741070 0x556842affec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe73affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842af5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:32846/0 (socket says 172.21.15.31:32846) 2024-09-21T06:12:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe73affd640 1 -- 172.21.15.31:0/1007735312 learned_addr learned my addr 172.21.15.31:0/1007735312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.287+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842ae9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.288+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c116810 con 0x556842720390 2024-09-21T06:12:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.288+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1c7560 con 0x556842741070 2024-09-21T06:12:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.288+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c0734f0 con 0x5568427181b0 2024-09-21T06:12:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.288+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842741070 0x556842affec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.288+0000 7fe73affd640 1 -- 172.21.15.31:0/1007735312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842741070 msgr2=0x556842affec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.289+0000 7fe73affd640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842741070 0x556842affec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.289+0000 7fe73affd640 1 -- 172.21.15.31:0/1007735312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 msgr2=0x556842ae9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.289+0000 7fe73affd640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842ae9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.289+0000 7fe73affd640 1 -- 172.21.15.31:0/1007735312 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556842aeea70 con 0x5568427181b0 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.289+0000 7fe73a7fc640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842ae9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.290+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842741070 0x556842affec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.290+0000 7fe73affd640 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842af5cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe734009d80 tx=0x7fe734016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f93f18bb598396 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.290+0000 7fe730ff9640 1 -- 172.21.15.31:0/1007735312 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734004230 con 0x5568427181b0 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.290+0000 7fe730ff9640 1 -- 172.21.15.31:0/1007735312 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7340043d0 con 0x5568427181b0 2024-09-21T06:12:17.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.291+0000 7fe730ff9640 1 -- 172.21.15.31:0/1007735312 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73402a760 con 0x5568427181b0 2024-09-21T06:12:17.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.292+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 msgr2=0x556842af5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.292+0000 7fe747825740 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842af5cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe734009d80 tx=0x7fe734016000 comp rx=0 tx=0).stop 2024-09-21T06:12:17.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.293+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 shutdown_connections 2024-09-21T06:12:17.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.293+0000 7fe747825740 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556842741070 0x556842affec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.293+0000 7fe747825740 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842ae9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.293+0000 7fe747825740 1 --2- 172.21.15.31:0/1007735312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5568427181b0 0x556842af5cb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.293+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 >> 172.21.15.31:0/1007735312 conn(0x556842735460 msgr2=0x556842ae96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.294+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 shutdown_connections 2024-09-21T06:12:17.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.294+0000 7fe747825740 1 -- 172.21.15.31:0/1007735312 wait complete. 2024-09-21T06:12:17.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 -- start start 2024-09-21T06:12:17.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 Processor -- start 2024-09-21T06:12:17.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 0x556842afa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842afafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842b68590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c073b40 con 0x556842720390 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c074fb0 con 0x5568427181b0 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c074210 con 0x556842741070 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe747825740 10 client.0 _ll_register_callbacks cb 0x7fe72400cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe73affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 0x556842afa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 0x556842afa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37418/0 (socket says 172.21.15.31:37418) 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73affd640 1 -- 172.21.15.31:0/1668578704 learned_addr learned my addr 172.21.15.31:0/1668578704 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842b68590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.295+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842afafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 msgr2=0x556842afa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 0x556842afa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1668578704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 msgr2=0x556842afafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842afafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 -- 172.21.15.31:0/1668578704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556842b68c00 con 0x556842741070 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73affd640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 0x556842afa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe73b7fe640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842b68590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe7240aa700 tx=0x7fe724005460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25706506d6303114 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.296+0000 7fe7227fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.297+0000 7fe7227fc640 1 -- 172.21.15.31:0/1668578704 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72401c040 con 0x556842741070 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.297+0000 7fe7227fc640 1 -- 172.21.15.31:0/1668578704 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe7240c8ae0 con 0x556842741070 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.297+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7340043d0 con 0x556842741070 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe7227fc640 1 -- 172.21.15.31:0/1668578704 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe7240ad260 con 0x556842741070 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe747825740 10 client.25360 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:17.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556842b71120 con 0x556842741070 2024-09-21T06:12:17.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe7227fc640 1 -- 172.21.15.31:0/1668578704 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72c036040 con 0x556842741070 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe7227fc640 1 -- 172.21.15.31:0/1668578704 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe7240c3240 con 0x556842741070 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe747825740 10 client.25360 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.298+0000 7fe747825740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.299+0000 7fe747825740 20 client.25360 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.299+0000 7fe747825740 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426d4da0 0x556842b73840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.299+0000 7fe747825740 4 client.25360 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.299+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe7340043d0 con 0x5568426d4da0 2024-09-21T06:12:17.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.299+0000 7fe73affd640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426d4da0 0x556842b73840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:17.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.300+0000 7fe7227fc640 10 client.25360 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:17.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.300+0000 7fe73affd640 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426d4da0 0x556842b73840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7cbdbf1327d9627 server_cookie=7f670f0d10800d55 in_seq=0 out_seq=0 2024-09-21T06:12:17.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe7227fc640 1 -- 172.21.15.31:0/1668578704 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe7340043d0 con 0x5568426d4da0 2024-09-21T06:12:17.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe7227fc640 10 client.25360 handle_command_reply: tid=0 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 client.25360 shutdown 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 msgr2=0x556842b68590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842b68590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe7240aa700 tx=0x7fe724005460 comp rx=0 tx=0).stop 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 shutdown_connections 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5568426d4da0 0x556842b73840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556842741070 0x556842b68590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556842720390 0x556842afafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 --2- 172.21.15.31:0/1668578704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568427181b0 0x556842afa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 >> 172.21.15.31:0/1668578704 conn(0x556842735460 msgr2=0x556842af5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe7227fc640 0 client.25360 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.301+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 shutdown_connections 2024-09-21T06:12:17.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.302+0000 7fe747825740 1 -- 172.21.15.31:0/1668578704 wait complete. 2024-09-21T06:12:17.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.302+0000 7fe747825740 20 client.25360 trim_cache size 0 max 16384 2024-09-21T06:12:17.306 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.303+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe7280b67d0 msgr2=0x7fe7280b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.303+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe7280b67d0 0x7fe7280b8ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x556842ae7320 tx=0x7fe734005f90 comp rx=0 tx=0).stop 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.303+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 msgr2=0x7fe73c1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.303+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 0x7fe73c1105d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe72400e970 tx=0x7fe72400ee30 comp rx=0 tx=0).stop 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.303+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 shutdown_connections 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.304+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe73c107be0 0x7fe73c1105d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.304+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe7280b67d0 0x7fe7280b8ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.304+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe73c1030f0 0x7fe73c10ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.304+0000 7fe7412b1640 1 --2- 172.21.15.31:0/1582671990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe73c1027a0 0x7fe73c116d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:17.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.304+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 >> 172.21.15.31:0/1582671990 conn(0x7fe73c079730 msgr2=0x7fe73c1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:17.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.305+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 shutdown_connections 2024-09-21T06:12:17.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:17.305+0000 7fe7412b1640 1 -- 172.21.15.31:0/1582671990 wait complete. 2024-09-21T06:12:17.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:17 smithi031 ceph-mon[31237]: pgmap v340: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 56 MiB/s wr, 1.68k op/s 2024-09-21T06:12:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:17 smithi110 ceph-mon[34550]: pgmap v340: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 56 MiB/s wr, 1.68k op/s 2024-09-21T06:12:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:17 smithi107 ceph-mon[34380]: pgmap v340: 97 pgs: 97 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 56 MiB/s wr, 1.68k op/s 2024-09-21T06:12:19.145 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:12:19.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:12:19.591 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:19 smithi031 ceph-mon[31237]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 MiB/s wr, 1.13k op/s 2024-09-21T06:12:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:19 smithi110 ceph-mon[34550]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 MiB/s wr, 1.13k op/s 2024-09-21T06:12:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:19 smithi107 ceph-mon[34380]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 MiB/s wr, 1.13k op/s 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.172+0000 7f45c2801640 1 -- 172.21.15.31:0/2331997669 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc1083a0 msgr2=0x7f45bc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.172+0000 7f45c2801640 1 --2- 172.21.15.31:0/2331997669 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc1083a0 0x7f45bc1087b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009900 tx=0x7f45b002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 -- 172.21.15.31:0/2331997669 shutdown_connections 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 --2- 172.21.15.31:0/2331997669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45bc108cf0 0x7f45bc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 --2- 172.21.15.31:0/2331997669 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc1083a0 0x7f45bc1087b0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009900 tx=0x7f45b002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 --2- 172.21.15.31:0/2331997669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc102a60 0x7f45bc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 -- 172.21.15.31:0/2331997669 >> 172.21.15.31:0/2331997669 conn(0x7f45bc079730 msgr2=0x7f45bc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 -- 172.21.15.31:0/2331997669 shutdown_connections 2024-09-21T06:12:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.173+0000 7f45c2801640 1 -- 172.21.15.31:0/2331997669 wait complete. 2024-09-21T06:12:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.174+0000 7f45c2801640 1 Processor -- start 2024-09-21T06:12:20.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.174+0000 7f45c2801640 1 -- start start 2024-09-21T06:12:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45c2801640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 0x7f45bc073810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45c2801640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc108cf0 0x7f45bc073d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45c2801640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45bc074290 0x7f45bc1b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45c2801640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc112960 con 0x7f45bc108cf0 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45c2801640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc1127e0 con 0x7f45bc074290 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45c2801640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc112ae0 con 0x7f45bc102a60 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 0x7f45bc073810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 0x7f45bc073810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43466/0 (socket says 172.21.15.31:43466) 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45bbfff640 1 -- 172.21.15.31:0/3192788987 learned_addr learned my addr 172.21.15.31:0/3192788987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.175+0000 7f45bb7fe640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc108cf0 0x7f45bc073d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.176+0000 7f45bbfff640 1 -- 172.21.15.31:0/3192788987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45bc074290 msgr2=0x7f45bc1b0e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.176+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45bc074290 0x7f45bc1b0e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.176+0000 7f45bbfff640 1 -- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc108cf0 msgr2=0x7f45bc073d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.176+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc108cf0 0x7f45bc073d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.176+0000 7f45bbfff640 1 -- 172.21.15.31:0/3192788987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc1b1520 con 0x7f45bc102a60 2024-09-21T06:12:20.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.177+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 0x7f45bc073810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ea20 tx=0x7f45ac00eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.177+0000 7f45b97fa640 1 -- 172.21.15.31:0/3192788987 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00cd80 con 0x7f45bc102a60 2024-09-21T06:12:20.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.177+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc1b1810 con 0x7f45bc102a60 2024-09-21T06:12:20.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.177+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc1b1d70 con 0x7f45bc102a60 2024-09-21T06:12:20.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.180+0000 7f45b97fa640 1 -- 172.21.15.31:0/3192788987 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0108e0 con 0x7f45bc102a60 2024-09-21T06:12:20.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.180+0000 7f45b97fa640 1 -- 172.21.15.31:0/3192788987 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac018860 con 0x7f45bc102a60 2024-09-21T06:12:20.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.182+0000 7f45c8d75740 1 Processor -- start 2024-09-21T06:12:20.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.182+0000 7f45c8d75740 1 -- start start 2024-09-21T06:12:20.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d50350 0x556f440f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 0x556f440f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45c8d75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556f4406de60 con 0x556f43d27490 2024-09-21T06:12:20.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45c8d75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556f4405d290 con 0x556f43d2f670 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45c8d75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556f440610a0 con 0x556f43d50350 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 0x556f440f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 0x556f440f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34328/0 (socket says 172.21.15.31:34328) 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.183+0000 7f45bbfff640 1 -- 172.21.15.31:0/3094765157 learned_addr learned my addr 172.21.15.31:0/3094765157 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bbfff640 1 -- 172.21.15.31:0/3094765157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d50350 msgr2=0x556f440f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d50350 0x556f440f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bb7fe640 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bbfff640 1 -- 172.21.15.31:0/3094765157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 msgr2=0x556f440f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bbfff640 1 -- 172.21.15.31:0/3094765157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f43dbb930 con 0x556f43d27490 2024-09-21T06:12:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bb7fe640 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:20.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.184+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 0x556f440f7970 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00f010 tx=0x7f45ac010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0334b272e4b7033 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.187+0000 7f45b97fa640 1 -- 172.21.15.31:0/3192788987 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f45ac056030 con 0x7f45bc102a60 2024-09-21T06:12:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.187+0000 7f45a27fc640 1 -- 172.21.15.31:0/3094765157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0ea210 con 0x556f43d27490 2024-09-21T06:12:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.187+0000 7f45a27fc640 1 -- 172.21.15.31:0/3094765157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0eaa50 con 0x556f43d27490 2024-09-21T06:12:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.187+0000 7f45a27fc640 1 -- 172.21.15.31:0/3094765157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0ebd90 con 0x556f43d27490 2024-09-21T06:12:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.188+0000 7f45c8d75740 1 -- 172.21.15.31:0/3094765157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 msgr2=0x556f440f7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.188+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 0x556f440f7970 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00f010 tx=0x7f45ac010e60 comp rx=0 tx=0).stop 2024-09-21T06:12:20.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.188+0000 7f45b97fa640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45a40b6680 0x7f45a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.189+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45a40b6680 0x7f45a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.189+0000 7f45b97fa640 1 -- 172.21.15.31:0/3192788987 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f45ac054070 con 0x7f45bc102a60 2024-09-21T06:12:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.190+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45a40b6680 0x7f45a40b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005fd0 tx=0x7f45b4009940 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 -- 172.21.15.31:0/3094765157 shutdown_connections 2024-09-21T06:12:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d27490 0x556f440f7970 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3094765157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d50350 0x556f440f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 -- 172.21.15.31:0/3094765157 >> 172.21.15.31:0/3094765157 conn(0x556f43d44740 msgr2=0x556f440f5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 -- 172.21.15.31:0/3094765157 shutdown_connections 2024-09-21T06:12:20.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 -- 172.21.15.31:0/3094765157 wait complete. 2024-09-21T06:12:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.191+0000 7f45c8d75740 1 -- start start 2024-09-21T06:12:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.192+0000 7f45c8d75740 1 Processor -- start 2024-09-21T06:12:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.192+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d50350 0x556f441176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45c8d75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556f43ebe890 con 0x556f43d2f670 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45c8d75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556f43ff7d40 con 0x556f43d50350 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34340/0 (socket says 172.21.15.31:34340) 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45bb7fe640 1 -- 172.21.15.31:0/2183764040 learned_addr learned my addr 172.21.15.31:0/2183764040 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45bbfff640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556f4410e2e0 con 0x556f43d27490 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.193+0000 7f45c8d75740 10 client.0 _ll_register_callbacks cb 0x556f4405bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.194+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d50350 0x556f441176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.195+0000 7f45c0d9e640 1 -- 172.21.15.31:0/2183764040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 msgr2=0x556f4410b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.195+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.195+0000 7f45c0d9e640 1 -- 172.21.15.31:0/2183764040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 msgr2=0x556f4410bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.195+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.195+0000 7f45c0d9e640 1 -- 172.21.15.31:0/2183764040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f44117be0 con 0x556f43d50350 2024-09-21T06:12:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.196+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d50350 0x556f441176a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45b4010480 tx=0x7f45b4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c22785725b7f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.196+0000 7f459b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.196+0000 7f459b7fe640 1 -- 172.21.15.31:0/2183764040 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4021020 con 0x556f43d50350 2024-09-21T06:12:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.196+0000 7f459b7fe640 1 -- 172.21.15.31:0/2183764040 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f45b4004d80 con 0x556f43d50350 2024-09-21T06:12:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.196+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac0eaa50 con 0x556f43d50350 2024-09-21T06:12:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.198+0000 7f459b7fe640 1 -- 172.21.15.31:0/2183764040 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f45b402c050 con 0x556f43d50350 2024-09-21T06:12:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.198+0000 7f45c8d75740 10 client.25355 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.198+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556f4411a470 con 0x556f43d50350 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.198+0000 7f459b7fe640 1 -- 172.21.15.31:0/2183764040 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45b40054e0 con 0x556f43d50350 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f459b7fe640 1 -- 172.21.15.31:0/2183764040 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f45b401d800 con 0x556f43d50350 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45c8d75740 10 client.25355 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45c8d75740 10 client.25355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45c8d75740 20 client.25355 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d0ecd0 0x556f4411d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45c8d75740 4 client.25355 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45ac0eaa50 con 0x556f43d0ecd0 2024-09-21T06:12:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.199+0000 7f45bbfff640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d0ecd0 0x556f4411d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.200+0000 7f459b7fe640 10 client.25355 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:20.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.200+0000 7f45bbfff640 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d0ecd0 0x556f4411d940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73abff987384aa7a server_cookie=e12ea137f3fdb9a8 in_seq=0 out_seq=0 2024-09-21T06:12:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.201+0000 7f459b7fe640 1 -- 172.21.15.31:0/2183764040 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f45ac0eaa50 con 0x556f43d0ecd0 2024-09-21T06:12:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.201+0000 7f459b7fe640 10 client.25355 handle_command_reply: tid=0 2024-09-21T06:12:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.201+0000 7f45c8d75740 1 client.25355 shutdown 2024-09-21T06:12:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 40.0 s, 4535.4 tps, lat 21.938 ms stddev 22.645 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d50350 msgr2=0x556f441176a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d50350 0x556f441176a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45b4010480 tx=0x7f45b4004a40 comp rx=0 tx=0).stop 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 shutdown_connections 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d0ecd0 0x556f4411d940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d50350 0x556f441176a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f459b7fe640 0 client.25355 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2183764040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 >> 172.21.15.31:0/2183764040 conn(0x556f43d44740 msgr2=0x556f440f3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 shutdown_connections 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 1 -- 172.21.15.31:0/2183764040 wait complete. 2024-09-21T06:12:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.202+0000 7f45c8d75740 20 client.25355 trim_cache size 0 max 16384 2024-09-21T06:12:20.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.231+0000 7f45c8d75740 1 Processor -- start 2024-09-21T06:12:20.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 -- start start 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 0x556f4410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f441175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800c7d0 con 0x556f43d0ecd0 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800f050 con 0x556f43d2f670 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.232+0000 7f45c8d75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800ce40 con 0x556f43d27490 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 0x556f4410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 0x556f4410a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34354/0 (socket says 172.21.15.31:34354) 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 -- 172.21.15.31:0/2774756072 learned_addr learned my addr 172.21.15.31:0/2774756072 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:20.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45c0d9e640 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 -- 172.21.15.31:0/2774756072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 msgr2=0x556f441175a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:12:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f441175a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 -- 172.21.15.31:0/2774756072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 msgr2=0x556f440f8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.233+0000 7f45bbfff640 1 -- 172.21.15.31:0/2774756072 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f440f0aa0 con 0x556f43d0ecd0 2024-09-21T06:12:20.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.234+0000 7f45bbfff640 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 0x556f4410a3d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0ad120 tx=0x7f45ac009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ff80c6c8c17206 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.236+0000 7f45a1ffb640 1 -- 172.21.15.31:0/2774756072 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac053040 con 0x556f43d0ecd0 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.236+0000 7f45a1ffb640 1 -- 172.21.15.31:0/2774756072 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0af9c0 con 0x556f43d0ecd0 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.236+0000 7f45a1ffb640 1 -- 172.21.15.31:0/2774756072 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac053040 con 0x556f43d0ecd0 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 -- 172.21.15.31:0/2774756072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 msgr2=0x556f4410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 0x556f4410a3d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0ad120 tx=0x7f45ac009700 comp rx=0 tx=0).stop 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 -- 172.21.15.31:0/2774756072 shutdown_connections 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d2f670 0x556f440f8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f441175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 --2- 172.21.15.31:0/2774756072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d0ecd0 0x556f4410a3d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.237+0000 7f45c8d75740 1 -- 172.21.15.31:0/2774756072 >> 172.21.15.31:0/2774756072 conn(0x556f43d44740 msgr2=0x556f440f8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.238+0000 7f45c8d75740 1 -- 172.21.15.31:0/2774756072 shutdown_connections 2024-09-21T06:12:20.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.238+0000 7f45c8d75740 1 -- 172.21.15.31:0/2774756072 wait complete. 2024-09-21T06:12:20.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.238+0000 7f45c8d75740 1 -- start start 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 Processor -- start 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d0ecd0 0x556f4410e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800d510 con 0x556f43d2f670 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800e980 con 0x556f43d0ecd0 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800dbe0 con 0x556f43d27490 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c8d75740 10 client.0 _ll_register_callbacks cb 0x7f45b4014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.239+0000 7f45c0d9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.240+0000 7f45bb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.240+0000 7f45c0d9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34360/0 (socket says 172.21.15.31:34360) 2024-09-21T06:12:20.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.240+0000 7f45c0d9e640 1 -- 172.21.15.31:0/3945851755 learned_addr learned my addr 172.21.15.31:0/3945851755 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.240+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d0ecd0 0x556f4410e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.241+0000 7f45bbfff640 1 -- 172.21.15.31:0/3945851755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 msgr2=0x556f4410eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.241+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.241+0000 7f45bbfff640 1 -- 172.21.15.31:0/3945851755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 msgr2=0x556f4410f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:20.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.241+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.241+0000 7f45bbfff640 1 -- 172.21.15.31:0/3945851755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f441037c0 con 0x556f43d0ecd0 2024-09-21T06:12:20.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.241+0000 7f45bb7fe640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.244+0000 7f45bbfff640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d0ecd0 0x556f4410e610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0ad120 tx=0x7f45ac005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92d2762a237d6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.244+0000 7f459b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.244+0000 7f459b7fe640 1 -- 172.21.15.31:0/3945851755 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0c9030 con 0x556f43d0ecd0 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.245+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac0af9c0 con 0x556f43d0ecd0 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.245+0000 7f459b7fe640 1 -- 172.21.15.31:0/3945851755 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f45ac0eb4b0 con 0x556f43d0ecd0 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.245+0000 7f459b7fe640 1 -- 172.21.15.31:0/3945851755 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f45ac053040 con 0x556f43d0ecd0 2024-09-21T06:12:20.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.245+0000 7f45c8d75740 10 client.25361 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.245+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556f44104a40 con 0x556f43d0ecd0 2024-09-21T06:12:20.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f459b7fe640 1 -- 172.21.15.31:0/3945851755 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45ac0af9c0 con 0x556f43d0ecd0 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f459b7fe640 1 -- 172.21.15.31:0/3945851755 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f45ac0ac120 con 0x556f43d0ecd0 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45c8d75740 10 client.25361 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45c8d75740 10 client.25361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45c8d75740 20 client.25361 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d50350 0x556f44104d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45c8d75740 4 client.25361 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f4406fdb0 con 0x556f43d50350 2024-09-21T06:12:20.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.246+0000 7f45bb7fe640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d50350 0x556f44104d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.248+0000 7f45bb7fe640 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d50350 0x556f44104d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c791d8985201c122 server_cookie=19cc262aec8db47 in_seq=0 out_seq=0 2024-09-21T06:12:20.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.248+0000 7f459b7fe640 10 client.25361 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.251+0000 7f459b7fe640 1 -- 172.21.15.31:0/3945851755 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f4406fdb0 con 0x556f43d50350 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.251+0000 7f459b7fe640 10 client.25361 handle_command_reply: tid=0 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 client.25361 shutdown 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d0ecd0 msgr2=0x556f4410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d0ecd0 0x556f4410e610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0ad120 tx=0x7f45ac005ab0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 shutdown_connections 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556f43d50350 0x556f44104d80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f43d2f670 0x556f4410f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f43d27490 0x556f4410eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 --2- 172.21.15.31:0/3945851755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556f43d0ecd0 0x556f4410e610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 >> 172.21.15.31:0/3945851755 conn(0x556f43d44740 msgr2=0x556f4410d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.252+0000 7f459b7fe640 0 client.25361 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:20.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.253+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 shutdown_connections 2024-09-21T06:12:20.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.253+0000 7f45c8d75740 1 -- 172.21.15.31:0/3945851755 wait complete. 2024-09-21T06:12:20.257 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:12:20.257 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:12:20.257 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "8107147d-48c1-4246-823e-f82266cad4b4", 2024-09-21T06:12:20.257 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:12:20.257 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:12:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.253+0000 7f45c8d75740 20 client.25361 trim_cache size 0 max 16384 2024-09-21T06:12:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45a40b6680 msgr2=0x7f45a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45a40b6680 0x7f45a40b8b90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005fd0 tx=0x7f45b4009940 comp rx=0 tx=0).stop 2024-09-21T06:12:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 msgr2=0x7f45bc073810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:20.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 0x7f45bc073810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00ea20 tx=0x7f45ac00eee0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 shutdown_connections 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45bc074290 0x7f45bc1b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f45a40b6680 0x7f45a40b8b90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45bc108cf0 0x7f45bc073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 --2- 172.21.15.31:0/3192788987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f45bc102a60 0x7f45bc073810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 >> 172.21.15.31:0/3192788987 conn(0x7f45bc079730 msgr2=0x7f45bc101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 shutdown_connections 2024-09-21T06:12:20.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:20.254+0000 7f45c2801640 1 -- 172.21.15.31:0/3192788987 wait complete. 2024-09-21T06:12:21.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:12:21.545 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:21 smithi031 ceph-mon[31237]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-09-21T06:12:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:21 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:12:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:21 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:21 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:12:21.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:21 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:21 smithi110 ceph-mon[34550]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-09-21T06:12:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:21 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:12:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:21 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:21 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:12:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:21 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:21 smithi107 ceph-mon[34380]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-09-21T06:12:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:21 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:12:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:21 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:21 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:12:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:21 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.139+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/3143882237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48102a60 msgr2=0x7f1f48102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.139+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/3143882237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48102a60 0x7f1f48102e70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009be0 tx=0x7f1f4002a6d0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/3143882237 shutdown_connections 2024-09-21T06:12:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/3143882237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f48108cf0 0x7f1f4810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/3143882237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f481083a0 0x7f1f481087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/3143882237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f48102a60 0x7f1f48102e70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/3143882237 >> 172.21.15.31:0/3143882237 conn(0x7f1f48079730 msgr2=0x7f1f48079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/3143882237 shutdown_connections 2024-09-21T06:12:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.140+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/3143882237 wait complete. 2024-09-21T06:12:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.141+0000 7f1f4e3e3640 1 Processor -- start 2024-09-21T06:12:22.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.141+0000 7f1f4e3e3640 1 -- start start 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f4e3e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f48102a60 0x7f1f481119c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f4e3e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 0x7f1f4810ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f4e3e3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f48108cf0 0x7f1f481104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f4e3e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48113ab0 con 0x7f1f48102a60 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f4e3e3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48113930 con 0x7f1f481083a0 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f4e3e3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48113c30 con 0x7f1f48108cf0 2024-09-21T06:12:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 0x7f1f4810ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 0x7f1f4810ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38608/0 (socket says 172.21.15.31:38608) 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.142+0000 7f1f477fe640 1 -- 172.21.15.31:0/1918868481 learned_addr learned my addr 172.21.15.31:0/1918868481 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f48108cf0 0x7f1f481104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f477fe640 1 -- 172.21.15.31:0/1918868481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f48108cf0 msgr2=0x7f1f481104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f477fe640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f48108cf0 0x7f1f481104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f477fe640 1 -- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f48102a60 msgr2=0x7f1f481119c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f477fe640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f48102a60 0x7f1f481119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f477fe640 1 -- 172.21.15.31:0/1918868481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f48110d10 con 0x7f1f481083a0 2024-09-21T06:12:22.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.143+0000 7f1f477fe640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 0x7f1f4810ffb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e9c0 tx=0x7f1f3c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.144+0000 7f1f457fa640 1 -- 172.21.15.31:0/1918868481 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00ccc0 con 0x7f1f481083a0 2024-09-21T06:12:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.144+0000 7f1f457fa640 1 -- 172.21.15.31:0/1918868481 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c0054d0 con 0x7f1f481083a0 2024-09-21T06:12:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.144+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f481accb0 con 0x7f1f481083a0 2024-09-21T06:12:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.144+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481ad1e0 con 0x7f1f481083a0 2024-09-21T06:12:22.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.145+0000 7f1f457fa640 1 -- 172.21.15.31:0/1918868481 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c010430 con 0x7f1f481083a0 2024-09-21T06:12:22.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.150+0000 7f1f457fa640 1 -- 172.21.15.31:0/1918868481 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1f3c010690 con 0x7f1f481083a0 2024-09-21T06:12:22.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.151+0000 7f1f54957740 1 Processor -- start 2024-09-21T06:12:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.152+0000 7f1f54957740 1 -- start start 2024-09-21T06:12:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.152+0000 7f1f54957740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0b13070 0x55dea0eba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f54957740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ebab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f54957740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ebb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f54957740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dea0e30e50 con 0x55dea0b13070 2024-09-21T06:12:22.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f54957740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dea0e20280 con 0x55dea0af2390 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f54957740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dea0e24090 con 0x55dea0aea1b0 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ebab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ebab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43532/0 (socket says 172.21.15.31:43532) 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 -- 172.21.15.31:0/657502871 learned_addr learned my addr 172.21.15.31:0/657502871 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f4c980640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0b13070 0x55dea0eba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f457fa640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f300b6730 0x7f1f300b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f477fe640 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ebb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 -- 172.21.15.31:0/657502871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 msgr2=0x55dea0ebb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ebb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 -- 172.21.15.31:0/657502871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0b13070 msgr2=0x55dea0eba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f47fff640 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0b13070 0x55dea0eba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.153+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f300b6730 0x7f1f300b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.154+0000 7f1f47fff640 1 -- 172.21.15.31:0/657502871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea0b7e650 con 0x55dea0aea1b0 2024-09-21T06:12:22.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.154+0000 7f1f457fa640 1 -- 172.21.15.31:0/1918868481 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1f3c0e7a90 con 0x7f1f481083a0 2024-09-21T06:12:22.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.154+0000 7f1f47fff640 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ebab90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f4002acc0 tx=0x7f1f4000b050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d06b74cdbbd8522d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.155+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f300b6730 0x7f1f300b8c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f48112070 tx=0x7f1f38004730 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.156+0000 7f1f2e7fc640 1 -- 172.21.15.31:0/657502871 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40040020 con 0x55dea0aea1b0 2024-09-21T06:12:22.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.157+0000 7f1f2e7fc640 1 -- 172.21.15.31:0/657502871 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f40003e20 con 0x55dea0aea1b0 2024-09-21T06:12:22.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.157+0000 7f1f2e7fc640 1 -- 172.21.15.31:0/657502871 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40036040 con 0x55dea0aea1b0 2024-09-21T06:12:22.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.158+0000 7f1f54957740 1 -- 172.21.15.31:0/657502871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 msgr2=0x55dea0ebab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.158+0000 7f1f54957740 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ebab90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f4002acc0 tx=0x7f1f4000b050 comp rx=0 tx=0).stop 2024-09-21T06:12:22.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.158+0000 7f1f54957740 1 -- 172.21.15.31:0/657502871 shutdown_connections 2024-09-21T06:12:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ebb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ebab90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 --2- 172.21.15.31:0/657502871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0b13070 0x55dea0eba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 -- 172.21.15.31:0/657502871 >> 172.21.15.31:0/657502871 conn(0x55dea0b07460 msgr2=0x55dea0eb8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 -- 172.21.15.31:0/657502871 shutdown_connections 2024-09-21T06:12:22.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 -- 172.21.15.31:0/657502871 wait complete. 2024-09-21T06:12:22.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.159+0000 7f1f54957740 1 -- start start 2024-09-21T06:12:22.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.160+0000 7f1f54957740 1 Processor -- start 2024-09-21T06:12:22.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0aea1b0 0x55dea0ec5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0ec6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dea0c81700 con 0x55dea0af2390 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dea0dbb110 con 0x55dea0aea1b0 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dea0ec8e20 con 0x55dea0b13070 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f54957740 10 client.0 _ll_register_callbacks cb 0x55dea0dd4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34386/0 (socket says 172.21.15.31:34386) 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f477fe640 1 -- 172.21.15.31:0/2097310808 learned_addr learned my addr 172.21.15.31:0/2097310808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f4c980640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0ec6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.162+0000 7f1f47fff640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0aea1b0 0x55dea0ec5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.163+0000 7f1f4c980640 1 -- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0aea1b0 msgr2=0x55dea0ec5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.163+0000 7f1f4c980640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0aea1b0 0x55dea0ec5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.163+0000 7f1f4c980640 1 -- 172.21.15.31:0/2097310808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 msgr2=0x55dea0ec61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.163+0000 7f1f4c980640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.163+0000 7f1f4c980640 1 -- 172.21.15.31:0/2097310808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea0de8c40 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.164+0000 7f1f4c980640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0ec6720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005fd0 tx=0x7f1f380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667ecfc304fa88bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.164+0000 7f1f277fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.164+0000 7f1f277fe640 1 -- 172.21.15.31:0/2097310808 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3802a620 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.164+0000 7f1f277fe640 1 -- 172.21.15.31:0/2097310808 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1f3802a7c0 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.164+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f40003e20 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f277fe640 1 -- 172.21.15.31:0/2097310808 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1f3802a9c0 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 10 client.25384 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dea0ed4b90 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f277fe640 1 -- 172.21.15.31:0/2097310808 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f40003e20 con 0x55dea0b13070 2024-09-21T06:12:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f277fe640 1 -- 172.21.15.31:0/2097310808 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1f380209c0 con 0x55dea0b13070 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 10 client.25384 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 20 client.25384 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0ad19f0 0x55dea0ed8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 4 client.25384 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.165+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea0e32da0 con 0x55dea0ad19f0 2024-09-21T06:12:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.166+0000 7f1f47fff640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0ad19f0 0x55dea0ed8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.167+0000 7f1f277fe640 10 client.25384 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.167+0000 7f1f47fff640 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0ad19f0 0x55dea0ed8120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14e64959e24fae6 server_cookie=ae19588b4ba5ba99 in_seq=0 out_seq=0 2024-09-21T06:12:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f277fe640 1 -- 172.21.15.31:0/2097310808 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dea0e32da0 con 0x55dea0ad19f0 2024-09-21T06:12:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f277fe640 10 client.25384 handle_command_reply: tid=0 2024-09-21T06:12:22.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 client.25384 shutdown 2024-09-21T06:12:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 msgr2=0x55dea0ec6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0ec6720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005fd0 tx=0x7f1f380046c0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 shutdown_connections 2024-09-21T06:12:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0ad19f0 0x55dea0ed8120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0ec6720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 --2- 172.21.15.31:0/2097310808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0aea1b0 0x55dea0ec5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 >> 172.21.15.31:0/2097310808 conn(0x55dea0b07460 msgr2=0x55dea0ebd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.168+0000 7f1f277fe640 0 client.25384 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.169+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 shutdown_connections 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.169+0000 7f1f54957740 1 -- 172.21.15.31:0/2097310808 wait complete. 2024-09-21T06:12:22.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.169+0000 7f1f54957740 20 client.25384 trim_cache size 0 max 16384 2024-09-21T06:12:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.202+0000 7f1f54957740 1 Processor -- start 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.202+0000 7f1f54957740 1 -- start start 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.202+0000 7f1f54957740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ec7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.202+0000 7f1f54957740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.202+0000 7f1f54957740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0b13070 0x55dea0ec66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f54957740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3400c6f0 con 0x55dea0af2390 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f54957740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3400ef70 con 0x55dea0b13070 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f54957740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3400cd60 con 0x55dea0aea1b0 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ec7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ec7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43550/0 (socket says 172.21.15.31:43550) 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 -- 172.21.15.31:0/619191564 learned_addr learned my addr 172.21.15.31:0/619191564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f477fe640 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f4c980640 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0b13070 0x55dea0ec66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 -- 172.21.15.31:0/619191564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0b13070 msgr2=0x55dea0ec66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0b13070 0x55dea0ec66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 -- 172.21.15.31:0/619191564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 msgr2=0x55dea0ec6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.203+0000 7f1f47fff640 1 -- 172.21.15.31:0/619191564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea0ec6f60 con 0x55dea0aea1b0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.204+0000 7f1f47fff640 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ec7b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f4002ac40 tx=0x7f1f40030c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765d852012299ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.204+0000 7f1f4c980640 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0b13070 0x55dea0ec66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.204+0000 7f1f2dffb640 1 -- 172.21.15.31:0/619191564 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40040040 con 0x55dea0aea1b0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.204+0000 7f1f2dffb640 1 -- 172.21.15.31:0/619191564 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f40003970 con 0x55dea0aea1b0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.204+0000 7f1f2dffb640 1 -- 172.21.15.31:0/619191564 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40036040 con 0x55dea0aea1b0 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.205+0000 7f1f54957740 1 -- 172.21.15.31:0/619191564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 msgr2=0x55dea0ec7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.205+0000 7f1f54957740 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ec7b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f4002ac40 tx=0x7f1f40030c60 comp rx=0 tx=0).stop 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.205+0000 7f1f54957740 1 -- 172.21.15.31:0/619191564 shutdown_connections 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.206+0000 7f1f54957740 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0b13070 0x55dea0ec66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.206+0000 7f1f54957740 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0af2390 0x55dea0ec6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.206+0000 7f1f54957740 1 --2- 172.21.15.31:0/619191564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0aea1b0 0x55dea0ec7b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.206+0000 7f1f54957740 1 -- 172.21.15.31:0/619191564 >> 172.21.15.31:0/619191564 conn(0x55dea0b07460 msgr2=0x55dea0ec5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.206+0000 7f1f54957740 1 -- 172.21.15.31:0/619191564 shutdown_connections 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.206+0000 7f1f54957740 1 -- 172.21.15.31:0/619191564 wait complete. 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.207+0000 7f1f54957740 1 -- start start 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.207+0000 7f1f54957740 1 Processor -- start 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0aea1b0 0x55dea0ece7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ec9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0eca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3400d430 con 0x55dea0aea1b0 2024-09-21T06:12:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3400e8a0 con 0x55dea0af2390 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3400db00 con 0x55dea0b13070 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f54957740 10 client.0 _ll_register_callbacks cb 0x7f1f38009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0aea1b0 0x55dea0ece7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0aea1b0 0x55dea0ece7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34398/0 (socket says 172.21.15.31:34398) 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.208+0000 7f1f47fff640 1 -- 172.21.15.31:0/1466805648 learned_addr learned my addr 172.21.15.31:0/1466805648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0eca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f477fe640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ec9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 -- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 msgr2=0x55dea0ec9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ec9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 -- 172.21.15.31:0/1466805648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0aea1b0 msgr2=0x55dea0ece7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0aea1b0 0x55dea0ece7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 -- 172.21.15.31:0/1466805648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea0eca880 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f477fe640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ec9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f4c980640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0eca1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f38020e30 tx=0x7f1f38010dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f12bbdc118e91f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.209+0000 7f1f277fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.210+0000 7f1f277fe640 1 -- 172.21.15.31:0/1466805648 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38040060 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.210+0000 7f1f277fe640 1 -- 172.21.15.31:0/1466805648 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1f3802ad60 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.210+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f40003970 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.210+0000 7f1f277fe640 1 -- 172.21.15.31:0/1466805648 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1f38010740 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.210+0000 7f1f54957740 10 client.25390 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.210+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dea0f53930 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.211+0000 7f1f277fe640 1 -- 172.21.15.31:0/1466805648 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3802e030 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.211+0000 7f1f277fe640 1 -- 172.21.15.31:0/1466805648 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1f38021b70 con 0x55dea0b13070 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f54957740 10 client.25390 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f54957740 10 client.25390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f54957740 20 client.25390 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f54957740 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0aa6da0 0x55dea0f56070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f54957740 4 client.25390 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:12:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1f40003970 con 0x55dea0aa6da0 2024-09-21T06:12:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.212+0000 7f1f47fff640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0aa6da0 0x55dea0f56070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.214+0000 7f1f47fff640 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0aa6da0 0x55dea0f56070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b292fc16d5a3d22 server_cookie=f56623b133cdf3aa in_seq=0 out_seq=0 2024-09-21T06:12:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.215+0000 7f1f277fe640 10 client.25390 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.215+0000 7f1f277fe640 1 -- 172.21.15.31:0/1466805648 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x7f1f40003970 con 0x55dea0aa6da0 2024-09-21T06:12:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.215+0000 7f1f277fe640 10 client.25390 handle_command_reply: tid=0 2024-09-21T06:12:22.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.215+0000 7f1f54957740 1 client.25390 shutdown 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 msgr2=0x55dea0eca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0eca1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f38020e30 tx=0x7f1f38010dc0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 shutdown_connections 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dea0aa6da0 0x55dea0f56070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dea0b13070 0x55dea0eca1e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dea0af2390 0x55dea0ec9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 --2- 172.21.15.31:0/1466805648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dea0aea1b0 0x55dea0ece7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f277fe640 0 client.25390 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 >> 172.21.15.31:0/1466805648 conn(0x55dea0b07460 msgr2=0x55dea0ec6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 shutdown_connections 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 1 -- 172.21.15.31:0/1466805648 wait complete. 2024-09-21T06:12:22.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.216+0000 7f1f54957740 20 client.25390 trim_cache size 0 max 16384 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:22.224 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f300b6730 msgr2=0x7f1f300b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f300b6730 0x7f1f300b8c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f48112070 tx=0x7f1f38004730 comp rx=0 tx=0).stop 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 msgr2=0x7f1f4810ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 0x7f1f4810ffb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e9c0 tx=0x7f1f3c00edc0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 shutdown_connections 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f48108cf0 0x7f1f481104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f481083a0 0x7f1f4810ffb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f300b6730 0x7f1f300b8c40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 --2- 172.21.15.31:0/1918868481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f48102a60 0x7f1f481119c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 >> 172.21.15.31:0/1918868481 conn(0x7f1f48079730 msgr2=0x7f1f48106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:22.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.218+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 shutdown_connections 2024-09-21T06:12:22.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:22.219+0000 7f1f4e3e3640 1 -- 172.21.15.31:0/1918868481 wait complete. 2024-09-21T06:12:22.618 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:22 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:12:22.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:22 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:12:22.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:22 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:12:23.010 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:12:23.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:12:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:23.610 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:23 smithi031 ceph-mon[31237]: pgmap v343: 97 pgs: 97 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 1.22k op/s 2024-09-21T06:12:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:23 smithi110 ceph-mon[34550]: pgmap v343: 97 pgs: 97 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 1.22k op/s 2024-09-21T06:12:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:23 smithi107 ceph-mon[34380]: pgmap v343: 97 pgs: 97 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 1.22k op/s 2024-09-21T06:12:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.050+0000 7fbc1f59e640 1 -- 172.21.15.31:0/3355470892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 msgr2=0x7fbc18095430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.050+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/3355470892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 0x7fbc18095430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009900 tx=0x7fbc1002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.051+0000 7fbc1f59e640 1 -- 172.21.15.31:0/3355470892 shutdown_connections 2024-09-21T06:12:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.051+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/3355470892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc1809b2b0 0x7fbc18004780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.051+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/3355470892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc1809ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.051+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/3355470892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 0x7fbc18095430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.051+0000 7fbc1f59e640 1 -- 172.21.15.31:0/3355470892 >> 172.21.15.31:0/3355470892 conn(0x7fbc1800cd70 msgr2=0x7fbc1800d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.051+0000 7fbc1f59e640 1 -- 172.21.15.31:0/3355470892 shutdown_connections 2024-09-21T06:12:24.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.052+0000 7fbc1f59e640 1 -- 172.21.15.31:0/3355470892 wait complete. 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.053+0000 7fbc1f59e640 1 Processor -- start 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.053+0000 7fbc1f59e640 1 -- start start 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.053+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 0x7fbc180a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.053+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc180a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc1809b2b0 0x7fbc180a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180a82d0 con 0x7fbc1809a960 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180a8150 con 0x7fbc1809b2b0 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1f59e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180a8450 con 0x7fbc18095020 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc180a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc1809b2b0 0x7fbc180a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc180a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34416/0 (socket says 172.21.15.31:34416) 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.054+0000 7fbc1dd9b640 1 -- 172.21.15.31:0/2478318539 learned_addr learned my addr 172.21.15.31:0/2478318539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.055+0000 7fbc1dd9b640 1 -- 172.21.15.31:0/2478318539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 msgr2=0x7fbc180a7b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.055+0000 7fbc1dd9b640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 0x7fbc180a7b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.055+0000 7fbc1dd9b640 1 -- 172.21.15.31:0/2478318539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc1809b2b0 msgr2=0x7fbc180a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.055+0000 7fbc1dd9b640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc1809b2b0 0x7fbc180a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.055+0000 7fbc1dd9b640 1 -- 172.21.15.31:0/2478318539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1813f040 con 0x7fbc1809a960 2024-09-21T06:12:24.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.055+0000 7fbc1dd9b640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc180a3e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e9f0 tx=0x7fbc1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:24.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.056+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400ccc0 con 0x7fbc1809a960 2024-09-21T06:12:24.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.056+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1813f270 con 0x7fbc1809a960 2024-09-21T06:12:24.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.056+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1813f800 con 0x7fbc1809a960 2024-09-21T06:12:24.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.057+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc140108e0 con 0x7fbc1809a960 2024-09-21T06:12:24.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.057+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14018860 con 0x7fbc1809a960 2024-09-21T06:12:24.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.058+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc000052c0 con 0x7fbc1809a960 2024-09-21T06:12:24.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.063+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbc14010430 con 0x7fbc1809a960 2024-09-21T06:12:24.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.065+0000 7fbc0f7fe640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc080b6780 0x7fbc080b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:24.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.065+0000 7fbc1e59c640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc080b6780 0x7fbc080b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:24.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.066+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbc140e7af0 con 0x7fbc1809a960 2024-09-21T06:12:24.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.066+0000 7fbc1e59c640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc080b6780 0x7fbc080b8c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc1002a9e0 tx=0x7fbc10031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:24.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.068+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbc140ae020 con 0x7fbc1809a960 2024-09-21T06:12:24.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.497+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc000055b0 con 0x7fbc1809a960 2024-09-21T06:12:24.503 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:12:24.503 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:12:24.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.501+0000 7fbc0f7fe640 1 -- 172.21.15.31:0/2478318539 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fbc140b2400 con 0x7fbc1809a960 2024-09-21T06:12:24.503 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc080b6780 msgr2=0x7fbc080b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc080b6780 0x7fbc080b8c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc1002a9e0 tx=0x7fbc10031400 comp rx=0 tx=0).stop 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 msgr2=0x7fbc180a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc180a3e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e9f0 tx=0x7fbc1400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 shutdown_connections 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc1809b2b0 0x7fbc180a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc080b6780 0x7fbc080b8c90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc1809a960 0x7fbc180a3e10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 --2- 172.21.15.31:0/2478318539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18095020 0x7fbc180a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 >> 172.21.15.31:0/2478318539 conn(0x7fbc1800cd70 msgr2=0x7fbc180a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 shutdown_connections 2024-09-21T06:12:24.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:24.505+0000 7fbc1f59e640 1 -- 172.21.15.31:0/2478318539 wait complete. 2024-09-21T06:12:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 45.0 s, 4470.1 tps, lat 22.474 ms stddev 28.460 2024-09-21T06:12:25.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:12:25.557 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:25 smithi031 ceph-mon[31237]: pgmap v344: 97 pgs: 97 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-21T06:12:25.557 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:25 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2478318539' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:25.661 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:25 smithi110 ceph-mon[34550]: pgmap v344: 97 pgs: 97 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-21T06:12:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:25 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2478318539' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:25 smithi107 ceph-mon[34380]: pgmap v344: 97 pgs: 97 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-21T06:12:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:25 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2478318539' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:12:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 -- 172.21.15.31:0/3280499529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0074095030 msgr2=0x7f0074095440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 --2- 172.21.15.31:0/3280499529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0074095030 0x7f0074095440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0078009860 tx=0x7f007802a360 comp rx=0 tx=0).stop 2024-09-21T06:12:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 -- 172.21.15.31:0/3280499529 shutdown_connections 2024-09-21T06:12:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 --2- 172.21.15.31:0/3280499529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f007409b2c0 0x7f0074006300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 --2- 172.21.15.31:0/3280499529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f007409a970 0x7f007409ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 --2- 172.21.15.31:0/3280499529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0074095030 0x7f0074095440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.262+0000 7f0082046640 1 -- 172.21.15.31:0/3280499529 >> 172.21.15.31:0/3280499529 conn(0x7f007400cd80 msgr2=0x7f007400d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.263+0000 7f0082046640 1 -- 172.21.15.31:0/3280499529 shutdown_connections 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.263+0000 7f0082046640 1 -- 172.21.15.31:0/3280499529 wait complete. 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.263+0000 7f0082046640 1 Processor -- start 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.263+0000 7f0082046640 1 -- start start 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.264+0000 7f0082046640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 0x7f007412ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.264+0000 7f0082046640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f007409a970 0x7f007412f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.264+0000 7f0081044640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 0x7f007412ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.264+0000 7f0081044640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 0x7f007412ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34430/0 (socket says 172.21.15.31:34430) 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.265+0000 7f0082046640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f007409b2c0 0x7f0074134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.265+0000 7f0082046640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f00740a82e0 con 0x7f0074095030 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.265+0000 7f0081044640 1 -- 172.21.15.31:0/556286754 learned_addr learned my addr 172.21.15.31:0/556286754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.265+0000 7f0080843640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f007409a970 0x7f007412f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.265+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00740a8160 con 0x7f007409b2c0 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.265+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00740a8460 con 0x7f007409a970 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.266+0000 7f0081044640 1 -- 172.21.15.31:0/556286754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f007409a970 msgr2=0x7f007412f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.266+0000 7f0081044640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f007409a970 0x7f007412f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.266+0000 7f0081044640 1 -- 172.21.15.31:0/556286754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f007409b2c0 msgr2=0x7f0074134510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.266+0000 7f0081044640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f007409b2c0 0x7f0074134510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.266+0000 7f0081044640 1 -- 172.21.15.31:0/556286754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0074134c40 con 0x7f0074095030 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.267+0000 7f0081044640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 0x7f007412ecd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0078035040 tx=0x7f0078033e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.268+0000 7f00727fc640 1 -- 172.21.15.31:0/556286754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007803f030 con 0x7f0074095030 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.268+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0074134ed0 con 0x7f0074095030 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.268+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007413b690 con 0x7f0074095030 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.268+0000 7f00727fc640 1 -- 172.21.15.31:0/556286754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0078005100 con 0x7f0074095030 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.269+0000 7f00727fc640 1 -- 172.21.15.31:0/556286754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037040 con 0x7f0074095030 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 Processor -- start 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 -- start start 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da726070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da7265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da726af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618da693ea0 con 0x5618da355390 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618da6832d0 con 0x5618da34d1b0 2024-09-21T06:12:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.272+0000 7f008981c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618da6870e0 con 0x5618da376070 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.273+0000 7f0080843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da726af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.273+0000 7f0080843640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da726af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34434/0 (socket says 172.21.15.31:34434) 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.273+0000 7f0080843640 1 -- 172.21.15.31:0/2875432590 learned_addr learned my addr 172.21.15.31:0/2875432590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.275+0000 7f0080843640 1 -- 172.21.15.31:0/2875432590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 msgr2=0x5618da726070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.275+0000 7f0080843640 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da726070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.275+0000 7f0080843640 1 -- 172.21.15.31:0/2875432590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 msgr2=0x5618da7265b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.275+0000 7f0080843640 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da7265b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.275+0000 7f0080843640 1 -- 172.21.15.31:0/2875432590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618da3e1650 con 0x5618da355390 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.275+0000 7f0080843640 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da726af0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f006c00bc30 tx=0x7f006c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1fda644bf8fd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.276+0000 7f00677fe640 1 -- 172.21.15.31:0/2875432590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c03e480 con 0x5618da355390 2024-09-21T06:12:26.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.276+0000 7f00677fe640 1 -- 172.21.15.31:0/2875432590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f006c033900 con 0x5618da355390 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.276+0000 7f00677fe640 1 -- 172.21.15.31:0/2875432590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c03c700 con 0x5618da355390 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.277+0000 7f00727fc640 1 -- 172.21.15.31:0/556286754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0078042070 con 0x7f0074095030 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.278+0000 7f008981c740 1 -- 172.21.15.31:0/2875432590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 msgr2=0x5618da726af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.278+0000 7f008981c740 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da726af0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f006c00bc30 tx=0x7f006c0327b0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.278+0000 7f00727fc640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00680b6940 0x7f00680b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.278+0000 7f00727fc640 1 -- 172.21.15.31:0/556286754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f00780328f0 con 0x7f0074095030 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.279+0000 7f008981c740 1 -- 172.21.15.31:0/2875432590 shutdown_connections 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.279+0000 7f008981c740 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da726af0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.279+0000 7f008981c740 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da7265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.279+0000 7f008981c740 1 --2- 172.21.15.31:0/2875432590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da726070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.279+0000 7f008981c740 1 -- 172.21.15.31:0/2875432590 >> 172.21.15.31:0/2875432590 conn(0x5618da36a460 msgr2=0x5618da723f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.279+0000 7f008981c740 1 -- 172.21.15.31:0/2875432590 shutdown_connections 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.280+0000 7f008981c740 1 -- 172.21.15.31:0/2875432590 wait complete. 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.280+0000 7f0081845640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00680b6940 0x7f00680b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.280+0000 7f008981c740 1 -- start start 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.281+0000 7f008981c740 1 Processor -- start 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 0x5618da7297b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da729d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da376070 0x5618da72edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618da4e4700 con 0x5618da376070 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618da61e160 con 0x5618da355390 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618da7233f0 con 0x5618da34d1b0 2024-09-21T06:12:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.282+0000 7f008981c740 10 client.0 _ll_register_callbacks cb 0x5618da6379e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.283+0000 7f0081044640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 0x5618da7297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.283+0000 7f0080843640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da729d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.283+0000 7f0081044640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 0x5618da7297b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43598/0 (socket says 172.21.15.31:43598) 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.283+0000 7f0081044640 1 -- 172.21.15.31:0/655820749 learned_addr learned my addr 172.21.15.31:0/655820749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.283+0000 7f0081845640 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da376070 0x5618da72edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.283+0000 7f0081845640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00680b6940 0x7f00680b8e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f007c066c70 tx=0x7f007c0514d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.284+0000 7f0081044640 1 -- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 msgr2=0x5618da729d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.284+0000 7f0081044640 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da729d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.284+0000 7f0081044640 1 -- 172.21.15.31:0/655820749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da376070 msgr2=0x5618da72edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.284+0000 7f0081044640 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da376070 0x5618da72edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.284+0000 7f0081044640 1 -- 172.21.15.31:0/655820749 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618da64bc90 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.284+0000 7f0081044640 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 0x5618da7297b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f00780cc4e0 tx=0x7f00780d60b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9080aaaf9c8b84ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.285+0000 7f0064ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.285+0000 7f0064ff9640 1 -- 172.21.15.31:0/655820749 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037050 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.285+0000 7f0064ff9640 1 -- 172.21.15.31:0/655820749 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f00780c8130 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.285+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c033900 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.286+0000 7f0064ff9640 1 -- 172.21.15.31:0/655820749 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f00780cc2f0 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.288+0000 7f008981c740 10 client.25402 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.288+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5618da737d00 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f0064ff9640 1 -- 172.21.15.31:0/655820749 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00780ccee0 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f0064ff9640 1 -- 172.21.15.31:0/655820749 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f007803a030 con 0x5618da34d1b0 2024-09-21T06:12:26.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f008981c740 10 client.25402 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f008981c740 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f008981c740 20 client.25402 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f008981c740 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da3349f0 0x5618da73b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f008981c740 4 client.25402 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.289+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f006c033900 con 0x5618da3349f0 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.290+0000 7f0080843640 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da3349f0 0x5618da73b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.292+0000 7f0080843640 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da3349f0 0x5618da73b0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5226ae7d8a7a03a4 server_cookie=44d0f84fd379c2aa in_seq=0 out_seq=0 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.292+0000 7f0064ff9640 10 client.25402 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.292+0000 7f0064ff9640 1 -- 172.21.15.31:0/655820749 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f006c033900 con 0x5618da3349f0 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.292+0000 7f0064ff9640 10 client.25402 handle_command_reply: tid=0 2024-09-21T06:12:26.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.292+0000 7f008981c740 1 client.25402 shutdown 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 msgr2=0x5618da7297b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 0x5618da7297b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f00780cc4e0 tx=0x7f00780d60b0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 shutdown_connections 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da3349f0 0x5618da73b0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da376070 0x5618da72edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da729d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 --2- 172.21.15.31:0/655820749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da34d1b0 0x5618da7297b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 >> 172.21.15.31:0/655820749 conn(0x5618da36a460 msgr2=0x5618da727a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f0064ff9640 0 client.25402 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 shutdown_connections 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.293+0000 7f008981c740 1 -- 172.21.15.31:0/655820749 wait complete. 2024-09-21T06:12:26.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.294+0000 7f008981c740 20 client.25402 trim_cache size 0 max 16384 2024-09-21T06:12:26.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.315+0000 7f008981c740 1 Processor -- start 2024-09-21T06:12:26.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.315+0000 7f008981c740 1 -- start start 2024-09-21T06:12:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.315+0000 7f008981c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da723fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f008981c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da72eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f008981c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da71cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f008981c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f006000c7c0 con 0x5618da355390 2024-09-21T06:12:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f008981c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f006000f040 con 0x5618da34d1b0 2024-09-21T06:12:26.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f008981c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f006000ce30 con 0x5618da376070 2024-09-21T06:12:26.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da723fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da723fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38696/0 (socket says 172.21.15.31:38696) 2024-09-21T06:12:26.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 -- 172.21.15.31:0/3949485222 learned_addr learned my addr 172.21.15.31:0/3949485222 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:26.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081845640 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da71cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0080843640 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da72eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 -- 172.21.15.31:0/3949485222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 msgr2=0x5618da71cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da71cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 -- 172.21.15.31:0/3949485222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 msgr2=0x5618da72eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da72eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.316+0000 7f0081044640 1 -- 172.21.15.31:0/3949485222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618da7293e0 con 0x5618da34d1b0 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.317+0000 7f0081044640 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da723fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f007803b040 tx=0x7f007802a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b7ac2f0c368ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.317+0000 7f0066ffd640 1 -- 172.21.15.31:0/3949485222 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078037070 con 0x5618da34d1b0 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.317+0000 7f0066ffd640 1 -- 172.21.15.31:0/3949485222 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00780ccee0 con 0x5618da34d1b0 2024-09-21T06:12:26.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.317+0000 7f0066ffd640 1 -- 172.21.15.31:0/3949485222 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00780d6200 con 0x5618da34d1b0 2024-09-21T06:12:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.318+0000 7f008981c740 1 -- 172.21.15.31:0/3949485222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 msgr2=0x5618da723fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.318+0000 7f008981c740 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da723fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f007803b040 tx=0x7f007802a360 comp rx=0 tx=0).stop 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 -- 172.21.15.31:0/3949485222 shutdown_connections 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da71cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da355390 0x5618da72eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 --2- 172.21.15.31:0/3949485222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da34d1b0 0x5618da723fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 -- 172.21.15.31:0/3949485222 >> 172.21.15.31:0/3949485222 conn(0x5618da36a460 msgr2=0x5618da7238c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 -- 172.21.15.31:0/3949485222 shutdown_connections 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 -- 172.21.15.31:0/3949485222 wait complete. 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.319+0000 7f008981c740 1 -- start start 2024-09-21T06:12:26.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.320+0000 7f008981c740 1 Processor -- start 2024-09-21T06:12:26.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.320+0000 7f008981c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 0x5618da72ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.320+0000 7f008981c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da730520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.320+0000 7f008981c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da734e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.320+0000 7f008981c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f006000d500 con 0x5618da34d1b0 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f008981c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f006000e970 con 0x5618da355390 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081044640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 0x5618da72ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da734e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da734e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43618/0 (socket says 172.21.15.31:43618) 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081044640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 0x5618da72ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34462/0 (socket says 172.21.15.31:34462) 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 -- 172.21.15.31:0/3711977974 learned_addr learned my addr 172.21.15.31:0/3711977974 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f006000dbd0 con 0x5618da376070 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f008981c740 10 client.0 _ll_register_callbacks cb 0x7f00780c8e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 -- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 msgr2=0x5618da730520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da730520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 -- 172.21.15.31:0/3711977974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 msgr2=0x5618da72ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 0x5618da72ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081845640 1 -- 172.21.15.31:0/3711977974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618da7236d0 con 0x5618da376070 2024-09-21T06:12:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.321+0000 7f0081044640 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 0x5618da72ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.322+0000 7f0081845640 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da734e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f007c068d40 tx=0x7f007c069e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74187d5817753f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:26.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.322+0000 7f0064ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:26.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.323+0000 7f0064ff9640 1 -- 172.21.15.31:0/3711977974 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c08eb10 con 0x5618da376070 2024-09-21T06:12:26.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.323+0000 7f0064ff9640 1 -- 172.21.15.31:0/3711977974 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f007c08ecb0 con 0x5618da376070 2024-09-21T06:12:26.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.323+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00780ccee0 con 0x5618da376070 2024-09-21T06:12:26.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.324+0000 7f0064ff9640 1 -- 172.21.15.31:0/3711977974 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f007c080430 con 0x5618da376070 2024-09-21T06:12:26.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.324+0000 7f008981c740 10 client.25408 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:26.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.324+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5618da7a6140 con 0x5618da376070 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f0064ff9640 1 -- 172.21.15.31:0/3711977974 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00780ccee0 con 0x5618da376070 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f0064ff9640 1 -- 172.21.15.31:0/3711977974 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f007c0a1700 con 0x5618da376070 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f008981c740 10 client.25408 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f008981c740 10 client.25408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f008981c740 20 client.25408 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f008981c740 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da309da0 0x5618da7a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f008981c740 4 client.25408 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:12:26.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618da695df0 con 0x5618da309da0 2024-09-21T06:12:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.325+0000 7f0081044640 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da309da0 0x5618da7a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:26.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.326+0000 7f0081044640 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da309da0 0x5618da7a8970 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427d3a334cfc368a server_cookie=423b3901734b47d2 in_seq=0 out_seq=0 2024-09-21T06:12:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f0064ff9640 10 client.25408 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f0064ff9640 1 -- 172.21.15.31:0/3711977974 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618da695df0 con 0x5618da309da0 2024-09-21T06:12:26.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f0064ff9640 10 client.25408 handle_command_reply: tid=0 2024-09-21T06:12:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 client.25408 shutdown 2024-09-21T06:12:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 msgr2=0x5618da734e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da734e30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f007c068d40 tx=0x7f007c069e10 comp rx=0 tx=0).stop 2024-09-21T06:12:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 shutdown_connections 2024-09-21T06:12:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618da309da0 0x5618da7a8970 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f0064ff9640 0 client.25408 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618da376070 0x5618da734e30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618da355390 0x5618da730520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 --2- 172.21.15.31:0/3711977974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618da34d1b0 0x5618da72ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.328+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 >> 172.21.15.31:0/3711977974 conn(0x5618da36a460 msgr2=0x5618da72aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.329+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 shutdown_connections 2024-09-21T06:12:26.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.329+0000 7f008981c740 1 -- 172.21.15.31:0/3711977974 wait complete. 2024-09-21T06:12:26.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.330+0000 7f008981c740 20 client.25408 trim_cache size 0 max 16384 2024-09-21T06:12:26.332 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:12:26.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00680b6940 msgr2=0x7f00680b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00680b6940 0x7f00680b8e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f007c066c70 tx=0x7f007c0514d0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 msgr2=0x7f007412ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:26.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 0x7f007412ecd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0078035040 tx=0x7f0078033e00 comp rx=0 tx=0).stop 2024-09-21T06:12:26.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 shutdown_connections 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f007409b2c0 0x7f0074134510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f007409a970 0x7f007412f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00680b6940 0x7f00680b8e50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 --2- 172.21.15.31:0/556286754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074095030 0x7f007412ecd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 >> 172.21.15.31:0/556286754 conn(0x7f007400cd80 msgr2=0x7f0074099140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 shutdown_connections 2024-09-21T06:12:26.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:26.331+0000 7f0082046640 1 -- 172.21.15.31:0/556286754 wait complete. 2024-09-21T06:12:27.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:27 smithi031 ceph-mon[31237]: pgmap v345: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-09-21T06:12:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:27 smithi110 ceph-mon[34550]: pgmap v345: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-09-21T06:12:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:27 smithi107 ceph-mon[34380]: pgmap v345: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 1.14k op/s 2024-09-21T06:12:28.181 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:12:28.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:12:28.628 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.267+0000 7f1c95438640 1 -- 172.21.15.31:0/3577947889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90102750 msgr2=0x7f1c90102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.267+0000 7f1c95438640 1 --2- 172.21.15.31:0/3577947889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90102750 0x7f1c90102b60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009860 tx=0x7f1c8402a360 comp rx=0 tx=0).stop 2024-09-21T06:12:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.268+0000 7f1c95438640 1 -- 172.21.15.31:0/3577947889 shutdown_connections 2024-09-21T06:12:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.268+0000 7f1c95438640 1 --2- 172.21.15.31:0/3577947889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c90107c10 0x7f1c90108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.268+0000 7f1c95438640 1 --2- 172.21.15.31:0/3577947889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c901075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.268+0000 7f1c95438640 1 --2- 172.21.15.31:0/3577947889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90102750 0x7f1c90102b60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.268+0000 7f1c95438640 1 -- 172.21.15.31:0/3577947889 >> 172.21.15.31:0/3577947889 conn(0x7f1c900fe410 msgr2=0x7f1c90100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.270+0000 7f1c95438640 1 -- 172.21.15.31:0/3577947889 shutdown_connections 2024-09-21T06:12:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.270+0000 7f1c95438640 1 -- 172.21.15.31:0/3577947889 wait complete. 2024-09-21T06:12:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.271+0000 7f1c95438640 1 Processor -- start 2024-09-21T06:12:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 -- start start 2024-09-21T06:12:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c90102750 0x7f1c90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c9006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90107c10 0x7f1c9006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9011a110 con 0x7f1c90102750 2024-09-21T06:12:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90119f90 con 0x7f1c901030a0 2024-09-21T06:12:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.272+0000 7f1c95438640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9011a290 con 0x7f1c90107c10 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c9006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c9006e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44436/0 (socket says 172.21.15.31:44436) 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/2572831477 learned_addr learned my addr 172.21.15.31:0/2572831477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/2572831477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90107c10 msgr2=0x7f1c9006e8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90107c10 0x7f1c9006e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c90102750 msgr2=0x7f1c90074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c90102750 0x7f1c90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.273+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/2572831477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c90073bd0 con 0x7f1c901030a0 2024-09-21T06:12:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.274+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c9006e230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800efc0 tx=0x7f1c8800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.274+0000 7f1c77fff640 1 -- 172.21.15.31:0/2572831477 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88004b20 con 0x7f1c901030a0 2024-09-21T06:12:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.274+0000 7f1c77fff640 1 -- 172.21.15.31:0/2572831477 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c88004cc0 con 0x7f1c901030a0 2024-09-21T06:12:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.275+0000 7f1c77fff640 1 -- 172.21.15.31:0/2572831477 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88005480 con 0x7f1c901030a0 2024-09-21T06:12:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.275+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c90073ec0 con 0x7f1c901030a0 2024-09-21T06:12:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.275+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c900743c0 con 0x7f1c901030a0 2024-09-21T06:12:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.279+0000 7f1c9c485740 1 Processor -- start 2024-09-21T06:12:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 -- start start 2024-09-21T06:12:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139eb1b0 0x55ad13d909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d90ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c77fff640 1 -- 172.21.15.31:0/2572831477 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1c88005620 con 0x7f1c901030a0 2024-09-21T06:12:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d91430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ad13d056f0 con 0x55ad139eb1b0 2024-09-21T06:12:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ad13cf4b20 con 0x55ad139c22f0 2024-09-21T06:12:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.281+0000 7f1c9c485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad13cf8930 con 0x55ad139ca4d0 2024-09-21T06:12:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d90ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d90ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44460/0 (socket says 172.21.15.31:44460) 2024-09-21T06:12:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 -- 172.21.15.31:0/76004018 learned_addr learned my addr 172.21.15.31:0/76004018 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 -- 172.21.15.31:0/673035526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81049b0 msgr2=0x7fb6a8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 --2- 172.21.15.31:0/673035526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81049b0 0x7fb6a8104dc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb698009900 tx=0x7fb69802a440 comp rx=0 tx=0).stop 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 -- 172.21.15.31:0/673035526 shutdown_connections 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 --2- 172.21.15.31:0/673035526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 0x7fb6a8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 --2- 172.21.15.31:0/673035526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a8105300 0x7fb6a810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 --2- 172.21.15.31:0/673035526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81049b0 0x7fb6a8104dc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.281+0000 7fb6aea24640 1 -- 172.21.15.31:0/673035526 >> 172.21.15.31:0/673035526 conn(0x7fb6a8079730 msgr2=0x7fb6a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d91430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.282+0000 7fb6aea24640 1 -- 172.21.15.31:0/673035526 shutdown_connections 2024-09-21T06:12:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 -- 172.21.15.31:0/76004018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 msgr2=0x55ad13d91430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d91430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 -- 172.21.15.31:0/76004018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139eb1b0 msgr2=0x55ad13d909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139eb1b0 0x55ad13d909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 -- 172.21.15.31:0/76004018 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad13cba530 con 0x55ad139c22f0 2024-09-21T06:12:29.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.282+0000 7fb6aea24640 1 -- 172.21.15.31:0/673035526 wait complete. 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.282+0000 7f1c8effd640 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d90ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c8402a950 tx=0x7f1c840303d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102f87487ff786b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.283+0000 7f1c757fa640 1 -- 172.21.15.31:0/76004018 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84037040 con 0x55ad139c22f0 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.283+0000 7f1c757fa640 1 -- 172.21.15.31:0/76004018 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c84030dc0 con 0x55ad139c22f0 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.283+0000 7f1c757fa640 1 -- 172.21.15.31:0/76004018 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84003850 con 0x55ad139c22f0 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.283+0000 7f1c77fff640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c780b67b0 0x7f1c780b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.283+0000 7f1c77fff640 1 -- 172.21.15.31:0/2572831477 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1c880e79b0 con 0x7f1c901030a0 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 -- 172.21.15.31:0/76004018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 msgr2=0x55ad13d90ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d90ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c8402a950 tx=0x7f1c840303d0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 -- 172.21.15.31:0/76004018 shutdown_connections 2024-09-21T06:12:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d91430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d90ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 --2- 172.21.15.31:0/76004018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139eb1b0 0x55ad13d909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.284+0000 7f1c9c485740 1 -- 172.21.15.31:0/76004018 >> 172.21.15.31:0/76004018 conn(0x55ad139df300 msgr2=0x55ad13d8e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.285+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c780b67b0 0x7f1c780b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 Processor -- start 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 -- start start 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a81049b0 0x7fb6a81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 0x7fb6a81a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 0x7fb6a81a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a811a1a0 con 0x7fb6a810de10 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a811a020 con 0x7fb6a81049b0 2024-09-21T06:12:29.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.284+0000 7fb6aea24640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a811a320 con 0x7fb6a81a6410 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 0x7fb6a81a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 0x7fb6a81a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49508/0 (socket says 172.21.15.31:49508) 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 -- 172.21.15.31:0/4162471283 learned_addr learned my addr 172.21.15.31:0/4162471283 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 0x7fb6a81a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a81049b0 0x7fb6a81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 -- 172.21.15.31:0/4162471283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a81049b0 msgr2=0x7fb6a81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a81049b0 0x7fb6a81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.286+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c780b67b0 0x7f1c780b8cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c90103550 tx=0x7f1c80005ad0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 -- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 msgr2=0x7fb6a81a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.285+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 0x7fb6a81a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6acfc1640 1 -- 172.21.15.31:0/4162471283 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a81a6f20 con 0x7fb6a81a6410 2024-09-21T06:12:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 0x7fb6a81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:12:29.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a81049b0 0x7fb6a81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.286+0000 7f1c9c485740 1 -- 172.21.15.31:0/76004018 shutdown_connections 2024-09-21T06:12:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.286+0000 7f1c9c485740 1 -- 172.21.15.31:0/76004018 wait complete. 2024-09-21T06:12:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 0x7fb6a81a6820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b970 tx=0x7fb6a000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6a57fa640 1 -- 172.21.15.31:0/4162471283 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a003b070 con 0x7fb6a81a6410 2024-09-21T06:12:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6a57fa640 1 -- 172.21.15.31:0/4162471283 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a0036440 con 0x7fb6a81a6410 2024-09-21T06:12:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.286+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a81a7210 con 0x7fb6a81a6410 2024-09-21T06:12:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.287+0000 7fb6a57fa640 1 -- 172.21.15.31:0/4162471283 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00075d0 con 0x7fb6a81a6410 2024-09-21T06:12:29.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.287+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a806d170 con 0x7fb6a81a6410 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.287+0000 7f1c9c485740 1 -- start start 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.287+0000 7f1c9c485740 1 Processor -- start 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139c22f0 0x55ad13d9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d9c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ad13c2cab0 con 0x55ad139c22f0 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ad13c9a730 con 0x55ad139eb1b0 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad13d9f0d0 con 0x55ad139ca4d0 2024-09-21T06:12:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c9c485740 10 client.0 _ll_register_callbacks cb 0x55ad13cee6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c8f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d9c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c8f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44482/0 (socket says 172.21.15.31:44482) 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d9c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49524/0 (socket says 172.21.15.31:49524) 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.288+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/1569768658 learned_addr learned my addr 172.21.15.31:0/1569768658 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/1569768658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 msgr2=0x55ad13d9c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d9c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/1569768658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139c22f0 msgr2=0x55ad13d9c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139c22f0 0x55ad13d9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/1569768658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad13a56790 con 0x55ad139eb1b0 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da83f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000f6d0 tx=0x7f1c800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d396730d2a1005 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.289+0000 7f1c72ff5640 1 -- 172.21.15.31:0/1569768658 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80020070 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.290+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c84030dc0 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.290+0000 7f1c72ff5640 1 -- 172.21.15.31:0/1569768658 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1c8001c800 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.291+0000 7f1c72ff5640 1 -- 172.21.15.31:0/1569768658 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1c8002c050 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.291+0000 7f1c9c485740 10 client.25403 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.291+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ad13acf620 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.291+0000 7f1c72ff5640 1 -- 172.21.15.31:0/1569768658 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c84030dc0 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c72ff5640 1 -- 172.21.15.31:0/1569768658 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1c8001b070 con 0x55ad139eb1b0 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c9c485740 10 client.25403 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c9c485740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c9c485740 20 client.25403 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad139a9d80 0x55ad13dae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c9c485740 4 client.25403 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:29.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.292+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad13d07640 con 0x55ad139a9d80 2024-09-21T06:12:29.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.292+0000 7fb6b4f98740 1 Processor -- start 2024-09-21T06:12:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.293+0000 7f1c8effd640 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad139a9d80 0x55ad13dae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.293+0000 7f1c72ff5640 10 client.25403 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.293+0000 7f1c8effd640 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad139a9d80 0x55ad13dae7d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c898982011f7b780 server_cookie=81b1cc64f8fed320 in_seq=0 out_seq=0 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.293+0000 7fb6b4f98740 1 -- start start 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.293+0000 7fb6a57fa640 1 -- 172.21.15.31:0/4162471283 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb6a003e070 con 0x7fb6a81a6410 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.293+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.293+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 0x5622a1e5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a97670 0x5622a1e5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5622a1dd5e60 con 0x5622a1ab8350 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622a1dc5290 con 0x5622a1a8f490 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622a1dc90a0 con 0x5622a1a97670 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 0x5622a1e5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 0x5622a1e5f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44496/0 (socket says 172.21.15.31:44496) 2024-09-21T06:12:29.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 -- 172.21.15.31:0/2769568858 learned_addr learned my addr 172.21.15.31:0/2769568858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a97670 0x5622a1e5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.294+0000 7f1c72ff5640 1 -- 172.21.15.31:0/1569768658 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad13d07640 con 0x55ad139a9d80 2024-09-21T06:12:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.294+0000 7f1c72ff5640 10 client.25403 handle_command_reply: tid=0 2024-09-21T06:12:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.294+0000 7f1c9c485740 1 client.25403 shutdown 2024-09-21T06:12:29.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 -- 172.21.15.31:0/2769568858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a97670 msgr2=0x5622a1e5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a97670 0x5622a1e5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 -- 172.21.15.31:0/2769568858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 msgr2=0x5622a1e5f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e5f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a7fff640 1 -- 172.21.15.31:0/2769568858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a1b23930 con 0x5622a1a8f490 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.294+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e5f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:29.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.295+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 0x5622a1e5f970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb698004390 tx=0x7fb6980043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a33a74bc18cca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 msgr2=0x55ad13da83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da83f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000f6d0 tx=0x7f1c800046c0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 shutdown_connections 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad139a9d80 0x55ad13dae7d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da83f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139ca4d0 0x55ad13d9c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1569768658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139c22f0 0x55ad13d9c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.295+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 >> 172.21.15.31:0/1569768658 conn(0x55ad139df300 msgr2=0x55ad13d919e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.296+0000 7f1c72ff5640 0 client.25403 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.295+0000 7fb68e7fc640 1 -- 172.21.15.31:0/2769568858 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69803f070 con 0x5622a1a8f490 2024-09-21T06:12:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.295+0000 7fb68e7fc640 1 -- 172.21.15.31:0/2769568858 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6980046c0 con 0x5622a1a8f490 2024-09-21T06:12:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.295+0000 7fb68e7fc640 1 -- 172.21.15.31:0/2769568858 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698035030 con 0x5622a1a8f490 2024-09-21T06:12:29.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.295+0000 7fb6a57fa640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6900b6870 0x7fb6900b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.296+0000 7fb6a57fa640 1 -- 172.21.15.31:0/4162471283 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb6a0108c30 con 0x7fb6a81a6410 2024-09-21T06:12:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.297+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 shutdown_connections 2024-09-21T06:12:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.297+0000 7f1c9c485740 1 -- 172.21.15.31:0/1569768658 wait complete. 2024-09-21T06:12:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.297+0000 7f1c9c485740 20 client.25403 trim_cache size 0 max 16384 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.296+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2769568858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 msgr2=0x5622a1e5f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.296+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 0x5622a1e5f970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb698004390 tx=0x7fb6980043c0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2769568858 shutdown_connections 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a97670 0x5622a1e5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a8f490 0x5622a1e5f970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2769568858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e5f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2769568858 >> 172.21.15.31:0/2769568858 conn(0x5622a1aac740 msgr2=0x5622a1e5d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6900b6870 0x7fb6900b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2769568858 shutdown_connections 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.297+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2769568858 wait complete. 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.298+0000 7fb6b4f98740 1 -- start start 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.298+0000 7fb6b4f98740 1 Processor -- start 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.298+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6900b6870 0x7fb6900b8d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005fd0 tx=0x7fb69c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5622a1c26890 con 0x5622a1ab8350 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5622a1d5fd40 con 0x5622a1a97670 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622a1e6dbb0 con 0x5622a1a8f490 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6b4f98740 10 client.0 _ll_register_callbacks cb 0x5622a1dc3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e76ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57540/0 (socket says 172.21.15.31:57540) 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 -- 172.21.15.31:0/942738403 learned_addr learned my addr 172.21.15.31:0/942738403 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 -- 172.21.15.31:0/942738403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 msgr2=0x5622a1e6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e6b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 -- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 msgr2=0x5622a1e6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.299+0000 7fb6acfc1640 1 -- 172.21.15.31:0/942738403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a1e77410 con 0x5622a1ab8350 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.300+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e76ed0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0036ba0 tx=0x7fb6a00ce1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7a6e517fda69a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.300+0000 7fb6877fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.300+0000 7fb6877fe640 1 -- 172.21.15.31:0/942738403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00d3070 con 0x5622a1ab8350 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.300+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6980046c0 con 0x5622a1ab8350 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.301+0000 7fb6877fe640 1 -- 172.21.15.31:0/942738403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb6a01065e0 con 0x5622a1ab8350 2024-09-21T06:12:29.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.301+0000 7fb6877fe640 1 -- 172.21.15.31:0/942738403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb6a00cf160 con 0x5622a1ab8350 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.301+0000 7fb6b4f98740 10 client.15684 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.301+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5622a1e79ce0 con 0x5622a1ab8350 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6877fe640 1 -- 172.21.15.31:0/942738403 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6980046c0 con 0x5622a1ab8350 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6877fe640 1 -- 172.21.15.31:0/942738403 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb6a00cddd0 con 0x5622a1ab8350 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6b4f98740 10 client.15684 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6b4f98740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6b4f98740 20 client.15684 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a76cd0 0x5622a1e7d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6b4f98740 4 client.15684 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:29.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.303+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a1dd7db0 con 0x5622a1a76cd0 2024-09-21T06:12:29.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.304+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a76cd0 0x5622a1e7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.305+0000 7fb6877fe640 10 client.15684 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.305+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a76cd0 0x5622a1e7d180 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940b8aadb34f83b3 server_cookie=656398c8c5fa73b3 in_seq=0 out_seq=0 2024-09-21T06:12:29.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6877fe640 1 -- 172.21.15.31:0/942738403 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5622a1dd7db0 con 0x5622a1a76cd0 2024-09-21T06:12:29.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6877fe640 10 client.15684 handle_command_reply: tid=0 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 client.15684 shutdown 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 msgr2=0x5622a1e76ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e76ed0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0036ba0 tx=0x7fb6a00ce1d0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 shutdown_connections 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a76cd0 0x5622a1e7d180 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1ab8350 0x5622a1e76ed0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/942738403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6877fe640 0 client.15684 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.306+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 >> 172.21.15.31:0/942738403 conn(0x5622a1aac740 msgr2=0x5622a1e62610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.308+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 shutdown_connections 2024-09-21T06:12:29.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.308+0000 7fb6b4f98740 1 -- 172.21.15.31:0/942738403 wait complete. 2024-09-21T06:12:29.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.309+0000 7fb6b4f98740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T06:12:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.329+0000 7f1c9c485740 1 Processor -- start 2024-09-21T06:12:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 -- start start 2024-09-21T06:12:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139c22f0 0x55ad13d9c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d8e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901075c0 con 0x55ad139ca4d0 2024-09-21T06:12:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901c76f0 con 0x55ad139eb1b0 2024-09-21T06:12:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.330+0000 7f1c9c485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90110440 con 0x55ad139c22f0 2024-09-21T06:12:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.331+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d8e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.331+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d8e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57544/0 (socket says 172.21.15.31:57544) 2024-09-21T06:12:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.331+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/1812797466 learned_addr learned my addr 172.21.15.31:0/1812797466 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/1812797466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139c22f0 msgr2=0x55ad13d9c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139c22f0 0x55ad13d9c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/1812797466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 msgr2=0x55ad13da82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/1812797466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad13daeda0 con 0x55ad139ca4d0 2024-09-21T06:12:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.332+0000 7f1c8e7fc640 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d8e800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1c880a9db0 tx=0x7f1c8800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7def3999d33830 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.333+0000 7f1c74ff9640 1 -- 172.21.15.31:0/1812797466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880ad130 con 0x55ad139ca4d0 2024-09-21T06:12:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.334+0000 7f1c74ff9640 1 -- 172.21.15.31:0/1812797466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c880ad2f0 con 0x55ad139ca4d0 2024-09-21T06:12:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.334+0000 7f1c74ff9640 1 -- 172.21.15.31:0/1812797466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880acf50 con 0x55ad139ca4d0 2024-09-21T06:12:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.335+0000 7f1c9c485740 1 -- 172.21.15.31:0/1812797466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 msgr2=0x55ad13d8e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.335+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d8e800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1c880a9db0 tx=0x7f1c8800beb0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 -- 172.21.15.31:0/1812797466 shutdown_connections 2024-09-21T06:12:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139eb1b0 0x55ad13da82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d8e800 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 --2- 172.21.15.31:0/1812797466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139c22f0 0x55ad13d9c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 -- 172.21.15.31:0/1812797466 >> 172.21.15.31:0/1812797466 conn(0x55ad139df300 msgr2=0x55ad13d8e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 -- 172.21.15.31:0/1812797466 shutdown_connections 2024-09-21T06:12:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.336+0000 7f1c9c485740 1 -- 172.21.15.31:0/1812797466 wait complete. 2024-09-21T06:12:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 -- start start 2024-09-21T06:12:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 Processor -- start 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d9f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139eb1b0 0x55ad13d9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90110b10 con 0x55ad139ca4d0 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90111f80 con 0x55ad139c22f0 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901111e0 con 0x55ad139eb1b0 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.337+0000 7f1c9c485740 10 client.0 _ll_register_callbacks cb 0x7f1c8000a3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d9f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d9f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57552/0 (socket says 172.21.15.31:57552) 2024-09-21T06:12:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8e7fc640 1 -- 172.21.15.31:0/3994446606 learned_addr learned my addr 172.21.15.31:0/3994446606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139eb1b0 0x55ad13d9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8effd640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 msgr2=0x55ad13d9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/3994446606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 msgr2=0x55ad13d9f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d9f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 -- 172.21.15.31:0/3994446606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad13e172c0 con 0x55ad139eb1b0 2024-09-21T06:12:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c8f7fe640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139eb1b0 0x55ad13d9fe80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000fea0 tx=0x7f1c8000cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6c438cdbdcc9e36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.338+0000 7f1c72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.340+0000 7f1c72ff5640 1 -- 172.21.15.31:0/3994446606 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80020070 con 0x55ad139eb1b0 2024-09-21T06:12:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.340+0000 7f1c72ff5640 1 -- 172.21.15.31:0/3994446606 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1c80024880 con 0x55ad139eb1b0 2024-09-21T06:12:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.340+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c880ad2f0 con 0x55ad139eb1b0 2024-09-21T06:12:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.341+0000 7f1c72ff5640 1 -- 172.21.15.31:0/3994446606 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1c80024a80 con 0x55ad139eb1b0 2024-09-21T06:12:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.341+0000 7f1c9c485740 10 client.25429 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.341+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ad13e18550 con 0x55ad139eb1b0 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.341+0000 7f1c72ff5640 1 -- 172.21.15.31:0/3994446606 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c880ad2f0 con 0x55ad139eb1b0 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c72ff5640 1 -- 172.21.15.31:0/3994446606 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1c8001b040 con 0x55ad139eb1b0 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c9c485740 10 client.25429 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c9c485740 10 client.25429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c9c485740 20 client.25429 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c9c485740 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad1397f1c0 0x55ad13e1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c9c485740 4 client.25429 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:12:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.342+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad13d030f0 con 0x55ad1397f1c0 2024-09-21T06:12:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.343+0000 7f1c8effd640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad1397f1c0 0x55ad13e1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.343+0000 7f1c8effd640 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad1397f1c0 0x55ad13e1ac40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71907e3144598b8b server_cookie=fabfebd8b12ed01 in_seq=0 out_seq=0 2024-09-21T06:12:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.343+0000 7f1c72ff5640 10 client.25429 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.343+0000 7fb6b4f98740 1 Processor -- start 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.343+0000 7fb6b4f98740 1 -- start start 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e76dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a97670 0x5622a1e5d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1ab8350 0x5622a1e64510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a97670 0x5622a1e5d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a97670 0x5622a1e5d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57566/0 (socket says 172.21.15.31:57566) 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e76dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e76dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49554/0 (socket says 172.21.15.31:49554) 2024-09-21T06:12:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6a77fe640 1 -- 172.21.15.31:0/2503904879 learned_addr learned my addr 172.21.15.31:0/2503904879 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8073b50 con 0x5622a1a97670 2024-09-21T06:12:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1ab8350 0x5622a1e64510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81ae380 con 0x5622a1ab8350 2024-09-21T06:12:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.344+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80741c0 con 0x5622a1a8f490 2024-09-21T06:12:29.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.345+0000 7fb6acfc1640 1 -- 172.21.15.31:0/2503904879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 msgr2=0x5622a1e76dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.345+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e76dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.345+0000 7fb6acfc1640 1 -- 172.21.15.31:0/2503904879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a97670 msgr2=0x5622a1e5d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.345+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a97670 0x5622a1e5d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.345+0000 7fb6acfc1640 1 -- 172.21.15.31:0/2503904879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a1e5ca70 con 0x5622a1ab8350 2024-09-21T06:12:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.345+0000 7fb6acfc1640 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1ab8350 0x5622a1e64510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b940 tx=0x7fb6a00ce300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d2dc6bbfc2ca89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.346+0000 7fb68dffb640 1 -- 172.21.15.31:0/2503904879 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00d3040 con 0x5622a1ab8350 2024-09-21T06:12:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.346+0000 7fb68dffb640 1 -- 172.21.15.31:0/2503904879 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a00d3540 con 0x5622a1ab8350 2024-09-21T06:12:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.346+0000 7fb68dffb640 1 -- 172.21.15.31:0/2503904879 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00e9080 con 0x5622a1ab8350 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.347+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1ab8350 msgr2=0x5622a1e64510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.347+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1ab8350 0x5622a1e64510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b940 tx=0x7fb6a00ce300 comp rx=0 tx=0).stop 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.348+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 shutdown_connections 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.348+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1ab8350 0x5622a1e64510 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.348+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a97670 0x5622a1e5d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.348+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/2503904879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1a8f490 0x5622a1e76dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.348+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 >> 172.21.15.31:0/2503904879 conn(0x5622a1aac740 msgr2=0x5622a1e5cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.349+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 shutdown_connections 2024-09-21T06:12:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.350+0000 7fb6b4f98740 1 -- 172.21.15.31:0/2503904879 wait complete. 2024-09-21T06:12:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.350+0000 7fb6b4f98740 1 -- start start 2024-09-21T06:12:29.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.351+0000 7fb6b4f98740 1 Processor -- start 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 0x5622a1e6ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1ab8350 0x5622a1e6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8074890 con 0x5622a1a8f490 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81adcb0 con 0x5622a1a97670 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8074f60 con 0x5622a1ab8350 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.352+0000 7fb6b4f98740 10 client.0 _ll_register_callbacks cb 0x7fb6a0036c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.353+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 0x5622a1e6ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.353+0000 7fb6a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 0x5622a1e6ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57568/0 (socket says 172.21.15.31:57568) 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.353+0000 7fb6a7fff640 1 -- 172.21.15.31:0/1236295005 learned_addr learned my addr 172.21.15.31:0/1236295005 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6a7fff640 1 -- 172.21.15.31:0/1236295005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1ab8350 msgr2=0x5622a1e6e870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1ab8350 0x5622a1e6e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6a7fff640 1 -- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 msgr2=0x5622a1e6e330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6a7fff640 1 -- 172.21.15.31:0/1236295005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a1ee7b40 con 0x5622a1a8f490 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6a7fff640 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 0x5622a1e6ddf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb69802a960 tx=0x7fb698009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898968b3124e355d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:29.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.354+0000 7fb6877fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:12:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.355+0000 7fb6877fe640 1 -- 172.21.15.31:0/1236295005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69803f040 con 0x5622a1a8f490 2024-09-21T06:12:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.355+0000 7fb6877fe640 1 -- 172.21.15.31:0/1236295005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb6980048c0 con 0x5622a1a8f490 2024-09-21T06:12:29.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.355+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a00d3540 con 0x5622a1a8f490 2024-09-21T06:12:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6877fe640 1 -- 172.21.15.31:0/1236295005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb698035040 con 0x5622a1a8f490 2024-09-21T06:12:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6b4f98740 10 client.15696 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5622a1ee8e00 con 0x5622a1a8f490 2024-09-21T06:12:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6877fe640 1 -- 172.21.15.31:0/1236295005 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb698051070 con 0x5622a1a8f490 2024-09-21T06:12:29.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6877fe640 1 -- 172.21.15.31:0/1236295005 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb69803ba00 con 0x5622a1a8f490 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6b4f98740 10 client.15696 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6b4f98740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6b4f98740 20 client.15696 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.361+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a4c080 0x5622a1eeb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.362+0000 7fb6b4f98740 4 client.15696 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.362+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb6a00d3540 con 0x5622a1a4c080 2024-09-21T06:12:29.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.362+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a4c080 0x5622a1eeb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.362+0000 7fb6a77fe640 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a4c080 0x5622a1eeb0a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4ece39b850af06 server_cookie=4b74589f58fc909a in_seq=0 out_seq=0 2024-09-21T06:12:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.362+0000 7fb6877fe640 10 client.15696 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.364+0000 7fb6877fe640 1 -- 172.21.15.31:0/1236295005 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb6a00d3540 con 0x5622a1a4c080 2024-09-21T06:12:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.364+0000 7fb6877fe640 10 client.15696 handle_command_reply: tid=0 2024-09-21T06:12:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.364+0000 7fb6b4f98740 1 client.15696 shutdown 2024-09-21T06:12:29.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 msgr2=0x5622a1e6ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 0x5622a1e6ddf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb69802a960 tx=0x7fb698009260 comp rx=0 tx=0).stop 2024-09-21T06:12:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 shutdown_connections 2024-09-21T06:12:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5622a1a4c080 0x5622a1eeb0a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.368 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:29 smithi031 ceph-mon[31237]: pgmap v346: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 11 MiB/s wr, 899 op/s 2024-09-21T06:12:29.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6877fe640 0 client.15696 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622a1ab8350 0x5622a1e6e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5622a1a97670 0x5622a1e6e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 --2- 172.21.15.31:0/1236295005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5622a1a8f490 0x5622a1e6ddf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.365+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 >> 172.21.15.31:0/1236295005 conn(0x5622a1aac740 msgr2=0x5622a1e60440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.366+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 shutdown_connections 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.366+0000 7fb6b4f98740 1 -- 172.21.15.31:0/1236295005 wait complete. 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.366+0000 7fb6b4f98740 20 client.15696 trim_cache size 0 max 16384 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e", 2024-09-21T06:12:29.369 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:12:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:12:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.370+0000 7f1c72ff5640 1 -- 172.21.15.31:0/3994446606 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad13d030f0 con 0x55ad1397f1c0 2024-09-21T06:12:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.370+0000 7f1c72ff5640 10 client.25429 handle_command_reply: tid=0 2024-09-21T06:12:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.370+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6900b6870 msgr2=0x7fb6900b8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.370+0000 7fb6aea24640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6900b6870 0x7fb6900b8d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005fd0 tx=0x7fb69c005850 comp rx=0 tx=0).stop 2024-09-21T06:12:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.370+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 msgr2=0x7fb6a81a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.370+0000 7fb6aea24640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 0x7fb6a81a6820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b970 tx=0x7fb6a000be30 comp rx=0 tx=0).stop 2024-09-21T06:12:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.370+0000 7f1c9c485740 1 client.25429 shutdown 2024-09-21T06:12:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.370+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139eb1b0 msgr2=0x55ad13d9fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.370+0000 7f1c9c485740 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139eb1b0 0x55ad13d9fe80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000fea0 tx=0x7f1c8000cf00 comp rx=0 tx=0).stop 2024-09-21T06:12:29.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.370+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 shutdown_connections 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 shutdown_connections 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ad1397f1c0 0x55ad13e1ac40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad139eb1b0 0x55ad13d9fe80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ad139ca4d0 0x55ad13d9f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 --2- 172.21.15.31:0/3994446606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ad139c22f0 0x55ad13d9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 >> 172.21.15.31:0/3994446606 conn(0x55ad139df300 msgr2=0x55ad13d8cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c72ff5640 0 client.25429 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 shutdown_connections 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 1 -- 172.21.15.31:0/3994446606 wait complete. 2024-09-21T06:12:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.371+0000 7f1c9c485740 20 client.25429 trim_cache size 0 max 16384 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6a81a6410 0x7fb6a81a6820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6900b6870 0x7fb6900b8d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6a810de10 0x7fb6a81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 --2- 172.21.15.31:0/4162471283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6a81049b0 0x7fb6a81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 >> 172.21.15.31:0/4162471283 conn(0x7fb6a8079730 msgr2=0x7fb6a81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 shutdown_connections 2024-09-21T06:12:29.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:29.371+0000 7fb6aea24640 1 -- 172.21.15.31:0/4162471283 wait complete. 2024-09-21T06:12:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:12:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:12:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:12:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:12:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.373+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c780b67b0 msgr2=0x7f1c780b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.373+0000 7f1c95438640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c780b67b0 0x7f1c780b8cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1c90103550 tx=0x7f1c80005ad0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.373+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 msgr2=0x7f1c9006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.373+0000 7f1c95438640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c9006e230 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800efc0 tx=0x7f1c8800c800 comp rx=0 tx=0).stop 2024-09-21T06:12:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.379+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 shutdown_connections 2024-09-21T06:12:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.379+0000 7f1c95438640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c90107c10 0x7f1c9006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.380+0000 7f1c95438640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901030a0 0x7f1c9006e230 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.380+0000 7f1c95438640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c780b67b0 0x7f1c780b8cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.380+0000 7f1c95438640 1 --2- 172.21.15.31:0/2572831477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c90102750 0x7f1c90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.380+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 >> 172.21.15.31:0/2572831477 conn(0x7f1c900fe410 msgr2=0x7f1c90105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.381+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 shutdown_connections 2024-09-21T06:12:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:29.382+0000 7f1c95438640 1 -- 172.21.15.31:0/2572831477 wait complete. 2024-09-21T06:12:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:29 smithi110 ceph-mon[34550]: pgmap v346: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 11 MiB/s wr, 899 op/s 2024-09-21T06:12:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:29 smithi107 ceph-mon[34380]: pgmap v346: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 11 MiB/s wr, 899 op/s 2024-09-21T06:12:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 50.0 s, 4137.1 tps, lat 24.136 ms stddev 26.736 2024-09-21T06:12:30.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:12:30.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:30 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:12:30.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:30 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:30.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:30 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:12:30.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:30 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:30.706 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:12:30.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:30 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:12:30.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:30 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:30.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:30 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:12:30.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:30 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:30.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:30 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:12:30.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:30 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:30.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:30 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:12:30.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:30 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 -- 172.21.15.31:0/3702238518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec1083a0 msgr2=0x7feeec1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 --2- 172.21.15.31:0/3702238518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec1083a0 0x7feeec1087b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feee0009860 tx=0x7feee002a340 comp rx=0 tx=0).stop 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 -- 172.21.15.31:0/3702238518 shutdown_connections 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 --2- 172.21.15.31:0/3702238518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 --2- 172.21.15.31:0/3702238518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec1083a0 0x7feeec1087b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 --2- 172.21.15.31:0/3702238518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feeec102a60 0x7feeec102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 -- 172.21.15.31:0/3702238518 >> 172.21.15.31:0/3702238518 conn(0x7feeec079730 msgr2=0x7feeec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 -- 172.21.15.31:0/3702238518 shutdown_connections 2024-09-21T06:12:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.321+0000 7feef1785640 1 -- 172.21.15.31:0/3702238518 wait complete. 2024-09-21T06:12:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.322+0000 7feef1785640 1 Processor -- start 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.322+0000 7feef1785640 1 -- start start 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.323+0000 7feef1785640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec102a60 0x7feeec19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.323+0000 7feef1785640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feeec1083a0 0x7feeec1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.323+0000 7feef1785640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.323+0000 7feef1785640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feeec1118a0 con 0x7feeec1083a0 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.323+0000 7feef1785640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feeec111720 con 0x7feeec108cf0 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.323+0000 7feef1785640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feeec111a20 con 0x7feeec102a60 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44544/0 (socket says 172.21.15.31:44544) 2024-09-21T06:12:31.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 -- 172.21.15.31:0/2510383594 learned_addr learned my addr 172.21.15.31:0/2510383594 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeea7fc640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feeec1083a0 0x7feeec1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeaffd640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec102a60 0x7feeec19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 -- 172.21.15.31:0/2510383594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec102a60 msgr2=0x7feeec19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec102a60 0x7feeec19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 -- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feeec1083a0 msgr2=0x7feeec1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feeec1083a0 0x7feeec1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.324+0000 7feeeb7fe640 1 -- 172.21.15.31:0/2510383594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec19c390 con 0x7feeec108cf0 2024-09-21T06:12:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.326+0000 7feeeb7fe640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec1987c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb10 tx=0x7feee400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.326+0000 7feed3fff640 1 -- 172.21.15.31:0/2510383594 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4004b20 con 0x7feeec108cf0 2024-09-21T06:12:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.326+0000 7feed3fff640 1 -- 172.21.15.31:0/2510383594 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee4004cc0 con 0x7feeec108cf0 2024-09-21T06:12:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.326+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec19c680 con 0x7feeec108cf0 2024-09-21T06:12:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.326+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec19cb90 con 0x7feeec108cf0 2024-09-21T06:12:31.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.326+0000 7feed3fff640 1 -- 172.21.15.31:0/2510383594 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4010670 con 0x7feeec108cf0 2024-09-21T06:12:31.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.330+0000 7feed3fff640 1 -- 172.21.15.31:0/2510383594 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7feee40108f0 con 0x7feeec108cf0 2024-09-21T06:12:31.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.330+0000 7feef7cf9740 1 Processor -- start 2024-09-21T06:12:31.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 -- start start 2024-09-21T06:12:31.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 0x55d15bd02270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd02cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d15bc78e50 con 0x55d15b9321b0 2024-09-21T06:12:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d15bc68280 con 0x55d15b95b070 2024-09-21T06:12:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.331+0000 7feef7cf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d15bc6c090 con 0x55d15b93a390 2024-09-21T06:12:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feed3fff640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed40b6740 0x7feed40b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 0x55d15bd02270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 0x55d15bd02270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44546/0 (socket says 172.21.15.31:44546) 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feeea7fc640 1 -- 172.21.15.31:0/1472710700 learned_addr learned my addr 172.21.15.31:0/1472710700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feeea7fc640 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd02cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feeeb7fe640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed40b6740 0x7feed40b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.332+0000 7feeeaffd640 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeea7fc640 1 -- 172.21.15.31:0/1472710700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 msgr2=0x55d15bd02cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeea7fc640 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd02cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feed3fff640 1 -- 172.21.15.31:0/2510383594 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feee40e7b70 con 0x7feeec108cf0 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeea7fc640 1 -- 172.21.15.31:0/1472710700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 msgr2=0x55d15bd027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeea7fc640 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeea7fc640 1 -- 172.21.15.31:0/1472710700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15b9c6650 con 0x55d15b95b070 2024-09-21T06:12:31.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeea7fc640 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 0x55d15bd02270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feee0005eb0 tx=0x7feee0030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4663d88670aab92e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feeeb7fe640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed40b6740 0x7feed40b8c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feee4004af0 tx=0x7feee407be70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feed17fa640 1 -- 172.21.15.31:0/1472710700 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0047070 con 0x55d15b95b070 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.333+0000 7feed17fa640 1 -- 172.21.15.31:0/1472710700 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee0003ca0 con 0x55d15b95b070 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.334+0000 7feed17fa640 1 -- 172.21.15.31:0/1472710700 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0037040 con 0x55d15b95b070 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.334+0000 7feef7cf9740 1 -- 172.21.15.31:0/1472710700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 msgr2=0x55d15bd02270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.334+0000 7feef7cf9740 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 0x55d15bd02270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feee0005eb0 tx=0x7feee0030340 comp rx=0 tx=0).stop 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 -- 172.21.15.31:0/1472710700 shutdown_connections 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd02cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 --2- 172.21.15.31:0/1472710700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b95b070 0x55d15bd02270 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 -- 172.21.15.31:0/1472710700 >> 172.21.15.31:0/1472710700 conn(0x55d15b94f460 msgr2=0x55d15bd00160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 -- 172.21.15.31:0/1472710700 shutdown_connections 2024-09-21T06:12:31.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.335+0000 7feef7cf9740 1 -- 172.21.15.31:0/1472710700 wait complete. 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.336+0000 7feef7cf9740 1 -- start start 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.336+0000 7feef7cf9740 1 Processor -- start 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bd0d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd0de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b95b070 0x55d15bd2ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d15bac9700 con 0x55d15b95b070 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d15bc03110 con 0x55d15b9321b0 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d15bd21aa0 con 0x55d15b93a390 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feef7cf9740 10 client.0 _ll_register_callbacks cb 0x55d15bc1c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bd0d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bd0d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44560/0 (socket says 172.21.15.31:44560) 2024-09-21T06:12:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feeea7fc640 1 -- 172.21.15.31:0/992915433 learned_addr learned my addr 172.21.15.31:0/992915433 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:31.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.337+0000 7feeeaffd640 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd0de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeea7fc640 1 -- 172.21.15.31:0/992915433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 msgr2=0x55d15bd0de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeea7fc640 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd0de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeea7fc640 1 -- 172.21.15.31:0/992915433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b95b070 msgr2=0x55d15bd2ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeea7fc640 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b95b070 0x55d15bd2ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeea7fc640 1 -- 172.21.15.31:0/992915433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15bc30c40 con 0x55d15b9321b0 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeea7fc640 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bd0d950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feee0000c00 tx=0x7feee002a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682748a5b589636b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeceff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.338+0000 7feeceff5640 1 -- 172.21.15.31:0/992915433 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0047030 con 0x55d15b9321b0 2024-09-21T06:12:31.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.339+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee0003ca0 con 0x55d15b9321b0 2024-09-21T06:12:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.339+0000 7feeceff5640 1 -- 172.21.15.31:0/992915433 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feee00037f0 con 0x55d15b9321b0 2024-09-21T06:12:31.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.339+0000 7feeceff5640 1 -- 172.21.15.31:0/992915433 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7feee0037070 con 0x55d15b9321b0 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 10 client.25427 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d15bd2d800 con 0x55d15b9321b0 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feeceff5640 1 -- 172.21.15.31:0/992915433 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee0003ca0 con 0x55d15b9321b0 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feeceff5640 1 -- 172.21.15.31:0/992915433 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7feee0056530 con 0x55d15b9321b0 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 10 client.25427 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 10 client.25427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 20 client.25427 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b9199f0 0x55d15bd30b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 4 client.25427 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.340+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d15bc7ada0 con 0x55d15b9199f0 2024-09-21T06:12:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.341+0000 7feeeaffd640 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b9199f0 0x55d15bd30b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.342+0000 7feeeaffd640 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b9199f0 0x55d15bd30b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f512fa03901e9c server_cookie=7d9c1328aced4df1 in_seq=0 out_seq=0 2024-09-21T06:12:31.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.342+0000 7feeceff5640 10 client.25427 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.344+0000 7feeceff5640 1 -- 172.21.15.31:0/992915433 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d15bc7ada0 con 0x55d15b9199f0 2024-09-21T06:12:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.344+0000 7feeceff5640 10 client.25427 handle_command_reply: tid=0 2024-09-21T06:12:31.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.344+0000 7feef7cf9740 1 client.25427 shutdown 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 msgr2=0x55d15bd0d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bd0d950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feee0000c00 tx=0x7feee002a340 comp rx=0 tx=0).stop 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 shutdown_connections 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b9199f0 0x55d15bd30b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feeceff5640 0 client.25427 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b95b070 0x55d15bd2ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd0de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 --2- 172.21.15.31:0/992915433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bd0d950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 >> 172.21.15.31:0/992915433 conn(0x55d15b94f460 msgr2=0x55d15bd05400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 shutdown_connections 2024-09-21T06:12:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.345+0000 7feef7cf9740 1 -- 172.21.15.31:0/992915433 wait complete. 2024-09-21T06:12:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.346+0000 7feef7cf9740 20 client.25427 trim_cache size 0 max 16384 2024-09-21T06:12:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.380+0000 7feef7cf9740 1 Processor -- start 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 -- start start 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9199f0 0x55d15bd039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bcff710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bcffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feed800c770 con 0x55d15b9199f0 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feed800eff0 con 0x55d15b9321b0 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feef7cf9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feed800cde0 con 0x55d15b93a390 2024-09-21T06:12:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9199f0 0x55d15bd039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9199f0 0x55d15bd039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57602/0 (socket says 172.21.15.31:57602) 2024-09-21T06:12:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feeea7fc640 1 -- 172.21.15.31:0/280792073 learned_addr learned my addr 172.21.15.31:0/280792073 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feeeaffd640 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bcff710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.381+0000 7feeea7fc640 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bcffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.382+0000 7feeeaffd640 1 -- 172.21.15.31:0/280792073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 msgr2=0x55d15bcffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.382+0000 7feeeaffd640 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bcffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.382+0000 7feeeaffd640 1 -- 172.21.15.31:0/280792073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9199f0 msgr2=0x55d15bd039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.382+0000 7feeeaffd640 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9199f0 0x55d15bd039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.382+0000 7feeeaffd640 1 -- 172.21.15.31:0/280792073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d15bd0ec10 con 0x55d15b9321b0 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.382+0000 7feeeaffd640 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bcff710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55d15bc683a0 tx=0x7feedc014840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f45789796cd12db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.383+0000 7feed0ff9640 1 -- 172.21.15.31:0/280792073 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc014110 con 0x55d15b9321b0 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.383+0000 7feed0ff9640 1 -- 172.21.15.31:0/280792073 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feedc0142b0 con 0x55d15b9321b0 2024-09-21T06:12:31.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.383+0000 7feed0ff9640 1 -- 172.21.15.31:0/280792073 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc01e430 con 0x55d15b9321b0 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 -- 172.21.15.31:0/280792073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 msgr2=0x55d15bcff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bcff710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55d15bc683a0 tx=0x7feedc014840 comp rx=0 tx=0).stop 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 -- 172.21.15.31:0/280792073 shutdown_connections 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bcffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9321b0 0x55d15bcff710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 --2- 172.21.15.31:0/280792073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9199f0 0x55d15bd039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 -- 172.21.15.31:0/280792073 >> 172.21.15.31:0/280792073 conn(0x55d15b94f460 msgr2=0x55d15bd032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.385+0000 7feef7cf9740 1 -- 172.21.15.31:0/280792073 shutdown_connections 2024-09-21T06:12:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.386+0000 7feef7cf9740 1 -- 172.21.15.31:0/280792073 wait complete. 2024-09-21T06:12:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.386+0000 7feef7cf9740 1 -- start start 2024-09-21T06:12:31.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.387+0000 7feef7cf9740 1 Processor -- start 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.387+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 0x55d15bd344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feef7cf9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd31990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 0x55d15bd344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feeea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 0x55d15bd344c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44564/0 (socket says 172.21.15.31:44564) 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feeea7fc640 1 -- 172.21.15.31:0/4270332203 learned_addr learned my addr 172.21.15.31:0/4270332203 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd32010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feed800d4b0 con 0x55d15b9321b0 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feed800e920 con 0x55d15b9199f0 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feed800db80 con 0x55d15b93a390 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feef7cf9740 10 client.0 _ll_register_callbacks cb 0x7feee0045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:31.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feeeaffd640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd31990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.388+0000 7feeea7fc640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd32010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeea7fc640 1 -- 172.21.15.31:0/4270332203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 msgr2=0x55d15bd32010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeea7fc640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd32010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeea7fc640 1 -- 172.21.15.31:0/4270332203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 msgr2=0x55d15bd31990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeea7fc640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd31990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeea7fc640 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d15bd327a0 con 0x55d15b9199f0 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeea7fc640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 0x55d15bd344c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feee0056aa0 tx=0x7feee002a840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42bcfad9ff01695 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeceff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.389+0000 7feeceff5640 1 -- 172.21.15.31:0/4270332203 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee005b070 con 0x55d15b9199f0 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.390+0000 7feeceff5640 1 -- 172.21.15.31:0/4270332203 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feee0003c30 con 0x55d15b9199f0 2024-09-21T06:12:31.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.390+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d15bc76850 con 0x55d15b9199f0 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.390+0000 7feeceff5640 1 -- 172.21.15.31:0/4270332203 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7feee0047040 con 0x55d15b9199f0 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.390+0000 7feef7cf9740 10 client.25433 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.390+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d15bd9b370 con 0x55d15b9199f0 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feeceff5640 1 -- 172.21.15.31:0/4270332203 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee0048040 con 0x55d15b9199f0 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feeceff5640 1 -- 172.21.15.31:0/4270332203 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7feee005e020 con 0x55d15b9199f0 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feef7cf9740 10 client.25433 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feef7cf9740 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feef7cf9740 20 client.25433 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b95b070 0x55d15bd9dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:31.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feef7cf9740 4 client.25433 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:12:31.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.391+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d15bc76850 con 0x55d15b95b070 2024-09-21T06:12:31.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.392+0000 7feeeaffd640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b95b070 0x55d15bd9dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.393+0000 7feeceff5640 10 client.25433 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.393+0000 7feeeaffd640 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b95b070 0x55d15bd9dab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f5e5aa819e735 server_cookie=f92f46fb390108f0 in_seq=0 out_seq=0 2024-09-21T06:12:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.393+0000 7feeceff5640 1 -- 172.21.15.31:0/4270332203 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x55d15bc76850 con 0x55d15b95b070 2024-09-21T06:12:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.393+0000 7feeceff5640 10 client.25433 handle_command_reply: tid=0 2024-09-21T06:12:31.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.393+0000 7feef7cf9740 1 client.25433 shutdown 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.393+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 msgr2=0x55d15bd344c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 0x55d15bd344c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feee0056aa0 tx=0x7feee002a840 comp rx=0 tx=0).stop 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 shutdown_connections 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d15b95b070 0x55d15bd9dab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d15b93a390 0x55d15bd32010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d15b9321b0 0x55d15bd31990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 --2- 172.21.15.31:0/4270332203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d15b9199f0 0x55d15bd344c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feeceff5640 0 client.25433 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 >> 172.21.15.31:0/4270332203 conn(0x55d15b94f460 msgr2=0x55d15bd03170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.394+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 shutdown_connections 2024-09-21T06:12:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.396+0000 7feef7cf9740 1 -- 172.21.15.31:0/4270332203 wait complete. 2024-09-21T06:12:31.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.396+0000 7feef7cf9740 20 client.25433 trim_cache size 0 max 16384 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e", 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:12:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:12:31.401 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:12:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.396+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed40b6740 msgr2=0x7feed40b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.396+0000 7feef1785640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed40b6740 0x7feed40b8c50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feee4004af0 tx=0x7feee407be70 comp rx=0 tx=0).stop 2024-09-21T06:12:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.396+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 msgr2=0x7feeec1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec1987c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feee400eb10 tx=0x7feee400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 shutdown_connections 2024-09-21T06:12:31.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeec108cf0 0x7feeec1987c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed40b6740 0x7feed40b8c50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feeec1083a0 0x7feeec1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 --2- 172.21.15.31:0/2510383594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feeec102a60 0x7feeec19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.397+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 >> 172.21.15.31:0/2510383594 conn(0x7feeec079730 msgr2=0x7feeec104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.398+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 shutdown_connections 2024-09-21T06:12:31.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:12:31.398+0000 7feef1785640 1 -- 172.21.15.31:0/2510383594 wait complete. 2024-09-21T06:12:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:31 smithi031 ceph-mon[31237]: pgmap v347: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.7 MiB/s wr, 830 op/s 2024-09-21T06:12:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:31 smithi110 ceph-mon[34550]: pgmap v347: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.7 MiB/s wr, 830 op/s 2024-09-21T06:12:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:31 smithi107 ceph-mon[34380]: pgmap v347: 97 pgs: 97 active+clean; 1.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.7 MiB/s wr, 830 op/s 2024-09-21T06:12:32.259 INFO:tasks.cephfs.filesystem:scrub status for tag:4ad8636e-3cfe-4a90-96d1-05b9b1c9106e - {'path': '/', 'tag': '4ad8636e-3cfe-4a90-96d1-05b9b1c9106e', 'options': 'recursive,force'} 2024-09-21T06:12:32.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:32 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:12:32.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:32 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:12:32.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:32 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:12:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:33 smithi031 ceph-mon[31237]: pgmap v348: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T06:12:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:33 smithi110 ceph-mon[34550]: pgmap v348: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T06:12:33.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:33 smithi107 ceph-mon[34380]: pgmap v348: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-21T06:12:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 55.0 s, 4604.0 tps, lat 21.754 ms stddev 23.514 2024-09-21T06:12:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:35 smithi110 ceph-mon[34550]: pgmap v349: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 8.8 MiB/s wr, 838 op/s 2024-09-21T06:12:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:35 smithi107 ceph-mon[34380]: pgmap v349: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 8.8 MiB/s wr, 838 op/s 2024-09-21T06:12:35.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:35 smithi031 ceph-mon[31237]: pgmap v349: 97 pgs: 97 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 8.8 MiB/s wr, 838 op/s 2024-09-21T06:12:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:37 smithi110 ceph-mon[34550]: pgmap v350: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 12 MiB/s wr, 1.10k op/s 2024-09-21T06:12:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:37 smithi107 ceph-mon[34380]: pgmap v350: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 12 MiB/s wr, 1.10k op/s 2024-09-21T06:12:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:37 smithi031 ceph-mon[31237]: pgmap v350: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 12 MiB/s wr, 1.10k op/s 2024-09-21T06:12:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:39 smithi110 ceph-mon[34550]: pgmap v351: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.1 MiB/s wr, 822 op/s 2024-09-21T06:12:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:39 smithi107 ceph-mon[34380]: pgmap v351: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.1 MiB/s wr, 822 op/s 2024-09-21T06:12:39.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:39 smithi031 ceph-mon[31237]: pgmap v351: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.1 MiB/s wr, 822 op/s 2024-09-21T06:12:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 60.0 s, 4817.8 tps, lat 20.701 ms stddev 21.412 2024-09-21T06:12:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:41 smithi110 ceph-mon[34550]: pgmap v352: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 9.0 MiB/s wr, 806 op/s 2024-09-21T06:12:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:41 smithi107 ceph-mon[34380]: pgmap v352: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 9.0 MiB/s wr, 806 op/s 2024-09-21T06:12:41.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:41 smithi031 ceph-mon[31237]: pgmap v352: 97 pgs: 97 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 9.0 MiB/s wr, 806 op/s 2024-09-21T06:12:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.563+0000 7f19bd382640 1 -- 172.21.15.31:0/4152929248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b811cba0 msgr2=0x7f19b811efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.563+0000 7f19bd382640 1 --2- 172.21.15.31:0/4152929248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b811cba0 0x7f19b811efc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009ca0 tx=0x7f19b002a8d0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 -- 172.21.15.31:0/4152929248 shutdown_connections 2024-09-21T06:12:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 --2- 172.21.15.31:0/4152929248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b811cba0 0x7f19b811efc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 --2- 172.21.15.31:0/4152929248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b810a900 0x7f19b810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 --2- 172.21.15.31:0/4152929248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b8111ca0 0x7f19b810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 -- 172.21.15.31:0/4152929248 >> 172.21.15.31:0/4152929248 conn(0x7f19b8070c00 msgr2=0x7f19b8071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 -- 172.21.15.31:0/4152929248 shutdown_connections 2024-09-21T06:12:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.565+0000 7f19bd382640 1 -- 172.21.15.31:0/4152929248 wait complete. 2024-09-21T06:12:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.566+0000 7f19bd382640 1 Processor -- start 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 -- start start 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b810a900 0x7f19b81a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b8111ca0 0x7f19b81a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 0x7f19b81ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19b81226c0 con 0x7f19b811cba0 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8122540 con 0x7f19b8111ca0 2024-09-21T06:12:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.567+0000 7f19bd382640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19b8122840 con 0x7f19b810a900 2024-09-21T06:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.569+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b810a900 0x7f19b81a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.569+0000 7f19bcb81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 0x7f19b81ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.569+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b8111ca0 0x7f19b81a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.569+0000 7f19bcb81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 0x7f19b81ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46454/0 (socket says 172.21.15.31:46454) 2024-09-21T06:12:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.569+0000 7f19bcb81640 1 -- 172.21.15.31:0/3003324379 learned_addr learned my addr 172.21.15.31:0/3003324379 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.570+0000 7f19bcb81640 1 -- 172.21.15.31:0/3003324379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b810a900 msgr2=0x7f19b81a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.570+0000 7f19bcb81640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b810a900 0x7f19b81a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.570+0000 7f19bcb81640 1 -- 172.21.15.31:0/3003324379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b8111ca0 msgr2=0x7f19b81a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.570+0000 7f19bcb81640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b8111ca0 0x7f19b81a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.570+0000 7f19bcb81640 1 -- 172.21.15.31:0/3003324379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b81aef90 con 0x7f19b811cba0 2024-09-21T06:12:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.571+0000 7f19bcb81640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 0x7f19b81ae890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005c40 tx=0x7f19b0032fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.572+0000 7f19b57fa640 1 -- 172.21.15.31:0/3003324379 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0003bf0 con 0x7f19b811cba0 2024-09-21T06:12:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.572+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b81af220 con 0x7f19b811cba0 2024-09-21T06:12:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.572+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b8114390 con 0x7f19b811cba0 2024-09-21T06:12:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.572+0000 7f19b57fa640 1 -- 172.21.15.31:0/3003324379 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19b0003d90 con 0x7f19b811cba0 2024-09-21T06:12:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.572+0000 7f19b57fa640 1 -- 172.21.15.31:0/3003324379 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0030af0 con 0x7f19b811cba0 2024-09-21T06:12:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.576+0000 7f19c5633740 1 Processor -- start 2024-09-21T06:12:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.577+0000 7f19b57fa640 1 -- 172.21.15.31:0/3003324379 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f19b0030c90 con 0x7f19b811cba0 2024-09-21T06:12:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.577+0000 7f19c5633740 1 -- start start 2024-09-21T06:12:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.578+0000 7f19c5633740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e409991b0 0x561e40d3eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.579+0000 7f19b57fa640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6800 0x7f19a00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.579+0000 7f19b7fff640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6800 0x7f19a00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.579+0000 7f19b57fa640 1 -- 172.21.15.31:0/3003324379 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f19b01089d0 con 0x7f19b811cba0 2024-09-21T06:12:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.579+0000 7f19c5633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 0x561e40d3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.579+0000 7f19c5633740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 0x561e40d3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 0x561e40d3f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51094/0 (socket says 172.21.15.31:51094) 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 -- 172.21.15.31:0/1100979872 learned_addr learned my addr 172.21.15.31:0/1100979872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561e40cb36f0 con 0x561e409991b0 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b7fff640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6800 0x7f19a00b8d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19b80755c0 tx=0x7f19a800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19bcb81640 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561e40ca2b20 con 0x561e409784d0 2024-09-21T06:12:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561e40ca6930 con 0x561e409702f0 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e409991b0 0x561e40d3eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 -- 172.21.15.31:0/1100979872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 msgr2=0x561e40d3f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d3f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 -- 172.21.15.31:0/1100979872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e409991b0 msgr2=0x561e40d3eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e409991b0 0x561e40d3eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.580+0000 7f19b77fe640 1 -- 172.21.15.31:0/1100979872 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e40c68530 con 0x561e409702f0 2024-09-21T06:12:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.581+0000 7f19b77fe640 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 0x561e40d3f0e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00cf00 tx=0x7f19ac00c170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb37112b6bb79a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.582+0000 7f199e7fc640 1 -- 172.21.15.31:0/1100979872 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac01b070 con 0x561e409702f0 2024-09-21T06:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.582+0000 7f199e7fc640 1 -- 172.21.15.31:0/1100979872 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ac013e10 con 0x561e409702f0 2024-09-21T06:12:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.582+0000 7f199e7fc640 1 -- 172.21.15.31:0/1100979872 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac00f040 con 0x561e409702f0 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 msgr2=0x561e40d3f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 0x561e40d3f0e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00cf00 tx=0x7f19ac00c170 comp rx=0 tx=0).stop 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 shutdown_connections 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409702f0 0x561e40d3f0e0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00cf00 tx=0x7f19ac00c170 comp rx=0 tx=0).stop 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 --2- 172.21.15.31:0/1100979872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e409991b0 0x561e40d3eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.583+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 >> 172.21.15.31:0/1100979872 conn(0x561e4098d300 msgr2=0x561e40d3ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.584+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 shutdown_connections 2024-09-21T06:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.585+0000 7f19c5633740 1 -- 172.21.15.31:0/1100979872 wait complete. 2024-09-21T06:12:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.585+0000 7f19c5633740 1 -- start start 2024-09-21T06:12:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.585+0000 7f19c5633740 1 Processor -- start 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d4a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561e40bdaab0 con 0x561e40957d80 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561e40c48730 con 0x561e409784d0 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561e40d4d370 con 0x561e409991b0 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d4a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d4a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51108/0 (socket says 172.21.15.31:51108) 2024-09-21T06:12:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19b77fe640 1 -- 172.21.15.31:0/685211456 learned_addr learned my addr 172.21.15.31:0/685211456 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.586+0000 7f19c5633740 10 client.0 _ll_register_callbacks cb 0x561e40c9c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.587+0000 7f19b7fff640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19bcb81640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19b77fe640 1 -- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 msgr2=0x561e40d4a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19b77fe640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d4a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19b77fe640 1 -- 172.21.15.31:0/685211456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 msgr2=0x561e40d566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19b77fe640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19b77fe640 1 -- 172.21.15.31:0/685211456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e40a04790 con 0x561e409991b0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19b77fe640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d4a9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00ced0 tx=0x7f19ac016b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94aa88c63d7fd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19977fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19977fe640 1 -- 172.21.15.31:0/685211456 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac024020 con 0x561e409991b0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19977fe640 1 -- 172.21.15.31:0/685211456 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f19ac013c10 con 0x561e409991b0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.588+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ac013e10 con 0x561e409991b0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.589+0000 7f19977fe640 1 -- 172.21.15.31:0/685211456 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f19ac02f050 con 0x561e409991b0 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.589+0000 7f19c5633740 10 client.25459 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.589+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561e40a7d620 con 0x561e409991b0 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.589+0000 7f19977fe640 1 -- 172.21.15.31:0/685211456 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ac013e10 con 0x561e409991b0 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.589+0000 7f19977fe640 1 -- 172.21.15.31:0/685211456 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f19ac015070 con 0x561e409991b0 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.589+0000 7f19c5633740 10 client.25459 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.590+0000 7f19c5633740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.590+0000 7f19c5633740 20 client.25459 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.590+0000 7f19c5633740 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e4092d1c0 0x561e40d5c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.590+0000 7f19c5633740 4 client.25459 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.590+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e40cb5640 con 0x561e4092d1c0 2024-09-21T06:12:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.590+0000 7f19b7fff640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e4092d1c0 0x561e40d5c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.591+0000 7f19b7fff640 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e4092d1c0 0x561e40d5c470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6a4aed1b12968c server_cookie=5ac7e32e07099903 in_seq=0 out_seq=0 2024-09-21T06:12:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.591+0000 7f19977fe640 10 client.25459 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.592+0000 7f19977fe640 1 -- 172.21.15.31:0/685211456 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e40cb5640 con 0x561e4092d1c0 2024-09-21T06:12:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.592+0000 7f19977fe640 10 client.25459 handle_command_reply: tid=0 2024-09-21T06:12:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.592+0000 7f19c5633740 1 client.25459 shutdown 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 msgr2=0x561e40d4a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d4a9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19ac00ced0 tx=0x7f19ac016b40 comp rx=0 tx=0).stop 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 shutdown_connections 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e4092d1c0 0x561e40d5c470 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d4a9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 --2- 172.21.15.31:0/685211456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d4a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 >> 172.21.15.31:0/685211456 conn(0x561e4098d300 msgr2=0x561e40d3fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19977fe640 0 client.25459 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 shutdown_connections 2024-09-21T06:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 1 -- 172.21.15.31:0/685211456 wait complete. 2024-09-21T06:12:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.593+0000 7f19c5633740 20 client.25459 trim_cache size 0 max 16384 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 Processor -- start 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 -- start start 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d56660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d56ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19a400c700 con 0x561e40957d80 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19a400ef80 con 0x561e409784d0 2024-09-21T06:12:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19c5633740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19a400cd70 con 0x561e409991b0 2024-09-21T06:12:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19bcb81640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d56ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d56660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19bcb81640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d56ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51124/0 (socket says 172.21.15.31:51124) 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.624+0000 7f19bcb81640 1 -- 172.21.15.31:0/867869712 learned_addr learned my addr 172.21.15.31:0/867869712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f19b77fe640 1 -- 172.21.15.31:0/867869712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 msgr2=0x561e40d56ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f19b77fe640 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d56ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f19b77fe640 1 -- 172.21.15.31:0/867869712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 msgr2=0x561e40d40290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f19b77fe640 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f19b77fe640 1 -- 172.21.15.31:0/867869712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e40d36df0 con 0x561e409784d0 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f19b77fe640 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d56660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f19ac015f90 tx=0x7f19ac00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df5bb512fe2e8617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.625+0000 7f199dffb640 1 -- 172.21.15.31:0/867869712 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac038070 con 0x561e409784d0 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.626+0000 7f199dffb640 1 -- 172.21.15.31:0/867869712 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ac01c040 con 0x561e409784d0 2024-09-21T06:12:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.626+0000 7f199dffb640 1 -- 172.21.15.31:0/867869712 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac024040 con 0x561e409784d0 2024-09-21T06:12:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.627+0000 7f19c5633740 1 -- 172.21.15.31:0/867869712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 msgr2=0x561e40d56660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.627+0000 7f19c5633740 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d56660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f19ac015f90 tx=0x7f19ac00c7d0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.628+0000 7f19c5633740 1 -- 172.21.15.31:0/867869712 shutdown_connections 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.628+0000 7f19c5633740 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409991b0 0x561e40d56ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.628+0000 7f19c5633740 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409784d0 0x561e40d56660 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.628+0000 7f19c5633740 1 --2- 172.21.15.31:0/867869712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d40290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.628+0000 7f19c5633740 1 -- 172.21.15.31:0/867869712 >> 172.21.15.31:0/867869712 conn(0x561e4098d300 msgr2=0x561e40d3fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.628+0000 7f19c5633740 1 -- 172.21.15.31:0/867869712 shutdown_connections 2024-09-21T06:12:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.629+0000 7f19c5633740 1 -- 172.21.15.31:0/867869712 wait complete. 2024-09-21T06:12:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.630+0000 7f19c5633740 1 -- start start 2024-09-21T06:12:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.630+0000 7f19c5633740 1 Processor -- start 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.630+0000 7f19c5633740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.630+0000 7f19c5633740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 0x561e40d5d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.630+0000 7f19c5633740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409991b0 0x561e40dbebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 0x561e40d5d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 0x561e40d5d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51128/0 (socket says 172.21.15.31:51128) 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d5cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46494/0 (socket says 172.21.15.31:46494) 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 -- 172.21.15.31:0/3367818202 learned_addr learned my addr 172.21.15.31:0/3367818202 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19a400d440 con 0x561e40957d80 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19a400e8b0 con 0x561e409991b0 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19a400db10 con 0x561e409784d0 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19bcb81640 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409991b0 0x561e40dbebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19c5633740 10 client.0 _ll_register_callbacks cb 0x7f19ac019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 -- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409991b0 msgr2=0x561e40dbebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409991b0 0x561e40dbebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 -- 172.21.15.31:0/3367818202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 msgr2=0x561e40d5cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d5cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e40dbf220 con 0x561e409784d0 2024-09-21T06:12:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19b77fe640 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 0x561e40d5d310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19ac020470 tx=0x7f19ac0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=598f4f94392760e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.631+0000 7f19977fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.632+0000 7f19977fe640 1 -- 172.21.15.31:0/3367818202 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac041060 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.632+0000 7f19977fe640 1 -- 172.21.15.31:0/3367818202 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f19ac015680 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.632+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19ac01c040 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.632+0000 7f19977fe640 1 -- 172.21.15.31:0/3367818202 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f19ac038070 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.632+0000 7f19c5633740 10 client.25465 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561e40dc64f0 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19977fe640 1 -- 172.21.15.31:0/3367818202 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19ac016700 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19977fe640 1 -- 172.21.15.31:0/3367818202 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f19ac020b90 con 0x561e409784d0 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 10 client.25465 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:42.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 20 client.25465 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e40dc6b30 0x561e40dc8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 4 client.25465 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:12:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.633+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f19ac01c040 con 0x561e40dc6b30 2024-09-21T06:12:42.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.634+0000 7f19b7fff640 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e40dc6b30 0x561e40dc8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.636+0000 7f19b7fff640 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e40dc6b30 0x561e40dc8f50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509f1f1abaeae674 server_cookie=e97621b56f696093 in_seq=0 out_seq=0 2024-09-21T06:12:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.636+0000 7f19977fe640 10 client.25465 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19977fe640 1 -- 172.21.15.31:0/3367818202 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f19ac01c040 con 0x561e40dc6b30 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19977fe640 10 client.25465 handle_command_reply: tid=0 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19c5633740 1 client.25465 shutdown 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 msgr2=0x561e40d5d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19c5633740 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 0x561e40d5d310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f19ac020470 tx=0x7f19ac0047d0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 shutdown_connections 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.658+0000 7f19c5633740 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561e40dc6b30 0x561e40dc8f50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e409991b0 0x561e40dbebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561e409784d0 0x561e40d5d310 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 1 --2- 172.21.15.31:0/3367818202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561e40957d80 0x561e40d5cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 >> 172.21.15.31:0/3367818202 conn(0x561e4098d300 msgr2=0x561e40d3c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:42.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19977fe640 0 client.25465 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 shutdown_connections 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 1 -- 172.21.15.31:0/3367818202 wait complete. 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19c5633740 20 client.25465 trim_cache size 0 max 16384 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:12:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6800 msgr2=0x7f19a00b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.659+0000 7f19bd382640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6800 0x7f19a00b8d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f19b80755c0 tx=0x7f19a800b390 comp rx=0 tx=0).stop 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 msgr2=0x7f19b81ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 0x7f19b81ae890 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005c40 tx=0x7f19b0032fe0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 shutdown_connections 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19a00b6800 0x7f19a00b8d10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19b811cba0 0x7f19b81ae890 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b8111ca0 0x7f19b81a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 --2- 172.21.15.31:0/3003324379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19b810a900 0x7f19b81a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 >> 172.21.15.31:0/3003324379 conn(0x7f19b8070c00 msgr2=0x7f19b810f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 shutdown_connections 2024-09-21T06:12:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:42.660+0000 7f19bd382640 1 -- 172.21.15.31:0/3003324379 wait complete. 2024-09-21T06:12:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:43 smithi110 ceph-mon[34550]: pgmap v353: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-09-21T06:12:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:43 smithi107 ceph-mon[34380]: pgmap v353: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-09-21T06:12:43.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:43 smithi031 ceph-mon[31237]: pgmap v353: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2024-09-21T06:12:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 65.0 s, 5514.2 tps, lat 18.152 ms stddev 19.924 2024-09-21T06:12:45.687 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: pgmap v354: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:45 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: pgmap v354: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:45 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: pgmap v354: 97 pgs: 97 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:45 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:46.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:12:46.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:12:46.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:12:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:47 smithi110 ceph-mon[34550]: pgmap v355: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T06:12:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:47 smithi107 ceph-mon[34380]: pgmap v355: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T06:12:47.793 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:47 smithi031 ceph-mon[31237]: pgmap v355: 97 pgs: 97 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 15 MiB/s wr, 1.10k op/s 2024-09-21T06:12:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:49 smithi110 ceph-mon[34550]: pgmap v356: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 880 op/s 2024-09-21T06:12:49.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:49 smithi107 ceph-mon[34380]: pgmap v356: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 880 op/s 2024-09-21T06:12:49.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:49 smithi031 ceph-mon[31237]: pgmap v356: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 880 op/s 2024-09-21T06:12:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 70.0 s, 4979.8 tps, lat 20.119 ms stddev 20.932 2024-09-21T06:12:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:51 smithi110 ceph-mon[34550]: pgmap v357: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-09-21T06:12:51.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:51 smithi107 ceph-mon[34380]: pgmap v357: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-09-21T06:12:51.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:51 smithi031 ceph-mon[31237]: pgmap v357: 97 pgs: 97 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-09-21T06:12:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:53 smithi110 ceph-mon[34550]: pgmap v358: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-09-21T06:12:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:53 smithi107 ceph-mon[34380]: pgmap v358: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-09-21T06:12:53.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:53 smithi031 ceph-mon[31237]: pgmap v358: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 1.13k op/s 2024-09-21T06:12:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 75.0 s, 5271.2 tps, lat 18.902 ms stddev 20.534 2024-09-21T06:12:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:55 smithi110 ceph-mon[34550]: pgmap v359: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 793 op/s 2024-09-21T06:12:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:55 smithi107 ceph-mon[34380]: pgmap v359: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 793 op/s 2024-09-21T06:12:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 -- 172.21.15.31:0/3828213336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 msgr2=0x7fe2d4116b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/3828213336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 0x7fe2d4116b90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009900 tx=0x7fe2cc02a400 comp rx=0 tx=0).stop 2024-09-21T06:12:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 -- 172.21.15.31:0/3828213336 shutdown_connections 2024-09-21T06:12:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/3828213336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 0x7fe2d4116b90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/3828213336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410fa10 0x7fe2d410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/3828213336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d410a0e0 0x7fe2d410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.821+0000 7fe2dc28b640 1 -- 172.21.15.31:0/3828213336 >> 172.21.15.31:0/3828213336 conn(0x7fe2d4079630 msgr2=0x7fe2d41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.822+0000 7fe2dc28b640 1 -- 172.21.15.31:0/3828213336 shutdown_connections 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.822+0000 7fe2dc28b640 1 -- 172.21.15.31:0/3828213336 wait complete. 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 Processor -- start 2024-09-21T06:12:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 -- start start 2024-09-21T06:12:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 0x7fe2d41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d410fa10 0x7fe2d41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 0x7fe2d41a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d411a050 con 0x7fe2d410fa10 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4119ed0 con 0x7fe2d4110360 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2dc28b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d411a1d0 con 0x7fe2d410a0e0 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 0x7fe2d41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 0x7fe2d41a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50684/0 (socket says 172.21.15.31:50684) 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.823+0000 7fe2da027640 1 -- 172.21.15.31:0/1389083965 learned_addr learned my addr 172.21.15.31:0/1389083965 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.824+0000 7fe2da828640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 0x7fe2d41a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.824+0000 7fe2da027640 1 -- 172.21.15.31:0/1389083965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 msgr2=0x7fe2d41a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.824+0000 7fe2da027640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 0x7fe2d41a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.824+0000 7fe2da027640 1 -- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d410fa10 msgr2=0x7fe2d41a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.824+0000 7fe2da027640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d410fa10 0x7fe2d41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.824+0000 7fe2da027640 1 -- 172.21.15.31:0/1389083965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d41a6960 con 0x7fe2d410a0e0 2024-09-21T06:12:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.827+0000 7fe2da027640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 0x7fe2d41a0a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00b4a0 tx=0x7fe2bc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.829+0000 7fe2d37fe640 1 -- 172.21.15.31:0/1389083965 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc014070 con 0x7fe2d410a0e0 2024-09-21T06:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.829+0000 7fe2d37fe640 1 -- 172.21.15.31:0/1389083965 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc010ac0 con 0x7fe2d410a0e0 2024-09-21T06:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.829+0000 7fe2d37fe640 1 -- 172.21.15.31:0/1389083965 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc017c30 con 0x7fe2d410a0e0 2024-09-21T06:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.829+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d41a6c50 con 0x7fe2d410a0e0 2024-09-21T06:12:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.829+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d41ad3e0 con 0x7fe2d410a0e0 2024-09-21T06:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.834+0000 7fe2e31f6740 1 Processor -- start 2024-09-21T06:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.837+0000 7fe2d37fe640 1 -- 172.21.15.31:0/1389083965 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe2bc017070 con 0x7fe2d410a0e0 2024-09-21T06:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.838+0000 7fe2d37fe640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2c00b67b0 0x7fe2c00b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.838+0000 7fe2d37fe640 1 -- 172.21.15.31:0/1389083965 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe2bc0e6c70 con 0x7fe2d410a0e0 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.843+0000 7fe2da828640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2c00b67b0 0x7fe2c00b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 -- start start 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb6c2f0 0x55858ef3aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 0x55858ef3b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55858eeaf6f0 con 0x55858eb6c2f0 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55858ee9eb20 con 0x55858eb951b0 2024-09-21T06:12:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.845+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55858eea2930 con 0x55858eb744d0 2024-09-21T06:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.846+0000 7fe2da828640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 0x55858ef3b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.846+0000 7fe2da828640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 0x55858ef3b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50688/0 (socket says 172.21.15.31:50688) 2024-09-21T06:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.846+0000 7fe2da828640 1 -- 172.21.15.31:0/372421389 learned_addr learned my addr 172.21.15.31:0/372421389 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2d9826640 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb6c2f0 0x55858ef3aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2da828640 1 -- 172.21.15.31:0/372421389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 msgr2=0x55858ef3a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2da828640 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2da828640 1 -- 172.21.15.31:0/372421389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb6c2f0 msgr2=0x55858ef3aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2da828640 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb6c2f0 0x55858ef3aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2da828640 1 -- 172.21.15.31:0/372421389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858ee64530 con 0x55858eb744d0 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2d9826640 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb6c2f0 0x55858ef3aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.847+0000 7fe2da828640 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 0x55858ef3b430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0096c0 tx=0x7fe2cc044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172bbe8c152711e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.848+0000 7fe2da828640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2c00b67b0 0x7fe2c00b8cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc031440 tx=0x7fe2cc004550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.849+0000 7fe2d0bfe640 1 -- 172.21.15.31:0/372421389 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc035040 con 0x55858eb744d0 2024-09-21T06:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.849+0000 7fe2d0bfe640 1 -- 172.21.15.31:0/372421389 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2cc04f070 con 0x55858eb744d0 2024-09-21T06:12:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.849+0000 7fe2d0bfe640 1 -- 172.21.15.31:0/372421389 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc04aaa0 con 0x55858eb744d0 2024-09-21T06:12:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.851+0000 7fe2e31f6740 1 -- 172.21.15.31:0/372421389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 msgr2=0x55858ef3b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.851+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 0x55858ef3b430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc0096c0 tx=0x7fe2cc044000 comp rx=0 tx=0).stop 2024-09-21T06:12:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.853+0000 7fe2e31f6740 1 -- 172.21.15.31:0/372421389 shutdown_connections 2024-09-21T06:12:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.853+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb744d0 0x55858ef3b430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.853+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb6c2f0 0x55858ef3aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.853+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/372421389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.853+0000 7fe2e31f6740 1 -- 172.21.15.31:0/372421389 >> 172.21.15.31:0/372421389 conn(0x55858eb89300 msgr2=0x55858ef388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.853+0000 7fe2e31f6740 1 -- 172.21.15.31:0/372421389 shutdown_connections 2024-09-21T06:12:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.854+0000 7fe2e31f6740 1 -- 172.21.15.31:0/372421389 wait complete. 2024-09-21T06:12:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.854+0000 7fe2e31f6740 1 -- start start 2024-09-21T06:12:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.855+0000 7fe2e31f6740 1 Processor -- start 2024-09-21T06:12:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.855+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef46610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50692/0 (socket says 172.21.15.31:50692) 2024-09-21T06:12:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2da027640 1 -- 172.21.15.31:0/1872737776 learned_addr learned my addr 172.21.15.31:0/1872737776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55858edd6ab0 con 0x55858eb744d0 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55858ee44730 con 0x55858eb951b0 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2da828640 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.856+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55858ef5a0e0 con 0x55858eb6c2f0 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.857+0000 7fe2da027640 1 -- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 msgr2=0x55858ef633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.857+0000 7fe2da027640 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.857+0000 7fe2da027640 1 -- 172.21.15.31:0/1872737776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 msgr2=0x55858ef46610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.857+0000 7fe2da027640 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef46610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.857+0000 7fe2da027640 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858ec00790 con 0x55858eb6c2f0 2024-09-21T06:12:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.857+0000 7fe2da027640 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef460d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0a9d10 tx=0x7fe2bc0ac4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd144f760cc6a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.858+0000 7fe2c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:12:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.858+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/1872737776 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc01e070 con 0x55858eb6c2f0 2024-09-21T06:12:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.858+0000 7fe2e31f6740 10 client.0 _ll_register_callbacks cb 0x55858ee986b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.858+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/1872737776 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe2bc0e4620 con 0x55858eb6c2f0 2024-09-21T06:12:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.858+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2cc04f070 con 0x55858eb6c2f0 2024-09-21T06:12:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.858+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/1872737776 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe2bc0e4820 con 0x55858eb6c2f0 2024-09-21T06:12:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.859+0000 7fe2e31f6740 10 client.25477 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.859+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55858ec79620 con 0x55858eb6c2f0 2024-09-21T06:12:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.859+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/1872737776 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe2bc010610 con 0x55858eb6c2f0 2024-09-21T06:12:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.860+0000 7fe2e31f6740 10 client.25477 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.860+0000 7fe2e31f6740 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.860+0000 7fe2e31f6740 20 client.25477 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.860+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb53d80 0x55858ef68ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.861+0000 7fe2e31f6740 4 client.25477 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:12:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.861+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe2cc04f070 con 0x55858eb53d80 2024-09-21T06:12:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.861+0000 7fe2d9826640 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb53d80 0x55858ef68ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.863+0000 7fe2c9ffb640 10 client.25477 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:55.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.864+0000 7fe2d9826640 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb53d80 0x55858ef68ee0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe9c2ee5852c6d4 server_cookie=70e0d6f42efeba8f in_seq=0 out_seq=0 2024-09-21T06:12:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.865+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/1872737776 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe2cc04f070 con 0x55858eb53d80 2024-09-21T06:12:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.865+0000 7fe2c9ffb640 10 client.25477 handle_command_reply: tid=0 2024-09-21T06:12:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.865+0000 7fe2e31f6740 1 client.25477 shutdown 2024-09-21T06:12:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 msgr2=0x55858ef460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef460d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0a9d10 tx=0x7fe2bc0ac4b0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 shutdown_connections 2024-09-21T06:12:55.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb53d80 0x55858ef68ee0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2c9ffb640 0 client.25477 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef46610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/1872737776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef460d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.867+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 >> 172.21.15.31:0/1872737776 conn(0x55858eb89300 msgr2=0x55858ef3b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.868+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 shutdown_connections 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.868+0000 7fe2e31f6740 1 -- 172.21.15.31:0/1872737776 wait complete. 2024-09-21T06:12:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.869+0000 7fe2e31f6740 20 client.25477 trim_cache size 0 max 16384 2024-09-21T06:12:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:55 smithi031 ceph-mon[31237]: pgmap v359: 97 pgs: 97 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 793 op/s 2024-09-21T06:12:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 Processor -- start 2024-09-21T06:12:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 -- start start 2024-09-21T06:12:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef47fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4082220 con 0x55858eb744d0 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41adcf0 con 0x55858eb951b0 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4082890 con 0x55858eb6c2f0 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50702/0 (socket says 172.21.15.31:50702) 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 -- 172.21.15.31:0/2335727841 learned_addr learned my addr 172.21.15.31:0/2335727841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da828640 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 -- 172.21.15.31:0/2335727841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 msgr2=0x55858ef3c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 -- 172.21.15.31:0/2335727841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 msgr2=0x55858ef47fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.897+0000 7fe2da027640 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef47fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.898+0000 7fe2da027640 1 -- 172.21.15.31:0/2335727841 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858ef48690 con 0x55858eb6c2f0 2024-09-21T06:12:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.898+0000 7fe2da027640 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef465b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009420 tx=0x7fe2bc0d0000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14e9af17a0627f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.898+0000 7fe2cbfff640 1 -- 172.21.15.31:0/2335727841 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc017360 con 0x55858eb6c2f0 2024-09-21T06:12:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.898+0000 7fe2cbfff640 1 -- 172.21.15.31:0/2335727841 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc0ac5b0 con 0x55858eb6c2f0 2024-09-21T06:12:55.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.899+0000 7fe2cbfff640 1 -- 172.21.15.31:0/2335727841 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0e4620 con 0x55858eb6c2f0 2024-09-21T06:12:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.900+0000 7fe2e31f6740 1 -- 172.21.15.31:0/2335727841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 msgr2=0x55858ef465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.901+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef465b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009420 tx=0x7fe2bc0d0000 comp rx=0 tx=0).stop 2024-09-21T06:12:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.901+0000 7fe2e31f6740 1 -- 172.21.15.31:0/2335727841 shutdown_connections 2024-09-21T06:12:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.901+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef3c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.901+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef47fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.901+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/2335727841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef465b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.901+0000 7fe2e31f6740 1 -- 172.21.15.31:0/2335727841 >> 172.21.15.31:0/2335727841 conn(0x55858eb89300 msgr2=0x55858ef3b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.902+0000 7fe2e31f6740 1 -- 172.21.15.31:0/2335727841 shutdown_connections 2024-09-21T06:12:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.902+0000 7fe2e31f6740 1 -- 172.21.15.31:0/2335727841 wait complete. 2024-09-21T06:12:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.903+0000 7fe2e31f6740 1 -- start start 2024-09-21T06:12:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.903+0000 7fe2e31f6740 1 Processor -- start 2024-09-21T06:12:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef5f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef5aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4082f60 con 0x55858eb744d0 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d41ad670 con 0x55858eb951b0 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4083630 con 0x55858eb6c2f0 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2e31f6740 10 client.0 _ll_register_callbacks cb 0x7fe2bc0ac870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da828640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef5f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da027640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef5f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50716/0 (socket says 172.21.15.31:50716) 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da828640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef5b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39690/0 (socket says 172.21.15.31:39690) 2024-09-21T06:12:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da027640 1 -- 172.21.15.31:0/698109160 learned_addr learned my addr 172.21.15.31:0/698109160 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da828640 1 -- 172.21.15.31:0/698109160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 msgr2=0x55858ef5f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da828640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef5f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da828640 1 -- 172.21.15.31:0/698109160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 msgr2=0x55858ef5aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2d9826640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef5aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.904+0000 7fe2da828640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef5aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2da828640 1 -- 172.21.15.31:0/698109160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858ef5b9e0 con 0x55858eb951b0 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2da027640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef5f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2d9826640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef5aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2da828640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef5b3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55858ef48d00 tx=0x7fe2cc04b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5a9444820ea97b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:12:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/698109160 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc035040 con 0x55858eb951b0 2024-09-21T06:12:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.905+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/698109160 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe2cc04bb70 con 0x55858eb951b0 2024-09-21T06:12:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.906+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2bc0ac5b0 con 0x55858eb951b0 2024-09-21T06:12:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.906+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/698109160 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe2cc04bd70 con 0x55858eb951b0 2024-09-21T06:12:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.906+0000 7fe2e31f6740 10 client.25463 fetch_fsmap learned FSMap version 19 2024-09-21T06:12:55.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.906+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55858efd3690 con 0x55858eb951b0 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.906+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/698109160 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc0ac5b0 con 0x55858eb951b0 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/698109160 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe2cc05e470 con 0x55858eb951b0 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2e31f6740 10 client.25463 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2e31f6740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2e31f6740 20 client.25463 populate_metadata read hostname 'smithi031' 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb291c0 0x55858efd58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2e31f6740 4 client.25463 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.907+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55858eeb1640 con 0x55858eb291c0 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.908+0000 7fe2da027640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb291c0 0x55858efd58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.908+0000 7fe2da027640 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb291c0 0x55858efd58d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff84ffe0ebc410 server_cookie=ae83f00775f9f8de in_seq=0 out_seq=0 2024-09-21T06:12:55.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.908+0000 7fe2c9ffb640 10 client.25463 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2c9ffb640 1 -- 172.21.15.31:0/698109160 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55858eeb1640 con 0x55858eb291c0 2024-09-21T06:12:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2c9ffb640 10 client.25463 handle_command_reply: tid=0 2024-09-21T06:12:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 client.25463 shutdown 2024-09-21T06:12:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 msgr2=0x55858ef5b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef5b3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55858ef48d00 tx=0x7fe2cc04b780 comp rx=0 tx=0).stop 2024-09-21T06:12:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 shutdown_connections 2024-09-21T06:12:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55858eb291c0 0x55858efd58d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55858eb951b0 0x55858ef5b3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55858eb744d0 0x55858ef5aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 --2- 172.21.15.31:0/698109160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55858eb6c2f0 0x55858ef5f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 >> 172.21.15.31:0/698109160 conn(0x55858eb89300 msgr2=0x55858ef3c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2c9ffb640 0 client.25463 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 shutdown_connections 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 1 -- 172.21.15.31:0/698109160 wait complete. 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.930+0000 7fe2e31f6740 20 client.25463 trim_cache size 0 max 16384 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:12:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:12:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:12:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:12:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.931+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2c00b67b0 msgr2=0x7fe2c00b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.931+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2c00b67b0 0x7fe2c00b8cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc031440 tx=0x7fe2cc004550 comp rx=0 tx=0).stop 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.931+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 msgr2=0x7fe2d41a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.931+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 0x7fe2d41a0a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00b4a0 tx=0x7fe2bc00b960 comp rx=0 tx=0).stop 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.931+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 shutdown_connections 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.931+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d4110360 0x7fe2d41a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.932+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2c00b67b0 0x7fe2c00b8cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.932+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2d410fa10 0x7fe2d41a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.932+0000 7fe2dc28b640 1 --2- 172.21.15.31:0/1389083965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2d410a0e0 0x7fe2d41a0a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.932+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 >> 172.21.15.31:0/1389083965 conn(0x7fe2d4079630 msgr2=0x7fe2d410bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.932+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 shutdown_connections 2024-09-21T06:12:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:12:55.932+0000 7fe2dc28b640 1 -- 172.21.15.31:0/1389083965 wait complete. 2024-09-21T06:12:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:57 smithi110 ceph-mon[34550]: pgmap v360: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-21T06:12:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:57 smithi107 ceph-mon[34380]: pgmap v360: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-21T06:12:57.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:57 smithi031 ceph-mon[31237]: pgmap v360: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-21T06:12:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:12:59 smithi110 ceph-mon[34550]: pgmap v361: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 784 op/s 2024-09-21T06:12:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:12:59 smithi107 ceph-mon[34380]: pgmap v361: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 784 op/s 2024-09-21T06:12:59.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:12:59 smithi031 ceph-mon[31237]: pgmap v361: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 784 op/s 2024-09-21T06:13:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 80.0 s, 4954.2 tps, lat 20.198 ms stddev 21.120 2024-09-21T06:13:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:01 smithi110 ceph-mon[34550]: pgmap v362: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-21T06:13:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:01 smithi107 ceph-mon[34380]: pgmap v362: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-21T06:13:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:01 smithi031 ceph-mon[31237]: pgmap v362: 97 pgs: 97 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-21T06:13:02.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:13:02.570 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.110+0000 7ff3693f4640 1 -- 172.21.15.31:0/938897208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364105300 msgr2=0x7ff36410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.110+0000 7ff3693f4640 1 --2- 172.21.15.31:0/938897208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364105300 0x7ff36410d850 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009900 tx=0x7ff34c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 -- 172.21.15.31:0/938897208 shutdown_connections 2024-09-21T06:13:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 --2- 172.21.15.31:0/938897208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff36410de10 0x7ff364110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 --2- 172.21.15.31:0/938897208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff364105300 0x7ff36410d850 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 --2- 172.21.15.31:0/938897208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff364104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 -- 172.21.15.31:0/938897208 >> 172.21.15.31:0/938897208 conn(0x7ff364079730 msgr2=0x7ff364079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 -- 172.21.15.31:0/938897208 shutdown_connections 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.111+0000 7ff3693f4640 1 -- 172.21.15.31:0/938897208 wait complete. 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 Processor -- start 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 -- start start 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff36419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 0x7ff36419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff36410de10 0x7ff36419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff364113ac0 con 0x7ff36410de10 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff364113940 con 0x7ff364105300 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3693f4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff364113c40 con 0x7ff3641049b0 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 0x7ff36419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff36410de10 0x7ff36419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff36419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 0x7ff36419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46972/0 (socket says 172.21.15.31:46972) 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff35bfff640 1 -- 172.21.15.31:0/3871491922 learned_addr learned my addr 172.21.15.31:0/3871491922 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff36419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:39946/0 (socket says 172.21.15.31:39946) 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.112+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff36410de10 0x7ff36419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59850/0 (socket says 172.21.15.31:59850) 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff35bfff640 1 -- 172.21.15.31:0/3871491922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 msgr2=0x7ff36419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff35bfff640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff36419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff35bfff640 1 -- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff36410de10 msgr2=0x7ff36419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff35bfff640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff36410de10 0x7ff36419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff35bfff640 1 -- 172.21.15.31:0/3871491922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3641a0400 con 0x7ff364105300 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff362ffd640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff36419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff35bfff640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 0x7ff36419aa30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005ab0 tx=0x7ff34c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff360ff9640 1 -- 172.21.15.31:0/3871491922 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c03f070 con 0x7ff364105300 2024-09-21T06:13:03.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3641a0690 con 0x7ff364105300 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff360ff9640 1 -- 172.21.15.31:0/3871491922 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff34c031b50 con 0x7ff364105300 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff360ff9640 1 -- 172.21.15.31:0/3871491922 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c035030 con 0x7ff364105300 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.113+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3641a6e50 con 0x7ff364105300 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.117+0000 7ff36f968740 1 Processor -- start 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.117+0000 7ff36f968740 1 -- start start 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.117+0000 7ff360ff9640 1 -- 172.21.15.31:0/3871491922 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff34c042070 con 0x7ff364105300 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff36f968740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff36f968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2eda2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff36f968740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff36f968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ed18e50 con 0x55ed2e9d21b0 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff36f968740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ed08280 con 0x55ed2e9fb070 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff36f968740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ed2ed0c090 con 0x55ed2e9da390 2024-09-21T06:13:03.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2eda2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2eda2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59854/0 (socket says 172.21.15.31:59854) 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 -- 172.21.15.31:0/2858208903 learned_addr learned my addr 172.21.15.31:0/2858208903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff3637fe640 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff35bfff640 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 -- 172.21.15.31:0/2858208903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 msgr2=0x55ed2eda30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 -- 172.21.15.31:0/2858208903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 msgr2=0x55ed2eda2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.118+0000 7ff362ffd640 1 -- 172.21.15.31:0/2858208903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2ea66650 con 0x55ed2e9d21b0 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff362ffd640 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2eda2b90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff3540114f0 tx=0x7ff3540119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccfbbab6117d0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff360ff9640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3500b6680 0x7ff3500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff3597fa640 1 -- 172.21.15.31:0/2858208903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35401e070 con 0x55ed2e9d21b0 2024-09-21T06:13:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff3637fe640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3500b6680 0x7ff3500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff3597fa640 1 -- 172.21.15.31:0/2858208903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff35400fa10 con 0x55ed2e9d21b0 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff3597fa640 1 -- 172.21.15.31:0/2858208903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354019af0 con 0x55ed2e9d21b0 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.119+0000 7ff360ff9640 1 -- 172.21.15.31:0/3871491922 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff34c107920 con 0x7ff364105300 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 -- 172.21.15.31:0/2858208903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 msgr2=0x55ed2eda2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2eda2b90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff3540114f0 tx=0x7ff3540119b0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff3637fe640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3500b6680 0x7ff3500b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff35c005fd0 tx=0x7ff35c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 -- 172.21.15.31:0/2858208903 shutdown_connections 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2eda2b90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 --2- 172.21.15.31:0/2858208903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 -- 172.21.15.31:0/2858208903 >> 172.21.15.31:0/2858208903 conn(0x55ed2e9ef460 msgr2=0x55ed2eda0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 -- 172.21.15.31:0/2858208903 shutdown_connections 2024-09-21T06:13:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.120+0000 7ff36f968740 1 -- 172.21.15.31:0/2858208903 wait complete. 2024-09-21T06:13:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.121+0000 7ff36f968740 1 -- start start 2024-09-21T06:13:03.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.121+0000 7ff36f968740 1 Processor -- start 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2edb2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ed2eb69700 con 0x55ed2e9fb070 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ed2eca3110 con 0x55ed2e9d21b0 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ed2edb5190 con 0x55ed2e9da390 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff36f968740 10 client.0 _ll_register_callbacks cb 0x55ed2ecbc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:03.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47002/0 (socket says 172.21.15.31:47002) 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 -- 172.21.15.31:0/1589930169 learned_addr learned my addr 172.21.15.31:0/1589930169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2edb2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff3637fe640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 -- 172.21.15.31:0/1589930169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 msgr2=0x55ed2edb2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2edb2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 -- 172.21.15.31:0/1589930169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9fb070 msgr2=0x55ed2edbe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff362ffd640 1 -- 172.21.15.31:0/1589930169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2ecd0c40 con 0x55ed2e9d21b0 2024-09-21T06:13:03.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.122+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2edb2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff362ffd640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb2110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3540114c0 tx=0x7ff35400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6992a19d0d17c2eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff34a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff34a7fc640 1 -- 172.21.15.31:0/1589930169 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354026020 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff34a7fc640 1 -- 172.21.15.31:0/1589930169 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff354004040 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35400fa10 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff34a7fc640 1 -- 172.21.15.31:0/1589930169 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff354031050 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff36f968740 10 client.25475 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ed2edc0ea0 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff34a7fc640 1 -- 172.21.15.31:0/1589930169 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff35400fa10 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.123+0000 7ff34a7fc640 1 -- 172.21.15.31:0/1589930169 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff354021030 con 0x55ed2e9d21b0 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff36f968740 10 client.25475 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff36f968740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:03.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff36f968740 20 client.25475 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff36f968740 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e9b99f0 0x55ed2edc42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff36f968740 4 client.25475 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed2ed1ada0 con 0x55ed2e9b99f0 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e9b99f0 0x55ed2edc42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff34a7fc640 10 client.25475 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.124+0000 7ff35bfff640 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e9b99f0 0x55ed2edc42b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3084ee300c7bb73 server_cookie=ff34140beb53c761 in_seq=0 out_seq=0 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff34a7fc640 1 -- 172.21.15.31:0/1589930169 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed2ed1ada0 con 0x55ed2e9b99f0 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff34a7fc640 10 client.25475 handle_command_reply: tid=0 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff36f968740 1 client.25475 shutdown 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 msgr2=0x55ed2edb2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff36f968740 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb2110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3540114c0 tx=0x7ff35400f790 comp rx=0 tx=0).stop 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 shutdown_connections 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff36f968740 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e9b99f0 0x55ed2edc42b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.125+0000 7ff36f968740 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff36f968740 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2edb2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff36f968740 1 --2- 172.21.15.31:0/1589930169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb2110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 >> 172.21.15.31:0/1589930169 conn(0x55ed2e9ef460 msgr2=0x55ed2ed9eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff34a7fc640 0 client.25475 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 shutdown_connections 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff36f968740 1 -- 172.21.15.31:0/1589930169 wait complete. 2024-09-21T06:13:03.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.126+0000 7ff36f968740 20 client.25475 trim_cache size 0 max 16384 2024-09-21T06:13:03.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 Processor -- start 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 -- start start 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2ed9fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ab490 con 0x55ed2e9d21b0 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.145+0000 7ff36f968740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641add10 con 0x55ed2e9fb070 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff36f968740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641abb00 con 0x55ed2e9da390 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2ed9fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59874/0 (socket says 172.21.15.31:59874) 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 -- 172.21.15.31:0/468348195 learned_addr learned my addr 172.21.15.31:0/468348195 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:03.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2ed9fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:39980/0 (socket says 172.21.15.31:39980) 2024-09-21T06:13:03.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 -- 172.21.15.31:0/468348195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 msgr2=0x55ed2ed9fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2ed9fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 -- 172.21.15.31:0/468348195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 msgr2=0x55ed2edbe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 -- 172.21.15.31:0/468348195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2edb1070 con 0x55ed2e9d21b0 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff362ffd640 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb25d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff3540114f0 tx=0x7ff354005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5783e4e8ca44c6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff358ff9640 1 -- 172.21.15.31:0/468348195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35403a070 con 0x55ed2e9d21b0 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff358ff9640 1 -- 172.21.15.31:0/468348195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff354019410 con 0x55ed2e9d21b0 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.146+0000 7ff358ff9640 1 -- 172.21.15.31:0/468348195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354026040 con 0x55ed2e9d21b0 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.147+0000 7ff36f968740 1 -- 172.21.15.31:0/468348195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 msgr2=0x55ed2edb25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.147+0000 7ff36f968740 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb25d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff3540114f0 tx=0x7ff354005fd0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.147+0000 7ff36f968740 1 -- 172.21.15.31:0/468348195 shutdown_connections 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.147+0000 7ff36f968740 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2edbe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2ed9fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 --2- 172.21.15.31:0/468348195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edb25d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 -- 172.21.15.31:0/468348195 >> 172.21.15.31:0/468348195 conn(0x55ed2e9ef460 msgr2=0x55ed2edb1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 -- 172.21.15.31:0/468348195 shutdown_connections 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 -- 172.21.15.31:0/468348195 wait complete. 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 -- start start 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 Processor -- start 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.148+0000 7ff36f968740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff36f968740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff36f968740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ac1d0 con 0x55ed2e9d21b0 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff36f968740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ad640 con 0x55ed2e9fb070 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff36f968740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3641ac8a0 con 0x55ed2e9da390 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff36f968740 10 client.0 _ll_register_callbacks cb 0x7ff35401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59882/0 (socket says 172.21.15.31:59882) 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 -- 172.21.15.31:0/434265847 learned_addr learned my addr 172.21.15.31:0/434265847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff3637fe640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff35bfff640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 -- 172.21.15.31:0/434265847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 msgr2=0x55ed2eda8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 -- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 msgr2=0x55ed2eda8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff362ffd640 1 -- 172.21.15.31:0/434265847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2eda9740 con 0x55ed2e9d21b0 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff35bfff640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.149+0000 7ff3637fe640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff362ffd640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edac5d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff354014040 tx=0x7ff354019cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a67f6b2b56f9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff34a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff34a7fc640 1 -- 172.21.15.31:0/434265847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354044060 con 0x55ed2e9d21b0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff34a7fc640 1 -- 172.21.15.31:0/434265847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff354035430 con 0x55ed2e9d21b0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff354019410 con 0x55ed2e9d21b0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff34a7fc640 1 -- 172.21.15.31:0/434265847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff35403a070 con 0x55ed2e9d21b0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff36f968740 10 client.15744 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ed2edbaa90 con 0x55ed2e9d21b0 2024-09-21T06:13:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff34a7fc640 1 -- 172.21.15.31:0/434265847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff354019410 con 0x55ed2e9d21b0 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff34a7fc640 1 -- 172.21.15.31:0/434265847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff354021030 con 0x55ed2e9d21b0 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff36f968740 10 client.15744 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.150+0000 7ff36f968740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff36f968740 20 client.15744 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff36f968740 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e98eda0 0x55ed2ee35310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff36f968740 4 client.15744 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed2ed16850 con 0x55ed2e98eda0 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff35bfff640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e98eda0 0x55ed2ee35310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff34a7fc640 10 client.15744 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff35bfff640 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e98eda0 0x55ed2ee35310 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f64bc218f63cb500 server_cookie=39f888f5efee37df in_seq=0 out_seq=0 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff34a7fc640 1 -- 172.21.15.31:0/434265847 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55ed2ed16850 con 0x55ed2e98eda0 2024-09-21T06:13:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff34a7fc640 10 client.15744 handle_command_reply: tid=0 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.151+0000 7ff36f968740 1 client.15744 shutdown 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 msgr2=0x55ed2edac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edac5d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff354014040 tx=0x7ff354019cc0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 shutdown_connections 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ed2e98eda0 0x55ed2ee35310 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff34a7fc640 0 client.15744 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ed2e9fb070 0x55ed2eda8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ed2e9da390 0x55ed2eda8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 --2- 172.21.15.31:0/434265847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed2e9d21b0 0x55ed2edac5d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 >> 172.21.15.31:0/434265847 conn(0x55ed2e9ef460 msgr2=0x55ed2edb1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 shutdown_connections 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 1 -- 172.21.15.31:0/434265847 wait complete. 2024-09-21T06:13:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff36f968740 20 client.15744 trim_cache size 0 max 16384 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:13:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:13:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:03.163 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:13:03.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3500b6680 msgr2=0x7ff3500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff3693f4640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3500b6680 0x7ff3500b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff35c005fd0 tx=0x7ff35c009500 comp rx=0 tx=0).stop 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 msgr2=0x7ff36419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.152+0000 7ff3693f4640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 0x7ff36419aa30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005ab0 tx=0x7ff34c003b90 comp rx=0 tx=0).stop 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 shutdown_connections 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3500b6680 0x7ff3500b8b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff36410de10 0x7ff36419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff364105300 0x7ff36419aa30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 --2- 172.21.15.31:0/3871491922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3641049b0 0x7ff36419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 >> 172.21.15.31:0/3871491922 conn(0x7ff364079730 msgr2=0x7ff364100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 shutdown_connections 2024-09-21T06:13:03.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:03.153+0000 7ff3693f4640 1 -- 172.21.15.31:0/3871491922 wait complete. 2024-09-21T06:13:03.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:03 smithi031 ceph-mon[31237]: pgmap v363: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-09-21T06:13:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:03 smithi110 ceph-mon[34550]: pgmap v363: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-09-21T06:13:03.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:03 smithi107 ceph-mon[34380]: pgmap v363: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2024-09-21T06:13:03.928 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:13:03.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:13:04.237 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:04.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:04 smithi031 ceph-mon[31237]: pgmap v364: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 737 op/s 2024-09-21T06:13:04.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:04 smithi110 ceph-mon[34550]: pgmap v364: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 737 op/s 2024-09-21T06:13:04.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:04 smithi107 ceph-mon[34380]: pgmap v364: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 737 op/s 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 -- 172.21.15.31:0/3036825680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 msgr2=0x7fd380102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 --2- 172.21.15.31:0/3036825680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 0x7fd380102e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd370009900 tx=0x7fd37002a410 comp rx=0 tx=0).stop 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 -- 172.21.15.31:0/3036825680 shutdown_connections 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 --2- 172.21.15.31:0/3036825680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd380108cf0 0x7fd38010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 --2- 172.21.15.31:0/3036825680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3801083a0 0x7fd3801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 --2- 172.21.15.31:0/3036825680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 0x7fd380102e70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.797+0000 7fd385618640 1 -- 172.21.15.31:0/3036825680 >> 172.21.15.31:0/3036825680 conn(0x7fd380079730 msgr2=0x7fd380079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.798+0000 7fd385618640 1 -- 172.21.15.31:0/3036825680 shutdown_connections 2024-09-21T06:13:04.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.798+0000 7fd385618640 1 -- 172.21.15.31:0/3036825680 wait complete. 2024-09-21T06:13:04.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.798+0000 7fd385618640 1 Processor -- start 2024-09-21T06:13:04.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 -- start start 2024-09-21T06:13:04.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 0x7fd380074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:04.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3801083a0 0x7fd38006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:04.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd380108cf0 0x7fd38006e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd380112960 con 0x7fd380102a60 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd3801127e0 con 0x7fd380108cf0 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd385618640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd380112ae0 con 0x7fd3801083a0 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd37effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 0x7fd380074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd37e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3801083a0 0x7fd38006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd37f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd380108cf0 0x7fd38006e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd37e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3801083a0 0x7fd38006e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40020/0 (socket says 172.21.15.31:40020) 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.799+0000 7fd37e7fc640 1 -- 172.21.15.31:0/4027262480 learned_addr learned my addr 172.21.15.31:0/4027262480 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd37f7fe640 1 -- 172.21.15.31:0/4027262480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3801083a0 msgr2=0x7fd38006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd37f7fe640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3801083a0 0x7fd38006e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd37f7fe640 1 -- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 msgr2=0x7fd380074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd37f7fe640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 0x7fd380074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd37f7fe640 1 -- 172.21.15.31:0/4027262480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38006ee90 con 0x7fd380108cf0 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd37f7fe640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd380108cf0 0x7fd38006e670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd378009fd0 tx=0x7fd37800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800be40 con 0x7fd380108cf0 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd378017450 con 0x7fd380108cf0 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd380073b10 con 0x7fd380108cf0 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378005280 con 0x7fd380108cf0 2024-09-21T06:13:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.800+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd380074050 con 0x7fd380108cf0 2024-09-21T06:13:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.801+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd380102e70 con 0x7fd380108cf0 2024-09-21T06:13:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.803+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd378004a40 con 0x7fd380108cf0 2024-09-21T06:13:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.804+0000 7fd367fff640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3680b6700 0x7fd3680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.804+0000 7fd37effd640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3680b6700 0x7fd3680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.804+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fd3780e9a60 con 0x7fd380108cf0 2024-09-21T06:13:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.804+0000 7fd37effd640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3680b6700 0x7fd3680b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd370031440 tx=0x7fd370009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:04.808+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd3780b43d0 con 0x7fd380108cf0 2024-09-21T06:13:05.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.091+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd38010a600 con 0x7fd380108cf0 2024-09-21T06:13:05.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.092+0000 7fd367fff640 1 -- 172.21.15.31:0/4027262480 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fd3780b9310 con 0x7fd380108cf0 2024-09-21T06:13:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:13:05.094 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:13:05.095 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:13:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.095+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3680b6700 msgr2=0x7fd3680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.095+0000 7fd385618640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3680b6700 0x7fd3680b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd370031440 tx=0x7fd370009560 comp rx=0 tx=0).stop 2024-09-21T06:13:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.095+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd380108cf0 msgr2=0x7fd38006e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.095+0000 7fd385618640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd380108cf0 0x7fd38006e670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd378009fd0 tx=0x7fd37800b690 comp rx=0 tx=0).stop 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 shutdown_connections 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd380108cf0 0x7fd38006e670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3801083a0 0x7fd38006e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd3680b6700 0x7fd3680b8c10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 --2- 172.21.15.31:0/4027262480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd380102a60 0x7fd380074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 >> 172.21.15.31:0/4027262480 conn(0x7fd380079730 msgr2=0x7fd380101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 shutdown_connections 2024-09-21T06:13:05.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:05.096+0000 7fd385618640 1 -- 172.21.15.31:0/4027262480 wait complete. 2024-09-21T06:13:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 85.0 s, 349.0 tps, lat 280.830 ms stddev 175.959 2024-09-21T06:13:05.774 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:05 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4027262480' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:05.777 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:05 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4027262480' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:05 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4027262480' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:05.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:13:06.119 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:06.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:06 smithi031 ceph-mon[31237]: pgmap v365: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-09-21T06:13:07.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:06 smithi110 ceph-mon[34550]: pgmap v365: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-09-21T06:13:07.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:06 smithi107 ceph-mon[34380]: pgmap v365: 97 pgs: 97 active+clean; 1.3 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-09-21T06:13:07.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 -- 172.21.15.31:0/2827454833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 msgr2=0x7fea641087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/2827454833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea641087b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fea58009900 tx=0x7fea5802a410 comp rx=0 tx=0).stop 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 -- 172.21.15.31:0/2827454833 shutdown_connections 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/2827454833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64108cf0 0x7fea6410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/2827454833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea641087b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/2827454833 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64102a60 0x7fea64102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 -- 172.21.15.31:0/2827454833 >> 172.21.15.31:0/2827454833 conn(0x7fea64079730 msgr2=0x7fea64079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 -- 172.21.15.31:0/2827454833 shutdown_connections 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.389+0000 7fea6a5aa640 1 -- 172.21.15.31:0/2827454833 wait complete. 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 Processor -- start 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 -- start start 2024-09-21T06:13:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64102a60 0x7fea640ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea640ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64108cf0 0x7fea641000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112960 con 0x7fea641083a0 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea641127e0 con 0x7fea64102a60 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea6a5aa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea64112ae0 con 0x7fea64108cf0 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea640ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea637fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea640ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59914/0 (socket says 172.21.15.31:59914) 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea637fe640 1 -- 172.21.15.31:0/3322666016 learned_addr learned my addr 172.21.15.31:0/3322666016 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea63fff640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64102a60 0x7fea640ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.390+0000 7fea68b47640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64108cf0 0x7fea641000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea68b47640 1 -- 172.21.15.31:0/3322666016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64102a60 msgr2=0x7fea640ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea68b47640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64102a60 0x7fea640ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea68b47640 1 -- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 msgr2=0x7fea640ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea68b47640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea640ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea68b47640 1 -- 172.21.15.31:0/3322666016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea641a8760 con 0x7fea64108cf0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea637fe640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea640ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea63fff640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64102a60 0x7fea640ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea68b47640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64108cf0 0x7fea641000f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009fd0 tx=0x7fea5c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea617fa640 1 -- 172.21.15.31:0/3322666016 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00ce20 con 0x7fea64108cf0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea641a8a50 con 0x7fea64108cf0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea617fa640 1 -- 172.21.15.31:0/3322666016 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea5c005020 con 0x7fea64108cf0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea617fa640 1 -- 172.21.15.31:0/3322666016 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c010690 con 0x7fea64108cf0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.391+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea641a8f90 con 0x7fea64108cf0 2024-09-21T06:13:07.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 Processor -- start 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea617fa640 1 -- 172.21.15.31:0/3322666016 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fea5c0054d0 con 0x7fea64108cf0 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 -- start start 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 0x55dd775e0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77218390 0x55dd775e1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd77556ea0 con 0x55dd77218390 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dd775462d0 con 0x55dd772101b0 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.394+0000 7fea70b1e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dd7754a0e0 con 0x55dd77239070 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea637fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 0x55dd775e0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 0x55dd775e0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47052/0 (socket says 172.21.15.31:47052) 2024-09-21T06:13:07.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea637fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40046/0 (socket says 172.21.15.31:40046) 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 -- 172.21.15.31:0/2133737788 learned_addr learned my addr 172.21.15.31:0/2133737788 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea637fe640 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77218390 0x55dd775e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 -- 172.21.15.31:0/2133737788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 msgr2=0x55dd775e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 -- 172.21.15.31:0/2133737788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77218390 msgr2=0x55dd775e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77218390 0x55dd775e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 -- 172.21.15.31:0/2133737788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd772a4650 con 0x55dd772101b0 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea637fe640 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77218390 0x55dd775e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea617fa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea4c0b6680 0x7fea4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea63fff640 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 0x55dd775e0be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea540114f0 tx=0x7fea540119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96827ddbc8225b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea637fe640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea4c0b6680 0x7fea4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea617fa640 1 -- 172.21.15.31:0/3322666016 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fea5c0e7d90 con 0x7fea64108cf0 2024-09-21T06:13:07.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.395+0000 7fea4a7fc640 1 -- 172.21.15.31:0/2133737788 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54004020 con 0x55dd772101b0 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea4a7fc640 1 -- 172.21.15.31:0/2133737788 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea540041c0 con 0x55dd772101b0 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea4a7fc640 1 -- 172.21.15.31:0/2133737788 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5400f840 con 0x55dd772101b0 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 -- 172.21.15.31:0/2133737788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 msgr2=0x55dd775e0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea637fe640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea4c0b6680 0x7fea4c0b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea58032fd0 tx=0x7fea58032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 0x55dd775e0be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea540114f0 tx=0x7fea540119b0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 -- 172.21.15.31:0/2133737788 shutdown_connections 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77218390 0x55dd775e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd772101b0 0x55dd775e0be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2133737788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 -- 172.21.15.31:0/2133737788 >> 172.21.15.31:0/2133737788 conn(0x55dd7722d460 msgr2=0x55dd775de590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 -- 172.21.15.31:0/2133737788 shutdown_connections 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.396+0000 7fea70b1e740 1 -- 172.21.15.31:0/2133737788 wait complete. 2024-09-21T06:13:07.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 -- start start 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 Processor -- start 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775ec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd773a7700 con 0x55dd772101b0 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dd774e1160 con 0x55dd77218390 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.397+0000 7fea70b1e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dd775eee80 con 0x55dd77239070 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea70b1e740 10 client.0 _ll_register_callbacks cb 0x55dd774fa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775ebca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59938/0 (socket says 172.21.15.31:59938) 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775ec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47068/0 (socket says 172.21.15.31:47068) 2024-09-21T06:13:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 -- 172.21.15.31:0/2892747283 learned_addr learned my addr 172.21.15.31:0/2892747283 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea68b47640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775ec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 -- 172.21.15.31:0/2892747283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 msgr2=0x55dd775ec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775ec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 -- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 msgr2=0x55dd775ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 -- 172.21.15.31:0/2892747283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd7750ec90 con 0x55dd772101b0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea637fe640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea63fff640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775ebca0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fea54009380 tx=0x7fea54009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40003d069f4d3fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea437fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea437fe640 1 -- 172.21.15.31:0/2892747283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54028020 con 0x55dd772101b0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea437fe640 1 -- 172.21.15.31:0/2892747283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fea54016040 con 0x55dd772101b0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.398+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea540041c0 con 0x55dd772101b0 2024-09-21T06:13:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea437fe640 1 -- 172.21.15.31:0/2892747283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fea54033050 con 0x55dd772101b0 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 10 client.15762 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dd775fab00 con 0x55dd772101b0 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea437fe640 1 -- 172.21.15.31:0/2892747283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea540041c0 con 0x55dd772101b0 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea437fe640 1 -- 172.21.15.31:0/2892747283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fea54004b00 con 0x55dd772101b0 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 10 client.15762 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 20 client.15762 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771f79f0 0x55dd775fdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 4 client.15762 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd77558df0 con 0x55dd771f79f0 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.399+0000 7fea637fe640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771f79f0 0x55dd775fdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.400+0000 7fea437fe640 10 client.15762 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.400+0000 7fea637fe640 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771f79f0 0x55dd775fdf70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a827f91d102ae2d server_cookie=5410a3b1d094c448 in_seq=0 out_seq=0 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.400+0000 7fea437fe640 1 -- 172.21.15.31:0/2892747283 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd77558df0 con 0x55dd771f79f0 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.400+0000 7fea437fe640 10 client.15762 handle_command_reply: tid=0 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.400+0000 7fea70b1e740 1 client.15762 shutdown 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 msgr2=0x55dd775ebca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775ebca0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fea54009380 tx=0x7fea54009d00 comp rx=0 tx=0).stop 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 shutdown_connections 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771f79f0 0x55dd775fdf70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77239070 0x55dd775ec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2892747283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775ebca0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea437fe640 0 client.15762 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 >> 172.21.15.31:0/2892747283 conn(0x55dd7722d460 msgr2=0x55dd775e3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 shutdown_connections 2024-09-21T06:13:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 1 -- 172.21.15.31:0/2892747283 wait complete. 2024-09-21T06:13:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.401+0000 7fea70b1e740 20 client.15762 trim_cache size 0 max 16384 2024-09-21T06:13:07.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 Processor -- start 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 -- start start 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd772101b0 0x55dd775edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77239070 0x55dd775e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea641af7d0 con 0x55dd77239070 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b2050 con 0x55dd77218390 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea70b1e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea641afe40 con 0x55dd772101b0 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea68b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77239070 0x55dd775e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea68b47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77239070 0x55dd775e1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59942/0 (socket says 172.21.15.31:59942) 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775e37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47084/0 (socket says 172.21.15.31:47084) 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea68b47640 1 -- 172.21.15.31:0/3292534814 learned_addr learned my addr 172.21.15.31:0/3292534814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:07.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea63fff640 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd772101b0 0x55dd775edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 -- 172.21.15.31:0/3292534814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd772101b0 msgr2=0x55dd775edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd772101b0 0x55dd775edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 -- 172.21.15.31:0/3292534814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77239070 msgr2=0x55dd775e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77239070 0x55dd775e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.415+0000 7fea637fe640 1 -- 172.21.15.31:0/3292534814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd775ea1a0 con 0x55dd77218390 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.416+0000 7fea637fe640 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775e37a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea58031440 tx=0x7fea5803f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104758d6b715c3da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.416+0000 7fea49ffb640 1 -- 172.21.15.31:0/3292534814 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58035040 con 0x55dd77218390 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.416+0000 7fea49ffb640 1 -- 172.21.15.31:0/3292534814 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea580054d0 con 0x55dd77218390 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.416+0000 7fea49ffb640 1 -- 172.21.15.31:0/3292534814 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58003850 con 0x55dd77218390 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 -- 172.21.15.31:0/3292534814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 msgr2=0x55dd775e37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775e37a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea58031440 tx=0x7fea5803f070 comp rx=0 tx=0).stop 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 -- 172.21.15.31:0/3292534814 shutdown_connections 2024-09-21T06:13:07.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd77239070 0x55dd775e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77218390 0x55dd775e37a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 --2- 172.21.15.31:0/3292534814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd772101b0 0x55dd775edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 -- 172.21.15.31:0/3292534814 >> 172.21.15.31:0/3292534814 conn(0x55dd7722d460 msgr2=0x55dd775e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 -- 172.21.15.31:0/3292534814 shutdown_connections 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 -- 172.21.15.31:0/3292534814 wait complete. 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.417+0000 7fea70b1e740 1 -- start start 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 Processor -- start 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77218390 0x55dd775f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77239070 0x55dd775f2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b0510 con 0x55dd772101b0 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b1980 con 0x55dd77239070 2024-09-21T06:13:07.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea641b0be0 con 0x55dd77218390 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea70b1e740 10 client.0 _ll_register_callbacks cb 0x7fea54005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59946/0 (socket says 172.21.15.31:59946) 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea63fff640 1 -- 172.21.15.31:0/2595877729 learned_addr learned my addr 172.21.15.31:0/2595877729 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea68b47640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77239070 0x55dd775f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.418+0000 7fea637fe640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77218390 0x55dd775f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea63fff640 1 -- 172.21.15.31:0/2595877729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77218390 msgr2=0x55dd775f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea63fff640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77218390 0x55dd775f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea63fff640 1 -- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77239070 msgr2=0x55dd775f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea63fff640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77239070 0x55dd775f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea63fff640 1 -- 172.21.15.31:0/2595877729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd775f25c0 con 0x55dd772101b0 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea68b47640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77239070 0x55dd775f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea63fff640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775f4670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fea54009bb0 tx=0x7fea5400f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216f095bfbe36e30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:07.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea437fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea437fe640 1 -- 172.21.15.31:0/2595877729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5403c060 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea437fe640 1 -- 172.21.15.31:0/2595877729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fea54004580 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea580054d0 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea437fe640 1 -- 172.21.15.31:0/2595877729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fea54028040 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea70b1e740 10 client.15768 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.419+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dd77668540 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea437fe640 1 -- 172.21.15.31:0/2595877729 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea580054d0 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea437fe640 1 -- 172.21.15.31:0/2595877729 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fea540194c0 con 0x55dd772101b0 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea70b1e740 10 client.15768 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea70b1e740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea70b1e740 20 client.15768 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771ccda0 0x55dd7766aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea70b1e740 4 client.15768 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:13:07.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd775548a0 con 0x55dd771ccda0 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.420+0000 7fea637fe640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771ccda0 0x55dd7766aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea637fe640 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771ccda0 0x55dd7766aca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea28a0d57a0f808a server_cookie=3548cfa929503bce in_seq=0 out_seq=0 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea437fe640 10 client.15768 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea437fe640 1 -- 172.21.15.31:0/2595877729 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd775548a0 con 0x55dd771ccda0 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea437fe640 10 client.15768 handle_command_reply: tid=0 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 client.15768 shutdown 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 msgr2=0x55dd775f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775f4670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fea54009bb0 tx=0x7fea5400f960 comp rx=0 tx=0).stop 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 shutdown_connections 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd771ccda0 0x55dd7766aca0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd77239070 0x55dd775f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd77218390 0x55dd775f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 --2- 172.21.15.31:0/2595877729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd772101b0 0x55dd775f4670 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 >> 172.21.15.31:0/2595877729 conn(0x55dd7722d460 msgr2=0x55dd775e16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea437fe640 0 client.15768 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:07.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 shutdown_connections 2024-09-21T06:13:07.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 1 -- 172.21.15.31:0/2595877729 wait complete. 2024-09-21T06:13:07.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.421+0000 7fea70b1e740 20 client.15768 trim_cache size 0 max 16384 2024-09-21T06:13:07.431 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea4c0b6680 msgr2=0x7fea4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea4c0b6680 0x7fea4c0b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea58032fd0 tx=0x7fea58032ed0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64108cf0 msgr2=0x7fea641000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64108cf0 0x7fea641000f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009fd0 tx=0x7fea5c00ef10 comp rx=0 tx=0).stop 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 shutdown_connections 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea64108cf0 0x7fea641000f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea4c0b6680 0x7fea4c0b8b90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea641083a0 0x7fea640ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 --2- 172.21.15.31:0/3322666016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea64102a60 0x7fea640ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 >> 172.21.15.31:0/3322666016 conn(0x7fea64079730 msgr2=0x7fea6406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:07.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 shutdown_connections 2024-09-21T06:13:07.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:07.422+0000 7fea6a5aa640 1 -- 172.21.15.31:0/3322666016 wait complete. 2024-09-21T06:13:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:08 smithi110 ceph-mon[34550]: pgmap v366: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.3 MiB/s wr, 782 op/s 2024-09-21T06:13:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:08 smithi107 ceph-mon[34380]: pgmap v366: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.3 MiB/s wr, 782 op/s 2024-09-21T06:13:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 -- 172.21.15.31:0/1593127149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d70078290 msgr2=0x7f0d70113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 --2- 172.21.15.31:0/1593127149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d70078290 0x7f0d70113f90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009900 tx=0x7f0d6c02a410 comp rx=0 tx=0).stop 2024-09-21T06:13:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 -- 172.21.15.31:0/1593127149 shutdown_connections 2024-09-21T06:13:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 --2- 172.21.15.31:0/1593127149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d701144d0 0x7f0d701168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 --2- 172.21.15.31:0/1593127149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d70078290 0x7f0d70113f90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 --2- 172.21.15.31:0/1593127149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70077940 0x7f0d70077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 -- 172.21.15.31:0/1593127149 >> 172.21.15.31:0/1593127149 conn(0x7f0d70101560 msgr2=0x7f0d701039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 -- 172.21.15.31:0/1593127149 shutdown_connections 2024-09-21T06:13:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.074+0000 7f0d7754b640 1 -- 172.21.15.31:0/1593127149 wait complete. 2024-09-21T06:13:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.075+0000 7f0d7754b640 1 Processor -- start 2024-09-21T06:13:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.075+0000 7f0d7754b640 1 -- start start 2024-09-21T06:13:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.075+0000 7f0d7754b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d70077940 0x7f0d70113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d7754b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 0x7f0d7010e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d76549640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d70077940 0x7f0d70113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d7754b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d701144d0 0x7f0d7010e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d76549640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d70077940 0x7f0d70113e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56266/0 (socket says 172.21.15.31:56266) 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 0x7f0d7010e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d7754b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70106000 con 0x7f0d701144d0 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d7754b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70105e80 con 0x7f0d70077940 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d7754b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70106180 con 0x7f0d70078290 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 0x7f0d7010e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58864/0 (socket says 172.21.15.31:58864) 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 -- 172.21.15.31:0/2971515818 learned_addr learned my addr 172.21.15.31:0/2971515818 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d76d4a640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d701144d0 0x7f0d7010e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 -- 172.21.15.31:0/2971515818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d70077940 msgr2=0x7f0d70113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d70077940 0x7f0d70113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 -- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d701144d0 msgr2=0x7f0d7010e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d701144d0 0x7f0d7010e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.076+0000 7f0d75d48640 1 -- 172.21.15.31:0/2971515818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701b5310 con 0x7f0d70078290 2024-09-21T06:13:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.077+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 0x7f0d7010e0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c031440 tx=0x7f0d6c003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.077+0000 7f0d677fe640 1 -- 172.21.15.31:0/2971515818 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c03f070 con 0x7f0d70078290 2024-09-21T06:13:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.077+0000 7f0d677fe640 1 -- 172.21.15.31:0/2971515818 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6c031b50 con 0x7f0d70078290 2024-09-21T06:13:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.077+0000 7f0d677fe640 1 -- 172.21.15.31:0/2971515818 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c035030 con 0x7f0d70078290 2024-09-21T06:13:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.077+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701b54e0 con 0x7f0d70078290 2024-09-21T06:13:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.077+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701b5890 con 0x7f0d70078290 2024-09-21T06:13:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.081+0000 7f0d7f7f8740 1 Processor -- start 2024-09-21T06:13:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.081+0000 7f0d7f7f8740 1 -- start start 2024-09-21T06:13:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.082+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 0x556330870ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.082+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x5563308710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304aa4d0 0x556330871620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5563307e56f0 con 0x5563304cb1b0 2024-09-21T06:13:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5563307d4b20 con 0x5563304a22f0 2024-09-21T06:13:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5563307d8930 con 0x5563304aa4d0 2024-09-21T06:13:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d76549640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x5563308710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d677fe640 1 -- 172.21.15.31:0/2971515818 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0d6c042070 con 0x7f0d70078290 2024-09-21T06:13:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d76d4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 0x556330870ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d76d4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 0x556330870ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58474/0 (socket says 172.21.15.31:58474) 2024-09-21T06:13:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.083+0000 7f0d76d4a640 1 -- 172.21.15.31:0/2735398401 learned_addr learned my addr 172.21.15.31:0/2735398401 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.084+0000 7f0d677fe640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0d5c0b6730 0x7f0d5c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304aa4d0 0x556330871620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d677fe640 1 -- 172.21.15.31:0/2971515818 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0d6c107e40 con 0x7f0d70078290 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d75d48640 1 -- 172.21.15.31:0/2735398401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 msgr2=0x5563308710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x5563308710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d75d48640 1 -- 172.21.15.31:0/2735398401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 msgr2=0x556330870ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 0x556330870ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.085+0000 7f0d75d48640 1 -- 172.21.15.31:0/2735398401 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633079a530 con 0x5563304aa4d0 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d75d48640 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304aa4d0 0x556330871620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c1055a0 tx=0x7f0d6c03b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cc43083b5c8c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d76d4a640 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 0x556330870ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d64ff9640 1 -- 172.21.15.31:0/2735398401 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c049070 con 0x5563304aa4d0 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d76549640 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x5563308710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d64ff9640 1 -- 172.21.15.31:0/2735398401 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6c0cd7e0 con 0x5563304aa4d0 2024-09-21T06:13:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d64ff9640 1 -- 172.21.15.31:0/2735398401 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0c9f40 con 0x5563304aa4d0 2024-09-21T06:13:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.086+0000 7f0d76d4a640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0d5c0b6730 0x7f0d5c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.087+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/2735398401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304aa4d0 msgr2=0x556330871620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.087+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304aa4d0 0x556330871620 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c1055a0 tx=0x7f0d6c03b640 comp rx=0 tx=0).stop 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/2735398401 shutdown_connections 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304aa4d0 0x556330871620 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x5563308710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/2735398401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304cb1b0 0x556330870ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/2735398401 >> 172.21.15.31:0/2735398401 conn(0x5563304bf300 msgr2=0x55633086ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/2735398401 shutdown_connections 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d76d4a640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0d5c0b6730 0x7f0d5c0b8c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d600095f0 tx=0x7f0d600046c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.089+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/2735398401 wait complete. 2024-09-21T06:13:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.090+0000 7f0d7f7f8740 1 -- start start 2024-09-21T06:13:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.091+0000 7f0d7f7f8740 1 Processor -- start 2024-09-21T06:13:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.091+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x55633087c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.091+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x55633087c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.092+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304cb1b0 0x556330888250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.092+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55633070cab0 con 0x5563304a22f0 2024-09-21T06:13:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.092+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55633077a730 con 0x5563304aa4d0 2024-09-21T06:13:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.092+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5563307d8930 con 0x5563304cb1b0 2024-09-21T06:13:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.092+0000 7f0d7f7f8740 10 client.0 _ll_register_callbacks cb 0x5563307ce6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.094+0000 7f0d75d48640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x55633087c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.094+0000 7f0d75d48640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x55633087c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56278/0 (socket says 172.21.15.31:56278) 2024-09-21T06:13:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.094+0000 7f0d75d48640 1 -- 172.21.15.31:0/3930209572 learned_addr learned my addr 172.21.15.31:0/3930209572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:09.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.095+0000 7f0d76549640 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x55633087c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.098+0000 7f0d75d48640 1 -- 172.21.15.31:0/3930209572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304cb1b0 msgr2=0x556330888250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.098+0000 7f0d75d48640 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304cb1b0 0x556330888250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.098+0000 7f0d75d48640 1 -- 172.21.15.31:0/3930209572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 msgr2=0x55633087c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.098+0000 7f0d75d48640 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x55633087c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.098+0000 7f0d75d48640 1 -- 172.21.15.31:0/3930209572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556330536790 con 0x5563304aa4d0 2024-09-21T06:13:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.106+0000 7f0d75d48640 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x55633087c850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c031f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20203de1c8fcebef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.106+0000 7f0d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.108+0000 7f0d59ffb640 1 -- 172.21.15.31:0/3930209572 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c049030 con 0x5563304aa4d0 2024-09-21T06:13:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.108+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c0cd7e0 con 0x5563304aa4d0 2024-09-21T06:13:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.108+0000 7f0d59ffb640 1 -- 172.21.15.31:0/3930209572 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0d6c0d7260 con 0x5563304aa4d0 2024-09-21T06:13:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.108+0000 7f0d59ffb640 1 -- 172.21.15.31:0/3930209572 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0d6c0ce3d0 con 0x5563304aa4d0 2024-09-21T06:13:09.114 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:13:09.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:13:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.112+0000 7f0d7f7f8740 10 client.25514 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.112+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5563305af620 con 0x5563304aa4d0 2024-09-21T06:13:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d59ffb640 1 -- 172.21.15.31:0/3930209572 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6c0cd7e0 con 0x5563304aa4d0 2024-09-21T06:13:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d59ffb640 1 -- 172.21.15.31:0/3930209572 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0d6c0efce0 con 0x5563304aa4d0 2024-09-21T06:13:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d7f7f8740 10 client.25514 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d7f7f8740 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d7f7f8740 20 client.25514 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556330489d80 0x55633088df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d7f7f8740 4 client.25514 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.114+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563307e7640 con 0x556330489d80 2024-09-21T06:13:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.115+0000 7f0d76549640 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556330489d80 0x55633088df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.121+0000 7f0d76549640 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556330489d80 0x55633088df90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c791caf94da77 server_cookie=f10750df6a83887a in_seq=0 out_seq=0 2024-09-21T06:13:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.121+0000 7f0d59ffb640 10 client.25514 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.123+0000 7f0d59ffb640 1 -- 172.21.15.31:0/3930209572 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563307e7640 con 0x556330489d80 2024-09-21T06:13:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.123+0000 7f0d59ffb640 10 client.25514 handle_command_reply: tid=0 2024-09-21T06:13:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.123+0000 7f0d7f7f8740 1 client.25514 shutdown 2024-09-21T06:13:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.131+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 msgr2=0x55633087c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.131+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x55633087c850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c031f30 comp rx=0 tx=0).stop 2024-09-21T06:13:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.133+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 shutdown_connections 2024-09-21T06:13:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.133+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556330489d80 0x55633088df90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.133+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304cb1b0 0x556330888250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.133+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x55633087c850 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.133+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/3930209572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x55633087c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.133+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 >> 172.21.15.31:0/3930209572 conn(0x5563304bf300 msgr2=0x556330871bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.134+0000 7f0d59ffb640 0 client.25514 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:09.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:08 smithi031 ceph-mon[31237]: pgmap v366: 97 pgs: 97 active+clean; 1.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.3 MiB/s wr, 782 op/s 2024-09-21T06:13:09.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.134+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 shutdown_connections 2024-09-21T06:13:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.134+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/3930209572 wait complete. 2024-09-21T06:13:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.134+0000 7f0d7f7f8740 20 client.25514 trim_cache size 0 max 16384 2024-09-21T06:13:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.169+0000 7f0d7f7f8740 1 Processor -- start 2024-09-21T06:13:09.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 -- start start 2024-09-21T06:13:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x55633087e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x556330888150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304aa4d0 0x556330888690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701bc2e0 con 0x5563304aa4d0 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701beb60 con 0x5563304a22f0 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.170+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701bc950 con 0x5563304bf300 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x55633087e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x55633087e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58900/0 (socket says 172.21.15.31:58900) 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 -- 172.21.15.31:0/568791910 learned_addr learned my addr 172.21.15.31:0/568791910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d75d48640 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x556330888150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 -- 172.21.15.31:0/568791910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 msgr2=0x556330888150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x556330888150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 -- 172.21.15.31:0/568791910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304aa4d0 msgr2=0x556330888690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304aa4d0 0x556330888690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 -- 172.21.15.31:0/568791910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563308730d0 con 0x5563304bf300 2024-09-21T06:13:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.171+0000 7f0d76549640 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x55633087e300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009fd0 tx=0x7f0d68014bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ca1983637b6987 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.172+0000 7f0d5bfff640 1 -- 172.21.15.31:0/568791910 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68014110 con 0x5563304bf300 2024-09-21T06:13:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.172+0000 7f0d5bfff640 1 -- 172.21.15.31:0/568791910 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68003e20 con 0x5563304bf300 2024-09-21T06:13:09.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.173+0000 7f0d5bfff640 1 -- 172.21.15.31:0/568791910 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68011e70 con 0x5563304bf300 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.173+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/568791910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 msgr2=0x55633087e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.173+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x55633087e300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009fd0 tx=0x7f0d68014bf0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/568791910 shutdown_connections 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304aa4d0 0x556330888690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304a22f0 0x556330888150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/568791910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x55633087e300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/568791910 >> 172.21.15.31:0/568791910 conn(0x55633088e280 msgr2=0x556330872590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/568791910 shutdown_connections 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/568791910 wait complete. 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.174+0000 7f0d7f7f8740 1 -- start start 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 Processor -- start 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x556330881c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x556330882260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x5563308ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701bd020 con 0x5563304a22f0 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701be490 con 0x5563304aa4d0 2024-09-21T06:13:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701bd6f0 con 0x5563304bf300 2024-09-21T06:13:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.175+0000 7f0d7f7f8740 10 client.0 _ll_register_callbacks cb 0x7f0d6c0095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.176+0000 7f0d76549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x556330881c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.176+0000 7f0d76549640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x556330881c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58504/0 (socket says 172.21.15.31:58504) 2024-09-21T06:13:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.176+0000 7f0d76549640 1 -- 172.21.15.31:0/484083226 learned_addr learned my addr 172.21.15.31:0/484083226 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:09.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.177+0000 7f0d76549640 1 -- 172.21.15.31:0/484083226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 msgr2=0x5563308ef910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:13:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.177+0000 7f0d76549640 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x5563308ef910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.177+0000 7f0d76549640 1 -- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 msgr2=0x556330882260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:13:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.177+0000 7f0d76549640 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x556330882260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.177+0000 7f0d76549640 1 -- 172.21.15.31:0/484083226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563308effb0 con 0x5563304a22f0 2024-09-21T06:13:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.180+0000 7f0d76549640 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x556330881c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d68012480 tx=0x7f0d680039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352279027fe3fc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:09.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.180+0000 7f0d59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.181+0000 7f0d59ffb640 1 -- 172.21.15.31:0/484083226 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68014320 con 0x5563304a22f0 2024-09-21T06:13:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.181+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d68003e20 con 0x5563304a22f0 2024-09-21T06:13:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d59ffb640 1 -- 172.21.15.31:0/484083226 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0d68011910 con 0x5563304a22f0 2024-09-21T06:13:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d59ffb640 1 -- 172.21.15.31:0/484083226 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0d680144c0 con 0x5563304a22f0 2024-09-21T06:13:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d7f7f8740 10 client.15780 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5563308f84d0 con 0x5563304a22f0 2024-09-21T06:13:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d59ffb640 1 -- 172.21.15.31:0/484083226 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68003e20 con 0x5563304a22f0 2024-09-21T06:13:09.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d59ffb640 1 -- 172.21.15.31:0/484083226 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0d68004000 con 0x5563304a22f0 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d7f7f8740 10 client.15780 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d7f7f8740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.182+0000 7f0d7f7f8740 20 client.15780 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.183+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5563304cb1b0 0x5563308fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.183+0000 7f0d7f7f8740 4 client.15780 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.183+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563307e30f0 con 0x5563304cb1b0 2024-09-21T06:13:09.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.183+0000 7f0d75d48640 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5563304cb1b0 0x5563308fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.185+0000 7f0d59ffb640 10 client.15780 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.185+0000 7f0d75d48640 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5563304cb1b0 0x5563308fac00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9602ffd2e5e8dd server_cookie=a35fc8b53d675817 in_seq=0 out_seq=0 2024-09-21T06:13:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d59ffb640 1 -- 172.21.15.31:0/484083226 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563307e30f0 con 0x5563304cb1b0 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d59ffb640 10 client.15780 handle_command_reply: tid=0 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 client.15780 shutdown 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 msgr2=0x556330881c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x556330881c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d68012480 tx=0x7f0d680039f0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 shutdown_connections 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5563304cb1b0 0x5563308fac00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5563304bf300 0x5563308ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5563304aa4d0 0x556330882260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 --2- 172.21.15.31:0/484083226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5563304a22f0 0x556330881c80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 >> 172.21.15.31:0/484083226 conn(0x55633088e280 msgr2=0x556330872310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d59ffb640 0 client.15780 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 shutdown_connections 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.469+0000 7f0d7f7f8740 1 -- 172.21.15.31:0/484083226 wait complete. 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7f7f8740 20 client.15780 trim_cache size 0 max 16384 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:13:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:13:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0d5c0b6730 msgr2=0x7f0d5c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0d5c0b6730 0x7f0d5c0b8c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d600095f0 tx=0x7f0d600046c0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 msgr2=0x7f0d7010e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 0x7f0d7010e0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c031440 tx=0x7f0d6c003b20 comp rx=0 tx=0).stop 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 shutdown_connections 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0d5c0b6730 0x7f0d5c0b8c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d701144d0 0x7f0d7010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0d70078290 0x7f0d7010e0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.470+0000 7f0d7754b640 1 --2- 172.21.15.31:0/2971515818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d70077940 0x7f0d70113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.471+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 >> 172.21.15.31:0/2971515818 conn(0x7f0d70101560 msgr2=0x7f0d701165d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.471+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 shutdown_connections 2024-09-21T06:13:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:09.471+0000 7f0d7754b640 1 -- 172.21.15.31:0/2971515818 wait complete. 2024-09-21T06:13:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 -- 172.21.15.31:0/3626604281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 msgr2=0x7f6e30107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 --2- 172.21.15.31:0/3626604281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e30107620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009890 tx=0x7f6e2402a360 comp rx=0 tx=0).stop 2024-09-21T06:13:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 -- 172.21.15.31:0/3626604281 shutdown_connections 2024-09-21T06:13:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 --2- 172.21.15.31:0/3626604281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e30107be0 0x7f6e301080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 --2- 172.21.15.31:0/3626604281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e30107620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 --2- 172.21.15.31:0/3626604281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e301027a0 0x7f6e30102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 -- 172.21.15.31:0/3626604281 >> 172.21.15.31:0/3626604281 conn(0x7f6e30079730 msgr2=0x7f6e30079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 -- 172.21.15.31:0/3626604281 shutdown_connections 2024-09-21T06:13:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.149+0000 7f6e3702d640 1 -- 172.21.15.31:0/3626604281 wait complete. 2024-09-21T06:13:10.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.150+0000 7f6e3702d640 1 Processor -- start 2024-09-21T06:13:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 -- start start 2024-09-21T06:13:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 0x7f6e30074f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e3006e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e30107be0 0x7f6e3006e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3011a040 con 0x7f6e30107be0 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30119ec0 con 0x7f6e301030f0 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e3702d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3011a1c0 con 0x7f6e301027a0 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e3006e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e34dc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 0x7f6e30074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e3006e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56318/0 (socket says 172.21.15.31:56318) 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e34dc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 0x7f6e30074f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58924/0 (socket says 172.21.15.31:58924) 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.151+0000 7f6e2ffff640 1 -- 172.21.15.31:0/1900119273 learned_addr learned my addr 172.21.15.31:0/1900119273 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e355ca640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e30107be0 0x7f6e3006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e34dc9640 1 -- 172.21.15.31:0/1900119273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 msgr2=0x7f6e3006e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e3006e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e34dc9640 1 -- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e30107be0 msgr2=0x7f6e3006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e30107be0 0x7f6e3006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e34dc9640 1 -- 172.21.15.31:0/1900119273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e30073b70 con 0x7f6e301027a0 2024-09-21T06:13:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.152+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 0x7f6e30074f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005fd0 tx=0x7f6e2000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.153+0000 7f6e2dffb640 1 -- 172.21.15.31:0/1900119273 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2000ccc0 con 0x7f6e301027a0 2024-09-21T06:13:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.153+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e30073e00 con 0x7f6e301027a0 2024-09-21T06:13:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.153+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e30074340 con 0x7f6e301027a0 2024-09-21T06:13:10.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.153+0000 7f6e2dffb640 1 -- 172.21.15.31:0/1900119273 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e2000ee70 con 0x7f6e301027a0 2024-09-21T06:13:10.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.153+0000 7f6e2dffb640 1 -- 172.21.15.31:0/1900119273 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20003e10 con 0x7f6e301027a0 2024-09-21T06:13:10.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.161+0000 7f6e2dffb640 1 -- 172.21.15.31:0/1900119273 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6e200104d0 con 0x7f6e301027a0 2024-09-21T06:13:10.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.162+0000 7f6e3d5a1740 1 Processor -- start 2024-09-21T06:13:10.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.162+0000 7f6e3d5a1740 1 -- start start 2024-09-21T06:13:10.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.163+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c5c3350 0x562d6c96a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.164+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c59a490 0x562d6c96a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.164+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.164+0000 7f6e355ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c5c3350 0x562d6c96a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.164+0000 7f6e355ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c5c3350 0x562d6c96a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58932/0 (socket says 172.21.15.31:58932) 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.164+0000 7f6e2dffb640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6e180b6850 0x7f6e180b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562d6c8e0e60 con 0x562d6c5a2670 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e2dffb640 1 -- 172.21.15.31:0/1900119273 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6e200e7750 con 0x7f6e301027a0 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e355ca640 1 -- 172.21.15.31:0/4063714295 learned_addr learned my addr 172.21.15.31:0/4063714295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562d6c8d0290 con 0x562d6c59a490 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562d6c8d40a0 con 0x562d6c5c3350 2024-09-21T06:13:10.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c59a490 0x562d6c96a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e34dc9640 1 -- 172.21.15.31:0/4063714295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c5c3350 msgr2=0x562d6c96a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c5c3350 0x562d6c96a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e34dc9640 1 -- 172.21.15.31:0/4063714295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 msgr2=0x562d6c96aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e34dc9640 1 -- 172.21.15.31:0/4063714295 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6c62e930 con 0x562d6c59a490 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.165+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:10.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.166+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c59a490 0x562d6c96a970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e200e4e80 tx=0x7f6e200acdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f189844730f559 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.166+0000 7f6e16ffd640 1 -- 172.21.15.31:0/4063714295 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200d4020 con 0x562d6c59a490 2024-09-21T06:13:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.166+0000 7f6e16ffd640 1 -- 172.21.15.31:0/4063714295 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e200b2060 con 0x562d6c59a490 2024-09-21T06:13:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.166+0000 7f6e16ffd640 1 -- 172.21.15.31:0/4063714295 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200a9440 con 0x562d6c59a490 2024-09-21T06:13:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c59a490 msgr2=0x562d6c96a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c59a490 0x562d6c96a970 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6e200e4e80 tx=0x7f6e200acdf0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e355ca640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6e180b6850 0x7f6e180b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 shutdown_connections 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c59a490 0x562d6c96a970 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/4063714295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c5c3350 0x562d6c96a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.168+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 >> 172.21.15.31:0/4063714295 conn(0x562d6c5b7740 msgr2=0x562d6c968320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.169+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 shutdown_connections 2024-09-21T06:13:10.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.169+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/4063714295 wait complete. 2024-09-21T06:13:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.170+0000 7f6e355ca640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6e180b6850 0x7f6e180b8d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009fd0 tx=0x7f6e28014040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.170+0000 7f6e3d5a1740 1 -- start start 2024-09-21T06:13:10.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.170+0000 7f6e3d5a1740 1 Processor -- start 2024-09-21T06:13:10.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.171+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c976080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.171+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c976600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.171+0000 7f6e34dc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c976080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e34dc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c976080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58940/0 (socket says 172.21.15.31:58940) 2024-09-21T06:13:10.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2490256443 learned_addr learned my addr 172.21.15.31:0/2490256443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:10.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c981f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562d6c731890 con 0x562d6c5a2670 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562d6c86ad40 con 0x562d6c5c3350 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562d6c978c10 con 0x562d6c59a490 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c976600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e355ca640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c981f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.172+0000 7f6e3d5a1740 10 client.0 _ll_register_callbacks cb 0x562d6c8cec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 msgr2=0x562d6c981f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c981f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2490256443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 msgr2=0x562d6c976600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c976600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6c982570 con 0x562d6c59a490 2024-09-21T06:13:10.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e200b2060 con 0x562d6c59a490 2024-09-21T06:13:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.173+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c976080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6e200aad20 tx=0x7f6e200aca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9903827006c56cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.174+0000 7f6e0ffff640 10 client.25549 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.174+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2490256443 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200d4040 con 0x562d6c59a490 2024-09-21T06:13:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.174+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2490256443 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6e200c8bd0 con 0x562d6c59a490 2024-09-21T06:13:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.174+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2490256443 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e200c8dd0 con 0x562d6c59a490 2024-09-21T06:13:10.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.174+0000 7f6e3d5a1740 10 client.25549 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.174+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x562d6c984d80 con 0x562d6c59a490 2024-09-21T06:13:10.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.175+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2490256443 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6e200cf0b0 con 0x562d6c59a490 2024-09-21T06:13:10.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.176+0000 7f6e3d5a1740 10 client.25549 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:10.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.176+0000 7f6e3d5a1740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:10.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.176+0000 7f6e3d5a1740 20 client.25549 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:10.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.176+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c581cd0 0x562d6c9880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.176+0000 7f6e3d5a1740 4 client.25549 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:10.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.176+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d6c8e2db0 con 0x562d6c581cd0 2024-09-21T06:13:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.178+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c581cd0 0x562d6c9880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.179+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c581cd0 0x562d6c9880e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07c52984236fdde server_cookie=eebeb6e41abb50ef in_seq=0 out_seq=0 2024-09-21T06:13:10.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.179+0000 7f6e0ffff640 10 client.25549 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:10.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.182+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2490256443 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d6c8e2db0 con 0x562d6c581cd0 2024-09-21T06:13:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.182+0000 7f6e0ffff640 10 client.25549 handle_command_reply: tid=0 2024-09-21T06:13:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.182+0000 7f6e3d5a1740 1 client.25549 shutdown 2024-09-21T06:13:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.182+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 msgr2=0x562d6c976080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.182+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c976080 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6e200aad20 tx=0x7f6e200aca10 comp rx=0 tx=0).stop 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 shutdown_connections 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c581cd0 0x562d6c9880e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c981f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c976600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2490256443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c976080 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e0ffff640 0 client.25549 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:10.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.184+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 >> 172.21.15.31:0/2490256443 conn(0x562d6c5b7740 msgr2=0x562d6c96d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.185+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 shutdown_connections 2024-09-21T06:13:10.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.185+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2490256443 wait complete. 2024-09-21T06:13:10.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.185+0000 7f6e3d5a1740 20 client.25549 trim_cache size 0 max 16384 2024-09-21T06:13:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 90.0 s, 961.0 tps, lat 106.213 ms stddev 392.209 2024-09-21T06:13:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.221+0000 7f6e3d5a1740 1 Processor -- start 2024-09-21T06:13:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.221+0000 7f6e3d5a1740 1 -- start start 2024-09-21T06:13:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c96f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c973070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00c7d0 con 0x562d6c5a2670 2024-09-21T06:13:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00f050 con 0x562d6c5c3350 2024-09-21T06:13:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00ce40 con 0x562d6c59a490 2024-09-21T06:13:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e355ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c973070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e355ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c973070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56354/0 (socket says 172.21.15.31:56354) 2024-09-21T06:13:10.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e355ca640 1 -- 172.21.15.31:0/1346885789 learned_addr learned my addr 172.21.15.31:0/1346885789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:10.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.222+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c96f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e355ca640 1 -- 172.21.15.31:0/1346885789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 msgr2=0x562d6c96f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e355ca640 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c96f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e355ca640 1 -- 172.21.15.31:0/1346885789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 msgr2=0x562d6c96fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e355ca640 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e355ca640 1 -- 172.21.15.31:0/1346885789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6c974de0 con 0x562d6c5c3350 2024-09-21T06:13:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.223+0000 7f6e355ca640 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c973070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e28012550 tx=0x7f6e28004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e668d1b5bda03d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.224+0000 7f6e167fc640 1 -- 172.21.15.31:0/1346885789 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280106e0 con 0x562d6c5c3350 2024-09-21T06:13:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.224+0000 7f6e167fc640 1 -- 172.21.15.31:0/1346885789 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e28010d30 con 0x562d6c5c3350 2024-09-21T06:13:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.224+0000 7f6e167fc640 1 -- 172.21.15.31:0/1346885789 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2801adf0 con 0x562d6c5c3350 2024-09-21T06:13:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/1346885789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 msgr2=0x562d6c973070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c973070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e28012550 tx=0x7f6e28004ab0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/1346885789 shutdown_connections 2024-09-21T06:13:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c973070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c96fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/1346885789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c96f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/1346885789 >> 172.21.15.31:0/1346885789 conn(0x562d6c5b7740 msgr2=0x562d6c971030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.225+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/1346885789 shutdown_connections 2024-09-21T06:13:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.227+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/1346885789 wait complete. 2024-09-21T06:13:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.227+0000 7f6e3d5a1740 1 -- start start 2024-09-21T06:13:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 Processor -- start 2024-09-21T06:13:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c98b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c989b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c98a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00d510 con 0x562d6c5a2670 2024-09-21T06:13:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00e980 con 0x562d6c5c3350 2024-09-21T06:13:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c00dbe0 con 0x562d6c59a490 2024-09-21T06:13:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e3d5a1740 10 client.0 _ll_register_callbacks cb 0x7f6e28009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e34dc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c98b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e34dc9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c98b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58964/0 (socket says 172.21.15.31:58964) 2024-09-21T06:13:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e355ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c98a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2626751982 learned_addr learned my addr 172.21.15.31:0/2626751982 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.228+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c989b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 msgr2=0x562d6c98a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c98a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2626751982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 msgr2=0x562d6c989b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c989b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e34dc9640 1 -- 172.21.15.31:0/2626751982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6c98a590 con 0x562d6c59a490 2024-09-21T06:13:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c989b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:10.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e34dc9640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c98b630 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ad230 tx=0x7f6e200aa230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7971e986656f1932 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:10.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.229+0000 7f6e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:10.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.230+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2626751982 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200c2060 con 0x562d6c59a490 2024-09-21T06:13:10.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.230+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2626751982 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6e200b3400 con 0x562d6c59a490 2024-09-21T06:13:10.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.231+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e28010d30 con 0x562d6c59a490 2024-09-21T06:13:10.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.231+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2626751982 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e200d4040 con 0x562d6c59a490 2024-09-21T06:13:10.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.231+0000 7f6e3d5a1740 10 client.25555 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:10.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.231+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562d6c9f3520 con 0x562d6c59a490 2024-09-21T06:13:10.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2626751982 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e28010d30 con 0x562d6c59a490 2024-09-21T06:13:10.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2626751982 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6e200c32e0 con 0x562d6c59a490 2024-09-21T06:13:10.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e3d5a1740 10 client.25555 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:10.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e3d5a1740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:10.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e3d5a1740 20 client.25555 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:10.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c557080 0x562d6c9f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:10.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e3d5a1740 4 client.25555 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:13:10.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.232+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d6c8de860 con 0x562d6c557080 2024-09-21T06:13:10.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.233+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c557080 0x562d6c9f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:10.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.233+0000 7f6e0ffff640 10 client.25555 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:10.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.235+0000 7f6e2ffff640 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c557080 0x562d6c9f5a60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7cacafe6685cd3 server_cookie=47de0e29e770e675 in_seq=0 out_seq=0 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.236+0000 7f6e0ffff640 1 -- 172.21.15.31:0/2626751982 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d6c8de860 con 0x562d6c557080 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.236+0000 7f6e0ffff640 10 client.25555 handle_command_reply: tid=0 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.237+0000 7f6e3d5a1740 1 client.25555 shutdown 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 msgr2=0x562d6c98b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c98b630 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e200ad230 tx=0x7f6e200aa230 comp rx=0 tx=0).stop 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 shutdown_connections 2024-09-21T06:13:10.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562d6c557080 0x562d6c9f5a60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6c5c3350 0x562d6c98a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562d6c5a2670 0x562d6c989b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 --2- 172.21.15.31:0/2626751982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562d6c59a490 0x562d6c98b630 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 >> 172.21.15.31:0/2626751982 conn(0x562d6c5b7740 msgr2=0x562d6c96b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e0ffff640 0 client.25555 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.238+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 shutdown_connections 2024-09-21T06:13:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.239+0000 7f6e3d5a1740 1 -- 172.21.15.31:0/2626751982 wait complete. 2024-09-21T06:13:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.239+0000 7f6e3d5a1740 20 client.25555 trim_cache size 0 max 16384 2024-09-21T06:13:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:13:10.246 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:13:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "16b72551-9797-4fb8-a10e-11da021b88cf", 2024-09-21T06:13:10.247 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:13:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:13:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.240+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6e180b6850 msgr2=0x7f6e180b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.240+0000 7f6e3702d640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6e180b6850 0x7f6e180b8d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009fd0 tx=0x7f6e28014040 comp rx=0 tx=0).stop 2024-09-21T06:13:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.240+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 msgr2=0x7f6e30074f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.240+0000 7f6e3702d640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 0x7f6e30074f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005fd0 tx=0x7f6e2000ec20 comp rx=0 tx=0).stop 2024-09-21T06:13:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 shutdown_connections 2024-09-21T06:13:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6e180b6850 0x7f6e180b8d60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e30107be0 0x7f6e3006e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301030f0 0x7f6e3006e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 --2- 172.21.15.31:0/1900119273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6e301027a0 0x7f6e30074f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:10.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 >> 172.21.15.31:0/1900119273 conn(0x7f6e30079730 msgr2=0x7f6e301055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:10.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 shutdown_connections 2024-09-21T06:13:10.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:10.241+0000 7f6e3702d640 1 -- 172.21.15.31:0/1900119273 wait complete. 2024-09-21T06:13:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:10 smithi031 ceph-mon[31237]: pgmap v367: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 809 op/s 2024-09-21T06:13:10.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:13:11.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:10 smithi110 ceph-mon[34550]: pgmap v367: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 809 op/s 2024-09-21T06:13:11.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:10 smithi107 ceph-mon[34380]: pgmap v367: 97 pgs: 97 active+clean; 1.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 13 MiB/s wr, 809 op/s 2024-09-21T06:13:11.471 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:11 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:13:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:11 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:11.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:11 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:13:11.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:11 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:12.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:11 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:13:12.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:11 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:12.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:11 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:13:12.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:11 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:11 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:13:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:11 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:11 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:13:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:11 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:12.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.094+0000 7f418c919640 1 -- 172.21.15.31:0/1015132923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184030d70 con 0x7f41881083a0 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 -- 172.21.15.31:0/1015132923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41881083a0 msgr2=0x7f41881087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1015132923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41881083a0 0x7f41881087b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4184009be0 tx=0x7f418402a6d0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 -- 172.21.15.31:0/1015132923 shutdown_connections 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1015132923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f418806ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1015132923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41881083a0 0x7f41881087b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1015132923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4188102a60 0x7f4188102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 -- 172.21.15.31:0/1015132923 >> 172.21.15.31:0/1015132923 conn(0x7f4188079730 msgr2=0x7f4188079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 -- 172.21.15.31:0/1015132923 shutdown_connections 2024-09-21T06:13:12.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.095+0000 7f418fb7f640 1 -- 172.21.15.31:0/1015132923 wait complete. 2024-09-21T06:13:12.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.096+0000 7f418fb7f640 1 Processor -- start 2024-09-21T06:13:12.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.096+0000 7f418fb7f640 1 -- start start 2024-09-21T06:13:12.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418fb7f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4188102a60 0x7f4188112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418fb7f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41881083a0 0x7f4188113050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418fb7f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f4188113590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418fb7f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4188115cc0 con 0x7f4188102a60 2024-09-21T06:13:12.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418fb7f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4188115b40 con 0x7f41881083a0 2024-09-21T06:13:12.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418fb7f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4188115e40 con 0x7f4188108cf0 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418e11c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f4188113590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418e11c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f4188113590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58984/0 (socket says 172.21.15.31:58984) 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.097+0000 7f418e11c640 1 -- 172.21.15.31:0/1237324783 learned_addr learned my addr 172.21.15.31:0/1237324783 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.098+0000 7f418e11c640 1 -- 172.21.15.31:0/1237324783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41881083a0 msgr2=0x7f4188113050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.098+0000 7f418e11c640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41881083a0 0x7f4188113050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.098+0000 7f418e11c640 1 -- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4188102a60 msgr2=0x7f4188112b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:13:12.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.098+0000 7f418e11c640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4188102a60 0x7f4188112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.098+0000 7f418e11c640 1 -- 172.21.15.31:0/1237324783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41881acbb0 con 0x7f4188108cf0 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.098+0000 7f418e11c640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f4188113590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f418000e970 tx=0x7f418000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.099+0000 7f417effd640 1 -- 172.21.15.31:0/1237324783 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418000ccc0 con 0x7f4188108cf0 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.099+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41881acea0 con 0x7f4188108cf0 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.099+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41881ad3b0 con 0x7f4188108cf0 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.099+0000 7f417effd640 1 -- 172.21.15.31:0/1237324783 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800054d0 con 0x7f4188108cf0 2024-09-21T06:13:12.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.099+0000 7f417effd640 1 -- 172.21.15.31:0/1237324783 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418000ccc0 con 0x7f4188108cf0 2024-09-21T06:13:12.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.103+0000 7f41960f3740 1 Processor -- start 2024-09-21T06:13:12.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.104+0000 7f417effd640 1 -- 172.21.15.31:0/1237324783 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4180004c30 con 0x7f4188108cf0 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 -- start start 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f417effd640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f41740b6a80 0x7f41740b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6e0070 0x55590aa87650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f417effd640 1 -- 172.21.15.31:0/1237324783 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f418000cac0 con 0x7f4188108cf0 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6b71b0 0x55590aa87b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aa880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55590a9fde50 con 0x55590a6b71b0 2024-09-21T06:13:12.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55590a9ed280 con 0x55590a6e0070 2024-09-21T06:13:12.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.106+0000 7f41960f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55590a9f1090 con 0x55590a6bf390 2024-09-21T06:13:12.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418e11c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aa880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418e11c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aa880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58996/0 (socket says 172.21.15.31:58996) 2024-09-21T06:13:12.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418e11c640 1 -- 172.21.15.31:0/3426038522 learned_addr learned my addr 172.21.15.31:0/3426038522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:12.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d91b640 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6e0070 0x55590aa87650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d91b640 1 -- 172.21.15.31:0/3426038522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 msgr2=0x55590aa880d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:12.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d91b640 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aa880d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d11a640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f41740b6a80 0x7f41740b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d91b640 1 -- 172.21.15.31:0/3426038522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6b71b0 msgr2=0x55590aa87b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d91b640 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6b71b0 0x55590aa87b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418d91b640 1 -- 172.21.15.31:0/3426038522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590a74b650 con 0x55590a6e0070 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.107+0000 7f418e11c640 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aa880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.108+0000 7f418d91b640 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6e0070 0x55590aa87650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4178009b10 tx=0x7f417800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5caa367de317a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.108+0000 7f4173fff640 1 -- 172.21.15.31:0/3426038522 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417800ed50 con 0x55590a6e0070 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.108+0000 7f4173fff640 1 -- 172.21.15.31:0/3426038522 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4178005020 con 0x55590a6e0070 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.108+0000 7f418d11a640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f41740b6a80 0x7f41740b8f90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f41840099a0 tx=0x7f4184000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 -- 172.21.15.31:0/3426038522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6e0070 msgr2=0x55590aa87650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6e0070 0x55590aa87650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4178009b10 tx=0x7f417800e350 comp rx=0 tx=0).stop 2024-09-21T06:13:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f4173fff640 1 -- 172.21.15.31:0/3426038522 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178004a90 con 0x55590a6e0070 2024-09-21T06:13:12.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 -- 172.21.15.31:0/3426038522 shutdown_connections 2024-09-21T06:13:12.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aa880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6b71b0 0x55590aa87b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 --2- 172.21.15.31:0/3426038522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6e0070 0x55590aa87650 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 -- 172.21.15.31:0/3426038522 >> 172.21.15.31:0/3426038522 conn(0x55590a6d4460 msgr2=0x55590aa85540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 -- 172.21.15.31:0/3426038522 shutdown_connections 2024-09-21T06:13:12.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.110+0000 7f41960f3740 1 -- 172.21.15.31:0/3426038522 wait complete. 2024-09-21T06:13:12.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.111+0000 7f41960f3740 1 -- start start 2024-09-21T06:13:12.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.111+0000 7f41960f3740 1 Processor -- start 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.111+0000 7f41960f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa92bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.111+0000 7f41960f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa93100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.111+0000 7f41960f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa93640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f41960f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55590a84e700 con 0x55590a6e0070 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f41960f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55590a988110 con 0x55590a6bf390 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f41960f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55590aaa6ee0 con 0x55590a6b71b0 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d11a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa93100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d91b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa92bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d11a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa93100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56400/0 (socket says 172.21.15.31:56400) 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d11a640 1 -- 172.21.15.31:0/1577914091 learned_addr learned my addr 172.21.15.31:0/1577914091 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418e11c640 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa93640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f41960f3740 10 client.0 _ll_register_callbacks cb 0x55590a9a1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d91b640 1 -- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 msgr2=0x55590aa93100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d91b640 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa93100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d91b640 1 -- 172.21.15.31:0/1577914091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 msgr2=0x55590aa93640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d91b640 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa93640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.112+0000 7f418d91b640 1 -- 172.21.15.31:0/1577914091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590a9b5c40 con 0x55590a6b71b0 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.113+0000 7f418d91b640 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa92bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4178000c00 tx=0x7f4178005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec34bee1452d5e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.113+0000 7f41717fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.113+0000 7f41717fa640 1 -- 172.21.15.31:0/1577914091 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178029060 con 0x55590a6b71b0 2024-09-21T06:13:12.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.113+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4178005020 con 0x55590a6b71b0 2024-09-21T06:13:12.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.113+0000 7f41717fa640 1 -- 172.21.15.31:0/1577914091 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4178020040 con 0x55590a6b71b0 2024-09-21T06:13:12.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.114+0000 7f41717fa640 1 -- 172.21.15.31:0/1577914091 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4178029060 con 0x55590a6b71b0 2024-09-21T06:13:12.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.114+0000 7f41960f3740 10 client.25567 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:12.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.114+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55590aab2ae0 con 0x55590a6b71b0 2024-09-21T06:13:12.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.114+0000 7f41717fa640 1 -- 172.21.15.31:0/1577914091 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4178005020 con 0x55590a6b71b0 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41717fa640 1 -- 172.21.15.31:0/1577914091 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f41780249e0 con 0x55590a6b71b0 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41960f3740 10 client.25567 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41960f3740 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41960f3740 20 client.25567 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41960f3740 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a69e9f0 0x55590aab6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41960f3740 4 client.25567 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:12.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.115+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55590a9ffda0 con 0x55590a69e9f0 2024-09-21T06:13:12.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.118+0000 7f418d11a640 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a69e9f0 0x55590aab6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.127+0000 7f418d11a640 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a69e9f0 0x55590aab6330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6679f98a0a1e376 server_cookie=74813e6e50505731 in_seq=0 out_seq=0 2024-09-21T06:13:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.127+0000 7f41717fa640 10 client.25567 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.128+0000 7f41717fa640 1 -- 172.21.15.31:0/1577914091 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55590a9ffda0 con 0x55590a69e9f0 2024-09-21T06:13:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.128+0000 7f41717fa640 10 client.25567 handle_command_reply: tid=0 2024-09-21T06:13:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.128+0000 7f41960f3740 1 client.25567 shutdown 2024-09-21T06:13:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.128+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 msgr2=0x55590aa92bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.128+0000 7f41960f3740 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa92bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4178000c00 tx=0x7f4178005400 comp rx=0 tx=0).stop 2024-09-21T06:13:12.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 shutdown_connections 2024-09-21T06:13:12.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a69e9f0 0x55590aab6330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa93100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 --2- 172.21.15.31:0/1577914091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa92bc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 >> 172.21.15.31:0/1577914091 conn(0x55590a6d4460 msgr2=0x55590aa8a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41717fa640 0 client.25567 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.129+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 shutdown_connections 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.131+0000 7f41960f3740 1 -- 172.21.15.31:0/1577914091 wait complete. 2024-09-21T06:13:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.132+0000 7f41960f3740 20 client.25567 trim_cache size 0 max 16384 2024-09-21T06:13:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 Processor -- start 2024-09-21T06:13:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 -- start start 2024-09-21T06:13:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa94aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa88d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa84b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f416c00c9d0 con 0x55590a6e0070 2024-09-21T06:13:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f416c00f2b0 con 0x55590a6bf390 2024-09-21T06:13:12.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.175+0000 7f41960f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f416c00d0a0 con 0x55590a6b71b0 2024-09-21T06:13:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.176+0000 7f418e11c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa84b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.176+0000 7f418e11c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa84b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58610/0 (socket says 172.21.15.31:58610) 2024-09-21T06:13:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.176+0000 7f418e11c640 1 -- 172.21.15.31:0/1700931924 learned_addr learned my addr 172.21.15.31:0/1700931924 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.177+0000 7f418e11c640 1 -- 172.21.15.31:0/1700931924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 msgr2=0x55590aa94aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:13:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.177+0000 7f418e11c640 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa94aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.177+0000 7f418e11c640 1 -- 172.21.15.31:0/1700931924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 msgr2=0x55590aa88d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:13:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.177+0000 7f418e11c640 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa88d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.177+0000 7f418e11c640 1 -- 172.21.15.31:0/1700931924 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55590aaa6f50 con 0x55590a6e0070 2024-09-21T06:13:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.177+0000 7f418e11c640 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa84b70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f41800e4b90 tx=0x7f41800aca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ad2fd44f8aed5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.178+0000 7f41737fe640 1 -- 172.21.15.31:0/1700931924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800b1cc0 con 0x55590a6e0070 2024-09-21T06:13:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.178+0000 7f41737fe640 1 -- 172.21.15.31:0/1700931924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800b6c00 con 0x55590a6e0070 2024-09-21T06:13:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.178+0000 7f41737fe640 1 -- 172.21.15.31:0/1700931924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800d0dc0 con 0x55590a6e0070 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 -- 172.21.15.31:0/1700931924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 msgr2=0x55590aa84b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa84b70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f41800e4b90 tx=0x7f41800aca10 comp rx=0 tx=0).stop 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 -- 172.21.15.31:0/1700931924 shutdown_connections 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590aa84b70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6bf390 0x55590aa88d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 --2- 172.21.15.31:0/1700931924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6b71b0 0x55590aa94aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.180+0000 7f41960f3740 1 -- 172.21.15.31:0/1700931924 >> 172.21.15.31:0/1700931924 conn(0x55590a6d4460 msgr2=0x55590aa88680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.181+0000 7f41960f3740 1 -- 172.21.15.31:0/1700931924 shutdown_connections 2024-09-21T06:13:12.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.181+0000 7f41960f3740 1 -- 172.21.15.31:0/1700931924 wait complete. 2024-09-21T06:13:12.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.182+0000 7f41960f3740 1 -- start start 2024-09-21T06:13:12.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.182+0000 7f41960f3740 1 Processor -- start 2024-09-21T06:13:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6b71b0 0x55590aaa7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aaa7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590ab1ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f416c00d770 con 0x55590a6e0070 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f416c00ebe0 con 0x55590a6b71b0 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f416c00de40 con 0x55590a6bf390 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f41960f3740 10 client.0 _ll_register_callbacks cb 0x7f4178009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f418d91b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6b71b0 0x55590aaa7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f418e11c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590ab1ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f418d11a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aaa7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f418e11c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590ab1ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58624/0 (socket says 172.21.15.31:58624) 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f418d11a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aaa7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59020/0 (socket says 172.21.15.31:59020) 2024-09-21T06:13:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.183+0000 7f418e11c640 1 -- 172.21.15.31:0/2526730368 learned_addr learned my addr 172.21.15.31:0/2526730368 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.184+0000 7f418e11c640 1 -- 172.21.15.31:0/2526730368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 msgr2=0x55590aaa7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.184+0000 7f418e11c640 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aaa7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.184+0000 7f418e11c640 1 -- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6b71b0 msgr2=0x55590aaa7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.184+0000 7f418e11c640 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6b71b0 0x55590aaa7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.184+0000 7f418e11c640 1 -- 172.21.15.31:0/2526730368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55590ab1f220 con 0x55590a6e0070 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.184+0000 7f418e11c640 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590ab1ece0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f41800e50e0 tx=0x7f4180009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8163662ac405980c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41717fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41717fa640 1 -- 172.21.15.31:0/2526730368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41800b1cc0 con 0x55590a6e0070 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41800b6c00 con 0x55590a6e0070 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41717fa640 1 -- 172.21.15.31:0/2526730368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f41800ac440 con 0x55590a6e0070 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41717fa640 1 -- 172.21.15.31:0/2526730368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f41800ac640 con 0x55590a6e0070 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41960f3740 10 client.15810 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.185+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55590ab20350 con 0x55590a6e0070 2024-09-21T06:13:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41717fa640 1 -- 172.21.15.31:0/2526730368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41800b6c00 con 0x55590a6e0070 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41717fa640 1 -- 172.21.15.31:0/2526730368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f41800abeb0 con 0x55590a6e0070 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41960f3740 10 client.15810 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41960f3740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41960f3740 20 client.15810 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41960f3740 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a673da0 0x55590ab229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41960f3740 4 client.15810 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.186+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55590a9fb850 con 0x55590a673da0 2024-09-21T06:13:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.187+0000 7f418d91b640 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a673da0 0x55590ab229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.187+0000 7f418d91b640 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a673da0 0x55590ab229c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a80468a4d3bfcb server_cookie=3250f6b779a7ee74 in_seq=0 out_seq=0 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.187+0000 7f41717fa640 10 client.15810 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.188+0000 7f41717fa640 1 -- 172.21.15.31:0/2526730368 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55590a9fb850 con 0x55590a673da0 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.188+0000 7f41717fa640 10 client.15810 handle_command_reply: tid=0 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.188+0000 7f41960f3740 1 client.15810 shutdown 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.188+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 msgr2=0x55590ab1ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.188+0000 7f41960f3740 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590ab1ece0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f41800e50e0 tx=0x7f4180009ef0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 shutdown_connections 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55590a673da0 0x55590ab229c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55590a6e0070 0x55590ab1ece0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55590a6bf390 0x55590aaa7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 --2- 172.21.15.31:0/2526730368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55590a6b71b0 0x55590aaa7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 >> 172.21.15.31:0/2526730368 conn(0x55590a6d4460 msgr2=0x55590aa88680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41717fa640 0 client.15810 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:12.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 shutdown_connections 2024-09-21T06:13:12.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.189+0000 7f41960f3740 1 -- 172.21.15.31:0/2526730368 wait complete. 2024-09-21T06:13:12.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.190+0000 7f41960f3740 20 client.15810 trim_cache size 0 max 16384 2024-09-21T06:13:12.193 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:13:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:13:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:13:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:13:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:13:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:13:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:13:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:13:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:13:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:13:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:13:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:13:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:13:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:12.202 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:13:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.191+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f41740b6a80 msgr2=0x7f41740b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f41740b6a80 0x7f41740b8f90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f41840099a0 tx=0x7f4184000f40 comp rx=0 tx=0).stop 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 msgr2=0x7f4188113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f4188113590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f418000e970 tx=0x7f418000ee30 comp rx=0 tx=0).stop 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 shutdown_connections 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4188108cf0 0x7f4188113590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41881083a0 0x7f4188113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f41740b6a80 0x7f41740b8f90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 --2- 172.21.15.31:0/1237324783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4188102a60 0x7f4188112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.192+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 >> 172.21.15.31:0/1237324783 conn(0x7f4188079730 msgr2=0x7f4188101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.193+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 shutdown_connections 2024-09-21T06:13:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:12.193+0000 7f418fb7f640 1 -- 172.21.15.31:0/1237324783 wait complete. 2024-09-21T06:13:12.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:13:12.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:13:13.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:12 smithi110 ceph-mon[34550]: pgmap v368: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 1.41k op/s 2024-09-21T06:13:13.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:12 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:13:13.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:12 smithi107 ceph-mon[34380]: pgmap v368: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 1.41k op/s 2024-09-21T06:13:13.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:12 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:13:13.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:12 smithi031 ceph-mon[31237]: pgmap v368: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 1.41k op/s 2024-09-21T06:13:13.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:12 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:13:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 -- 172.21.15.31:0/3546576132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 msgr2=0x7f33dc1080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 --2- 172.21.15.31:0/3546576132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 0x7f33dc1080f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009ca0 tx=0x7f33d402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 -- 172.21.15.31:0/3546576132 shutdown_connections 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 --2- 172.21.15.31:0/3546576132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 0x7f33dc1080f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 --2- 172.21.15.31:0/3546576132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 0x7f33dc1076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 --2- 172.21.15.31:0/3546576132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 0x7f33dc102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 -- 172.21.15.31:0/3546576132 >> 172.21.15.31:0/3546576132 conn(0x7f33dc0797b0 msgr2=0x7f33dc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:13.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 -- 172.21.15.31:0/3546576132 shutdown_connections 2024-09-21T06:13:13.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.916+0000 7f33e4303640 1 -- 172.21.15.31:0/3546576132 wait complete. 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.917+0000 7f33e4303640 1 Processor -- start 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.917+0000 7f33e4303640 1 -- start start 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.917+0000 7f33e4303640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 0x7f33dc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.917+0000 7f33e4303640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 0x7f33dc1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e4303640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 0x7f33dc1a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e4303640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc11a100 con 0x7f33dc107c30 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e4303640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc119f80 con 0x7f33dc102820 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e4303640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc11a280 con 0x7f33dc103170 2024-09-21T06:13:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e209f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 0x7f33dc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e209f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 0x7f33dc1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56442/0 (socket says 172.21.15.31:56442) 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e209f640 1 -- 172.21.15.31:0/1904073695 learned_addr learned my addr 172.21.15.31:0/1904073695 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e189e640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 0x7f33dc1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e28a0640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 0x7f33dc1a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e189e640 1 -- 172.21.15.31:0/1904073695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 msgr2=0x7f33dc1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e189e640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 0x7f33dc1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e189e640 1 -- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 msgr2=0x7f33dc1a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e189e640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 0x7f33dc1a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.918+0000 7f33e189e640 1 -- 172.21.15.31:0/1904073695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc1a69c0 con 0x7f33dc103170 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.919+0000 7f33e189e640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 0x7f33dc1a0ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33dc06ad30 tx=0x7f33cc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.919+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc00cc40 con 0x7f33dc103170 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.919+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33cc00cde0 con 0x7f33dc103170 2024-09-21T06:13:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.919+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0187f0 con 0x7f33dc103170 2024-09-21T06:13:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.919+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33dc1a6cb0 con 0x7f33dc103170 2024-09-21T06:13:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.919+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33dc1ad410 con 0x7f33dc103170 2024-09-21T06:13:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.921+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c00052c0 con 0x7f33dc103170 2024-09-21T06:13:13.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.924+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f33cc018990 con 0x7f33dc103170 2024-09-21T06:13:13.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.925+0000 7f33d37fe640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33c80b6780 0x7f33c80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:13.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.926+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f33cc0e78a0 con 0x7f33dc103170 2024-09-21T06:13:13.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.926+0000 7f33e209f640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33c80b6780 0x7f33c80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:13.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.927+0000 7f33e209f640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33c80b6780 0x7f33c80b8c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009c20 tx=0x7f33d8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:13.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:13.929+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33cc0addd0 con 0x7f33dc103170 2024-09-21T06:13:14.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.330+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c0005880 con 0x7f33dc103170 2024-09-21T06:13:14.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.331+0000 7f33d37fe640 1 -- 172.21.15.31:0/1904073695 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f33cc0b21b0 con 0x7f33dc103170 2024-09-21T06:13:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:13:14.334 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:13:14.334 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:13:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33c80b6780 msgr2=0x7f33c80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33c80b6780 0x7f33c80b8c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009c20 tx=0x7f33d8005f90 comp rx=0 tx=0).stop 2024-09-21T06:13:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 msgr2=0x7f33dc1a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 0x7f33dc1a0ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33dc06ad30 tx=0x7f33cc00ee50 comp rx=0 tx=0).stop 2024-09-21T06:13:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 shutdown_connections 2024-09-21T06:13:14.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33c80b6780 0x7f33c80b8c90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33dc107c30 0x7f33dc1a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33dc103170 0x7f33dc1a0ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 --2- 172.21.15.31:0/1904073695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33dc102820 0x7f33dc1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:14.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 >> 172.21.15.31:0/1904073695 conn(0x7f33dc0797b0 msgr2=0x7f33dc105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:14.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 shutdown_connections 2024-09-21T06:13:14.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:14.335+0000 7f33e4303640 1 -- 172.21.15.31:0/1904073695 wait complete. 2024-09-21T06:13:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 95.0 s, 4497.0 tps, lat 22.212 ms stddev 29.417 2024-09-21T06:13:15.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:13:15.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:15 smithi031 ceph-mon[31237]: pgmap v369: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.08k op/s 2024-09-21T06:13:15.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:15 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1904073695' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:15.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:15 smithi110 ceph-mon[34550]: pgmap v369: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.08k op/s 2024-09-21T06:13:15.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:15 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1904073695' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:15.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:15 smithi107 ceph-mon[34380]: pgmap v369: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.08k op/s 2024-09-21T06:13:15.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:15 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1904073695' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:16.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 -- 172.21.15.31:0/4232943754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 msgr2=0x7fa610103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 --2- 172.21.15.31:0/4232943754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa610103fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa600009860 tx=0x7fa60002a340 comp rx=0 tx=0).stop 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 -- 172.21.15.31:0/4232943754 shutdown_connections 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 --2- 172.21.15.31:0/4232943754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa610109e00 0x7fa610110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 --2- 172.21.15.31:0/4232943754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6101094b0 0x7fa6101098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 --2- 172.21.15.31:0/4232943754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa610103fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.251+0000 7fa6163c1640 1 -- 172.21.15.31:0/4232943754 >> 172.21.15.31:0/4232943754 conn(0x7fa61007b910 msgr2=0x7fa61007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.252+0000 7fa6163c1640 1 -- 172.21.15.31:0/4232943754 shutdown_connections 2024-09-21T06:13:16.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.253+0000 7fa6163c1640 1 -- 172.21.15.31:0/4232943754 wait complete. 2024-09-21T06:13:16.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.255+0000 7fa6163c1640 1 Processor -- start 2024-09-21T06:13:16.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.255+0000 7fa6163c1640 1 -- start start 2024-09-21T06:13:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa6163c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa61019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa6163c1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6101094b0 0x7fa61019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa6163c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa610109e00 0x7fa61019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa6163c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa610113ac0 con 0x7fa610109e00 2024-09-21T06:13:16.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa6163c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa610113940 con 0x7fa610103bc0 2024-09-21T06:13:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa6163c1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa610113c40 con 0x7fa6101094b0 2024-09-21T06:13:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa61019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa61019a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56454/0 (socket says 172.21.15.31:56454) 2024-09-21T06:13:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa60ffff640 1 -- 172.21.15.31:0/1686604064 learned_addr learned my addr 172.21.15.31:0/1686604064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6101094b0 0x7fa61019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.256+0000 7fa61495e640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa610109e00 0x7fa61019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.257+0000 7fa60ffff640 1 -- 172.21.15.31:0/1686604064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6101094b0 msgr2=0x7fa61019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.257+0000 7fa60ffff640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6101094b0 0x7fa61019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.257+0000 7fa60ffff640 1 -- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa610109e00 msgr2=0x7fa61019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.257+0000 7fa60ffff640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa610109e00 0x7fa61019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.257+0000 7fa60ffff640 1 -- 172.21.15.31:0/1686604064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101a01e0 con 0x7fa610103bc0 2024-09-21T06:13:16.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.257+0000 7fa60ffff640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa61019a270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa60002a930 tx=0x7fa600005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.258+0000 7fa60d7fa640 1 -- 172.21.15.31:0/1686604064 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600037040 con 0x7fa610103bc0 2024-09-21T06:13:16.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.258+0000 7fa60d7fa640 1 -- 172.21.15.31:0/1686604064 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa600041070 con 0x7fa610103bc0 2024-09-21T06:13:16.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.259+0000 7fa60d7fa640 1 -- 172.21.15.31:0/1686604064 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60003d8d0 con 0x7fa610103bc0 2024-09-21T06:13:16.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.259+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101a0470 con 0x7fa610103bc0 2024-09-21T06:13:16.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.259+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101a08d0 con 0x7fa610103bc0 2024-09-21T06:13:16.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.264+0000 7fa61c935740 1 Processor -- start 2024-09-21T06:13:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.264+0000 7fa60d7fa640 1 -- 172.21.15.31:0/1686604064 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa600030450 con 0x7fa610103bc0 2024-09-21T06:13:16.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 -- start start 2024-09-21T06:13:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d03390 0x561fa90cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561fa9041ea0 con 0x561fa8d24070 2024-09-21T06:13:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561fa90312d0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.265+0000 7fa61c935740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561fa90350e0 con 0x561fa8d03390 2024-09-21T06:13:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa61495e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa61495e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58660/0 (socket says 172.21.15.31:58660) 2024-09-21T06:13:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa61495e640 1 -- 172.21.15.31:0/672187118 learned_addr learned my addr 172.21.15.31:0/672187118 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d03390 0x561fa90cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa60d7fa640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5f80b6780 0x7fa5f80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa60d7fa640 1 -- 172.21.15.31:0/1686604064 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa60010b520 con 0x7fa610103bc0 2024-09-21T06:13:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa60f7fe640 1 -- 172.21.15.31:0/672187118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 msgr2=0x561fa90cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.266+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa60f7fe640 1 -- 172.21.15.31:0/672187118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 msgr2=0x561fa90cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa60f7fe640 1 -- 172.21.15.31:0/672187118 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa8d8f650 con 0x561fa8d03390 2024-09-21T06:13:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d03390 0x561fa90cc120 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa61006bb30 tx=0x7fa604011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e2d0294c16272c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa61495e640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5f80b6780 0x7fa5f80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa5f67fc640 1 -- 172.21.15.31:0/672187118 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60401e070 con 0x561fa8d03390 2024-09-21T06:13:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.267+0000 7fa5f67fc640 1 -- 172.21.15.31:0/672187118 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa604009d70 con 0x561fa8d03390 2024-09-21T06:13:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.268+0000 7fa5f67fc640 1 -- 172.21.15.31:0/672187118 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604018710 con 0x561fa8d03390 2024-09-21T06:13:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 -- 172.21.15.31:0/672187118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d03390 msgr2=0x561fa90cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d03390 0x561fa90cc120 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa61006bb30 tx=0x7fa604011810 comp rx=0 tx=0).stop 2024-09-21T06:13:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61495e640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5f80b6780 0x7fa5f80b8c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa608009fd0 tx=0x7fa608004ab0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 -- 172.21.15.31:0/672187118 shutdown_connections 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d03390 0x561fa90cc120 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 --2- 172.21.15.31:0/672187118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 -- 172.21.15.31:0/672187118 >> 172.21.15.31:0/672187118 conn(0x561fa8d18460 msgr2=0x561fa90c9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 -- 172.21.15.31:0/672187118 shutdown_connections 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.269+0000 7fa61c935740 1 -- 172.21.15.31:0/672187118 wait complete. 2024-09-21T06:13:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.270+0000 7fa61c935740 1 -- start start 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.270+0000 7fa61c935740 1 Processor -- start 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.270+0000 7fa61c935740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.270+0000 7fa61c935740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.270+0000 7fa61c935740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa61c935740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561fa8e92700 con 0x561fa8d24070 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa61c935740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561fa8fcc160 con 0x561fa8d03390 2024-09-21T06:13:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa61c935740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561fa90e2650 con 0x561fa8cfb1b0 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa61c935740 10 client.0 _ll_register_callbacks cb 0x561fa8fe59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90df520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59074/0 (socket says 172.21.15.31:59074) 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 -- 172.21.15.31:0/2921716312 learned_addr learned my addr 172.21.15.31:0/2921716312 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa61495e640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 -- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 msgr2=0x561fa90dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 -- 172.21.15.31:0/2921716312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 msgr2=0x561fa90eb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90eb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.271+0000 7fa60ffff640 1 -- 172.21.15.31:0/2921716312 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa8ff9c90 con 0x561fa8cfb1b0 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.272+0000 7fa60ffff640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90df520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa600009230 tx=0x7fa60003d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63698955be007ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.272+0000 7fa5ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.272+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/2921716312 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000d1120 con 0x561fa8cfb1b0 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.272+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa604009d70 con 0x561fa8cfb1b0 2024-09-21T06:13:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.272+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/2921716312 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa60010b220 con 0x561fa8cfb1b0 2024-09-21T06:13:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.273+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/2921716312 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa6000d0ac0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.273+0000 7fa61c935740 10 client.25588 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.273+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561fa90ee390 con 0x561fa8cfb1b0 2024-09-21T06:13:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.273+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/2921716312 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa604009d70 con 0x561fa8cfb1b0 2024-09-21T06:13:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.273+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/2921716312 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa600004040 con 0x561fa8cfb1b0 2024-09-21T06:13:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa61c935740 10 client.25588 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa61c935740 10 client.25588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa61c935740 20 client.25588 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa61c935740 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8ce29f0 0x561fa90f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa61c935740 4 client.25588 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fa9043df0 con 0x561fa8ce29f0 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8ce29f0 0x561fa90f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa5ef7fe640 10 client.25588 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.274+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8ce29f0 0x561fa90f1830 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3222894eaa3f92ea server_cookie=4d8521c188e225 in_seq=0 out_seq=0 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.275+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/2921716312 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fa9043df0 con 0x561fa8ce29f0 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.275+0000 7fa5ef7fe640 10 client.25588 handle_command_reply: tid=0 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 client.25588 shutdown 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 msgr2=0x561fa90df520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90df520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa600009230 tx=0x7fa60003d040 comp rx=0 tx=0).stop 2024-09-21T06:13:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 shutdown_connections 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8ce29f0 0x561fa90f1830 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90eb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 --2- 172.21.15.31:0/2921716312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90df520 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 >> 172.21.15.31:0/2921716312 conn(0x561fa8d18460 msgr2=0x561fa90ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.276+0000 7fa5ef7fe640 0 client.25588 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.277+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 shutdown_connections 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.277+0000 7fa61c935740 1 -- 172.21.15.31:0/2921716312 wait complete. 2024-09-21T06:13:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.277+0000 7fa61c935740 20 client.25588 trim_cache size 0 max 16384 2024-09-21T06:13:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.302+0000 7fa61c935740 1 Processor -- start 2024-09-21T06:13:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 -- start start 2024-09-21T06:13:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ab320 con 0x561fa8d24070 2024-09-21T06:13:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101adb80 con 0x561fa8d03390 2024-09-21T06:13:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.303+0000 7fa61c935740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ab970 con 0x561fa8cfb1b0 2024-09-21T06:13:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa61495e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90e1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59078/0 (socket says 172.21.15.31:59078) 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa61495e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58674/0 (socket says 172.21.15.31:58674) 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 -- 172.21.15.31:0/2103644242 learned_addr learned my addr 172.21.15.31:0/2103644242 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 -- 172.21.15.31:0/2103644242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 msgr2=0x561fa90ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 -- 172.21.15.31:0/2103644242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 msgr2=0x561fa90c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa60ffff640 1 -- 172.21.15.31:0/2103644242 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa90c49c0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.304+0000 7fa61495e640 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.305+0000 7fa60ffff640 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90e1400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa600038020 tx=0x7fa600005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12133c8d118d444 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.305+0000 7fa5f5ffb640 1 -- 172.21.15.31:0/2103644242 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000d0ca0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.305+0000 7fa5f5ffb640 1 -- 172.21.15.31:0/2103644242 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6000eb070 con 0x561fa8cfb1b0 2024-09-21T06:13:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.305+0000 7fa5f5ffb640 1 -- 172.21.15.31:0/2103644242 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ccfb0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.306+0000 7fa61c935740 1 -- 172.21.15.31:0/2103644242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 msgr2=0x561fa90e1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.306+0000 7fa61c935740 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90e1400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa600038020 tx=0x7fa600005fa0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.307+0000 7fa61c935740 1 -- 172.21.15.31:0/2103644242 shutdown_connections 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.307+0000 7fa61c935740 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d24070 0x561fa90c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.307+0000 7fa61c935740 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8d03390 0x561fa90ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.307+0000 7fa61c935740 1 --2- 172.21.15.31:0/2103644242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8cfb1b0 0x561fa90e1400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.307+0000 7fa61c935740 1 -- 172.21.15.31:0/2103644242 >> 172.21.15.31:0/2103644242 conn(0x561fa8d18460 msgr2=0x561fa90cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.308+0000 7fa61c935740 1 -- 172.21.15.31:0/2103644242 shutdown_connections 2024-09-21T06:13:16.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.309+0000 7fa61c935740 1 -- 172.21.15.31:0/2103644242 wait complete. 2024-09-21T06:13:16.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.310+0000 7fa61c935740 1 -- start start 2024-09-21T06:13:16.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.310+0000 7fa61c935740 1 Processor -- start 2024-09-21T06:13:16.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90e3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d03390 0x561fa90e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d24070 0x561fa90e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ac040 con 0x561fa8d03390 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ad4b0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ac710 con 0x561fa8d24070 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61c935740 10 client.0 _ll_register_callbacks cb 0x7fa600005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61495e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d24070 0x561fa90e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61495e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d24070 0x561fa90e4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59092/0 (socket says 172.21.15.31:59092) 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa61495e640 1 -- 172.21.15.31:0/4184324449 learned_addr learned my addr 172.21.15.31:0/4184324449 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.311+0000 7fa60ffff640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90e3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d03390 0x561fa90e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60ffff640 1 -- 172.21.15.31:0/4184324449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d24070 msgr2=0x561fa90e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60ffff640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d24070 0x561fa90e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60ffff640 1 -- 172.21.15.31:0/4184324449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d03390 msgr2=0x561fa90e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60ffff640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d03390 0x561fa90e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60ffff640 1 -- 172.21.15.31:0/4184324449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa915b1f0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d03390 0x561fa90e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa60ffff640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90e3900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6000ce5c0 tx=0x7fa600030f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afef1d3c373cdd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.312+0000 7fa5ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.313+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/4184324449 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000d0ca0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.313+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/4184324449 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa6000d05b0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.313+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6000eb070 con 0x561fa8cfb1b0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.313+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/4184324449 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa6000d1120 con 0x561fa8cfb1b0 2024-09-21T06:13:16.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.313+0000 7fa61c935740 10 client.25571 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:16.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.313+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561fa915b1f0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/4184324449 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6000dad70 con 0x561fa8cfb1b0 2024-09-21T06:13:16.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/4184324449 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa6000d6fc0 con 0x561fa8cfb1b0 2024-09-21T06:13:16.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa61c935740 10 client.25571 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:16.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa61c935740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:16.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa61c935740 20 client.25571 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:16.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa61c935740 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8cb7da0 0x561fa915e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa61c935740 4 client.25571 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:13:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.314+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa6000eb070 con 0x561fa8cb7da0 2024-09-21T06:13:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.315+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8cb7da0 0x561fa915e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.315+0000 7fa5ef7fe640 10 client.25571 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.316+0000 7fa60f7fe640 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8cb7da0 0x561fa915e630 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c0366e636da5da server_cookie=2b144e7fd80e77a in_seq=0 out_seq=0 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.316+0000 7fa5ef7fe640 1 -- 172.21.15.31:0/4184324449 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa6000eb070 con 0x561fa8cb7da0 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.316+0000 7fa5ef7fe640 10 client.25571 handle_command_reply: tid=0 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.317+0000 7fa61c935740 1 client.25571 shutdown 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.317+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 msgr2=0x561fa90e3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.317+0000 7fa61c935740 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90e3900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6000ce5c0 tx=0x7fa600030f50 comp rx=0 tx=0).stop 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.317+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 shutdown_connections 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa8cb7da0 0x561fa915e630 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa8d24070 0x561fa90e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa8d03390 0x561fa90e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa5ef7fe640 0 client.25571 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 --2- 172.21.15.31:0/4184324449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa8cfb1b0 0x561fa90e3900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 >> 172.21.15.31:0/4184324449 conn(0x561fa8d18460 msgr2=0x561fa90e10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 shutdown_connections 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.318+0000 7fa61c935740 1 -- 172.21.15.31:0/4184324449 wait complete. 2024-09-21T06:13:16.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.319+0000 7fa61c935740 20 client.25571 trim_cache size 0 max 16384 2024-09-21T06:13:16.323 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.319+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5f80b6780 msgr2=0x7fa5f80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.319+0000 7fa6163c1640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5f80b6780 0x7fa5f80b8c90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa608009fd0 tx=0x7fa608004ab0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 msgr2=0x7fa61019a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa61019a270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa60002a930 tx=0x7fa600005dc0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 shutdown_connections 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5f80b6780 0x7fa5f80b8c90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa610109e00 0x7fa61019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6101094b0 0x7fa61019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 --2- 172.21.15.31:0/1686604064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa610103bc0 0x7fa61019a270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.320+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 >> 172.21.15.31:0/1686604064 conn(0x7fa61007b910 msgr2=0x7fa610107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.321+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 shutdown_connections 2024-09-21T06:13:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:16.321+0000 7fa6163c1640 1 -- 172.21.15.31:0/1686604064 wait complete. 2024-09-21T06:13:17.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:17 smithi031 ceph-mon[31237]: pgmap v370: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-09-21T06:13:17.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:17 smithi110 ceph-mon[34550]: pgmap v370: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-09-21T06:13:17.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:17 smithi107 ceph-mon[34380]: pgmap v370: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-09-21T06:13:18.162 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:13:18.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:13:18.610 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:19.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.233+0000 7fac899b0640 1 -- 172.21.15.31:0/3712841341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 msgr2=0x7fac8410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.233+0000 7fac899b0640 1 --2- 172.21.15.31:0/3712841341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac8410a9a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fac74009900 tx=0x7fac7402a410 comp rx=0 tx=0).stop 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 -- 172.21.15.31:0/3712841341 shutdown_connections 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 --2- 172.21.15.31:0/3712841341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 0x7fac84118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 --2- 172.21.15.31:0/3712841341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8410aee0 0x7fac84115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 --2- 172.21.15.31:0/3712841341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac8410a9a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 -- 172.21.15.31:0/3712841341 >> 172.21.15.31:0/3712841341 conn(0x7fac84070ec0 msgr2=0x7fac840712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 -- 172.21.15.31:0/3712841341 shutdown_connections 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.235+0000 7fac899b0640 1 -- 172.21.15.31:0/3712841341 wait complete. 2024-09-21T06:13:19.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.236+0000 7fac899b0640 1 Processor -- start 2024-09-21T06:13:19.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.236+0000 7fac899b0640 1 -- start start 2024-09-21T06:13:19.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.237+0000 7fac899b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac841a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac841a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac841a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33508/0 (socket says 172.21.15.31:33508) 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac899b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8410aee0 0x7fac841a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac82ffd640 1 -- 172.21.15.31:0/2987717549 learned_addr learned my addr 172.21.15.31:0/2987717549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 0x7fac841aa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fac8411c3a0 con 0x7fac8410a590 2024-09-21T06:13:19.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac8411c220 con 0x7fac8410aee0 2024-09-21T06:13:19.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac827fc640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8410aee0 0x7fac841a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac8411c520 con 0x7fac84116640 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.238+0000 7fac837fe640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 0x7fac841aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac837fe640 1 -- 172.21.15.31:0/2987717549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8410aee0 msgr2=0x7fac841a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac837fe640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8410aee0 0x7fac841a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac837fe640 1 -- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 msgr2=0x7fac841a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac837fe640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac841a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac837fe640 1 -- 172.21.15.31:0/2987717549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac841aaf40 con 0x7fac84116640 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac837fe640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 0x7fac841aa810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00be20 tx=0x7fac7c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac889ae640 1 -- 172.21.15.31:0/2987717549 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c019070 con 0x7fac84116640 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac889ae640 1 -- 172.21.15.31:0/2987717549 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7c00cd70 con 0x7fac84116640 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.239+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841ab230 con 0x7fac84116640 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.240+0000 7fac889ae640 1 -- 172.21.15.31:0/2987717549 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c012720 con 0x7fac84116640 2024-09-21T06:13:19.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.240+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841b1990 con 0x7fac84116640 2024-09-21T06:13:19.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.243+0000 7fac8ff24740 1 Processor -- start 2024-09-21T06:13:19.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.244+0000 7fac889ae640 1 -- 172.21.15.31:0/2987717549 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fac7c01c070 con 0x7fac84116640 2024-09-21T06:13:19.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.245+0000 7fac8ff24740 1 -- start start 2024-09-21T06:13:19.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.245+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c15430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c15970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac8ff24740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556927b8be60 con 0x556927845490 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac8ff24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556927b7b290 con 0x55692784d670 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac8ff24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556927b7f0a0 con 0x55692786e350 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c15430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c15430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49034/0 (socket says 172.21.15.31:49034) 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 -- 172.21.15.31:0/3108309127 learned_addr learned my addr 172.21.15.31:0/3108309127 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac889ae640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fac6c0b6780 0x7fac6c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 -- 172.21.15.31:0/3108309127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 msgr2=0x556927c15eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c15eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 -- 172.21.15.31:0/3108309127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 msgr2=0x556927c15970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c15970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac827fc640 1 -- 172.21.15.31:0/3108309127 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569278d9930 con 0x55692786e350 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.246+0000 7fac837fe640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fac6c0b6780 0x7fac6c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.247+0000 7fac889ae640 1 -- 172.21.15.31:0/2987717549 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fac7c0e6cb0 con 0x7fac84116640 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.247+0000 7fac827fc640 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c15430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fac78009fd0 tx=0x7fac780094d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998f1e379fd7cd3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.247+0000 7fac69ffb640 1 -- 172.21.15.31:0/3108309127 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7800f040 con 0x55692786e350 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.247+0000 7fac69ffb640 1 -- 172.21.15.31:0/3108309127 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7801d070 con 0x55692786e350 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.247+0000 7fac837fe640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fac6c0b6780 0x7fac6c0b8c90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ac890 tx=0x7fac7c0a91e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.247+0000 7fac69ffb640 1 -- 172.21.15.31:0/3108309127 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac780137c0 con 0x55692786e350 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 -- 172.21.15.31:0/3108309127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 msgr2=0x556927c15430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c15430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fac78009fd0 tx=0x7fac780094d0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 -- 172.21.15.31:0/3108309127 shutdown_connections 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c15eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c15970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3108309127 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c15430 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 -- 172.21.15.31:0/3108309127 >> 172.21.15.31:0/3108309127 conn(0x556927862740 msgr2=0x556927c13320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 -- 172.21.15.31:0/3108309127 shutdown_connections 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.248+0000 7fac8ff24740 1 -- 172.21.15.31:0/3108309127 wait complete. 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.249+0000 7fac8ff24740 1 -- start start 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.249+0000 7fac8ff24740 1 Processor -- start 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c2a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c2f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5569279dc890 con 0x556927845490 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556927b15d40 con 0x55692786e350 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556927c23bb0 con 0x55692784d670 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac8ff24740 10 client.0 _ll_register_callbacks cb 0x556927b79c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:19.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c2a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.250+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c2a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49048/0 (socket says 172.21.15.31:49048) 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 -- 172.21.15.31:0/2360858064 learned_addr learned my addr 172.21.15.31:0/2360858064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac827fc640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c29ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac827fc640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c2f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 -- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 msgr2=0x556927c2f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c2f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 -- 172.21.15.31:0/2360858064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 msgr2=0x556927c29ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c29ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 -- 172.21.15.31:0/2360858064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556927c2fb10 con 0x55692784d670 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac82ffd640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c2a480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fac74000c00 tx=0x7fac740314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53ec8c614e37d55e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac827fc640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c2f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.251+0000 7fac62ffd640 1 -- 172.21.15.31:0/2360858064 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74035040 con 0x55692784d670 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac62ffd640 1 -- 172.21.15.31:0/2360858064 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fac74004240 con 0x55692784d670 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7801d070 con 0x55692784d670 2024-09-21T06:13:19.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac62ffd640 1 -- 172.21.15.31:0/2360858064 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fac740318e0 con 0x55692784d670 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 10 client.25603 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556927c385b0 con 0x55692784d670 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac62ffd640 1 -- 172.21.15.31:0/2360858064 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7801d070 con 0x55692784d670 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac62ffd640 1 -- 172.21.15.31:0/2360858064 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fac7404f4d0 con 0x55692784d670 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 10 client.25603 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 20 client.25603 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55692782ccd0 0x556927c3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 4 client.25603 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.252+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556927b8ddb0 con 0x55692782ccd0 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.253+0000 7fac827fc640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55692782ccd0 0x556927c3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.253+0000 7fac827fc640 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55692782ccd0 0x556927c3bb10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f092d13f2b87821d server_cookie=56cbefd372aded44 in_seq=0 out_seq=0 2024-09-21T06:13:19.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.253+0000 7fac62ffd640 10 client.25603 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:19.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.255+0000 7fac62ffd640 1 -- 172.21.15.31:0/2360858064 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556927b8ddb0 con 0x55692782ccd0 2024-09-21T06:13:19.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.255+0000 7fac62ffd640 10 client.25603 handle_command_reply: tid=0 2024-09-21T06:13:19.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.255+0000 7fac8ff24740 1 client.25603 shutdown 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 msgr2=0x556927c2a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c2a480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fac74000c00 tx=0x7fac740314a0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 shutdown_connections 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55692782ccd0 0x556927c3bb10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c2f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c2a480 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 --2- 172.21.15.31:0/2360858064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c29ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 >> 172.21.15.31:0/2360858064 conn(0x556927862740 msgr2=0x556927c185c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac62ffd640 0 client.25603 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:19.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 shutdown_connections 2024-09-21T06:13:19.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 1 -- 172.21.15.31:0/2360858064 wait complete. 2024-09-21T06:13:19.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.256+0000 7fac8ff24740 20 client.25603 trim_cache size 0 max 16384 2024-09-21T06:13:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 Processor -- start 2024-09-21T06:13:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 -- start start 2024-09-21T06:13:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c13420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c2f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fac841b3e70 con 0x556927845490 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac841b6630 con 0x55692784d670 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac8ff24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac841b44c0 con 0x55692786e350 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.284+0000 7fac827fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c2f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c2f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49052/0 (socket says 172.21.15.31:49052) 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac82ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 -- 172.21.15.31:0/3804015049 learned_addr learned my addr 172.21.15.31:0/3804015049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c13420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 -- 172.21.15.31:0/3804015049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 msgr2=0x556927c250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c250f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 -- 172.21.15.31:0/3804015049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 msgr2=0x556927c13420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c13420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 -- 172.21.15.31:0/3804015049 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556927c19540 con 0x55692786e350 2024-09-21T06:13:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac827fc640 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c2f310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac78013410 tx=0x7fac780133a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81298c939931758 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac697fa640 1 -- 172.21.15.31:0/3804015049 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78031020 con 0x55692786e350 2024-09-21T06:13:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac697fa640 1 -- 172.21.15.31:0/3804015049 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7801d050 con 0x55692786e350 2024-09-21T06:13:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.285+0000 7fac697fa640 1 -- 172.21.15.31:0/3804015049 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac780189d0 con 0x55692786e350 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 -- 172.21.15.31:0/3804015049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 msgr2=0x556927c2f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c2f310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac78013410 tx=0x7fac780133a0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 -- 172.21.15.31:0/3804015049 shutdown_connections 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692786e350 0x556927c2f310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692784d670 0x556927c250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3804015049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c13420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 -- 172.21.15.31:0/3804015049 >> 172.21.15.31:0/3804015049 conn(0x556927862740 msgr2=0x556927c12d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 -- 172.21.15.31:0/3804015049 shutdown_connections 2024-09-21T06:13:19.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 -- 172.21.15.31:0/3804015049 wait complete. 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.287+0000 7fac8ff24740 1 -- start start 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 Processor -- start 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c35720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fac841b4b10 con 0x556927845490 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fac841b5f60 con 0x55692786e350 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac841b51c0 con 0x55692784d670 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac8ff24740 10 client.0 _ll_register_callbacks cb 0x7fac74005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c35720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c35720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33538/0 (socket says 172.21.15.31:33538) 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac827fc640 1 -- 172.21.15.31:0/3103463600 learned_addr learned my addr 172.21.15.31:0/3103463600 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac827fc640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.288+0000 7fac82ffd640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac827fc640 1 -- 172.21.15.31:0/3103463600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 msgr2=0x556927c35ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac827fc640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c35ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac827fc640 1 -- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 msgr2=0x556927c304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac827fc640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac827fc640 1 -- 172.21.15.31:0/3103463600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556927c30bc0 con 0x556927845490 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac827fc640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c35720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fac78006b80 tx=0x7fac78025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ecccb98339860b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:19.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.289+0000 7fac62ffd640 1 -- 172.21.15.31:0/3103463600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78031070 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.290+0000 7fac62ffd640 1 -- 172.21.15.31:0/3103463600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fac78013990 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.290+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac7801d050 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.290+0000 7fac62ffd640 1 -- 172.21.15.31:0/3103463600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fac78013b90 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.290+0000 7fac8ff24740 10 client.15846 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.290+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556927ca6e30 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac62ffd640 1 -- 172.21.15.31:0/3103463600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7801d050 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac62ffd640 1 -- 172.21.15.31:0/3103463600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fac78015050 con 0x556927845490 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac8ff24740 10 client.15846 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac8ff24740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac8ff24740 20 client.15846 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556927802080 0x556927ca92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac8ff24740 4 client.15846 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:13:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.291+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556927b89860 con 0x556927802080 2024-09-21T06:13:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.292+0000 7fac82ffd640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556927802080 0x556927ca92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.292+0000 7fac62ffd640 10 client.15846 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:19.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.292+0000 7fac82ffd640 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556927802080 0x556927ca92e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40acd80b24b8239 server_cookie=1b35be9f5e15162b in_seq=0 out_seq=0 2024-09-21T06:13:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac62ffd640 1 -- 172.21.15.31:0/3103463600 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556927b89860 con 0x556927802080 2024-09-21T06:13:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac62ffd640 10 client.15846 handle_command_reply: tid=0 2024-09-21T06:13:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac8ff24740 1 client.15846 shutdown 2024-09-21T06:13:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 msgr2=0x556927c35720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c35720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fac78006b80 tx=0x7fac78025040 comp rx=0 tx=0).stop 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 shutdown_connections 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.294+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556927802080 0x556927ca92e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55692786e350 0x556927c304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac62ffd640 0 client.15846 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55692784d670 0x556927c35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 1 --2- 172.21.15.31:0/3103463600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556927845490 0x556927c35720 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 >> 172.21.15.31:0/3103463600 conn(0x556927862740 msgr2=0x556927c23df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 shutdown_connections 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 1 -- 172.21.15.31:0/3103463600 wait complete. 2024-09-21T06:13:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.295+0000 7fac8ff24740 20 client.15846 trim_cache size 0 max 16384 2024-09-21T06:13:19.299 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:13:19.299 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:13:19.299 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "b82343df-281a-40b4-a471-6077c3b4ef6f", 2024-09-21T06:13:19.299 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:13:19.299 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:13:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.296+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fac6c0b6780 msgr2=0x7fac6c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.296+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fac6c0b6780 0x7fac6c0b8c90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ac890 tx=0x7fac7c0a91e0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.296+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 msgr2=0x7fac841aa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.296+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 0x7fac841aa810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac7c00be20 tx=0x7fac7c009ed0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 shutdown_connections 2024-09-21T06:13:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac84116640 0x7fac841aa810 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac8410aee0 0x7fac841a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fac6c0b6780 0x7fac6c0b8c90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 --2- 172.21.15.31:0/2987717549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac8410a590 0x7fac841a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 >> 172.21.15.31:0/2987717549 conn(0x7fac84070ec0 msgr2=0x7fac8410ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 shutdown_connections 2024-09-21T06:13:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:19.297+0000 7fac899b0640 1 -- 172.21.15.31:0/2987717549 wait complete. 2024-09-21T06:13:19.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:19 smithi031 ceph-mon[31237]: pgmap v371: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 39 MiB/s wr, 1.39k op/s 2024-09-21T06:13:19.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:19 smithi110 ceph-mon[34550]: pgmap v371: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 39 MiB/s wr, 1.39k op/s 2024-09-21T06:13:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:19 smithi107 ceph-mon[34380]: pgmap v371: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 39 MiB/s wr, 1.39k op/s 2024-09-21T06:13:20.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:13:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 100.0 s, 4018.4 tps, lat 24.927 ms stddev 30.108 2024-09-21T06:13:20.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:20 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:13:20.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:20 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:20.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:20 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:13:20.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:20 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:20.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:20 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:13:20.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:20 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:20.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:20 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:13:20.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:20 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:20.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:20 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:13:20.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:20 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:20.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:20 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:13:20.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:20 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:13:20.655 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.227+0000 7f0b17310640 1 -- 172.21.15.31:0/766922468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 msgr2=0x7f0b1010a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.227+0000 7f0b17310640 1 --2- 172.21.15.31:0/766922468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 0x7f0b1010a5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009860 tx=0x7f0b0c02a340 comp rx=0 tx=0).stop 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.228+0000 7f0b17310640 1 -- 172.21.15.31:0/766922468 shutdown_connections 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.228+0000 7f0b17310640 1 --2- 172.21.15.31:0/766922468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b10110420 0x7f0b10116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.228+0000 7f0b17310640 1 --2- 172.21.15.31:0/766922468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b1010fad0 0x7f0b1010fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.228+0000 7f0b17310640 1 --2- 172.21.15.31:0/766922468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 0x7f0b1010a5a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.228+0000 7f0b17310640 1 -- 172.21.15.31:0/766922468 >> 172.21.15.31:0/766922468 conn(0x7f0b100780f0 msgr2=0x7f0b101081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.229+0000 7f0b17310640 1 -- 172.21.15.31:0/766922468 shutdown_connections 2024-09-21T06:13:21.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.229+0000 7f0b17310640 1 -- 172.21.15.31:0/766922468 wait complete. 2024-09-21T06:13:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.230+0000 7f0b17310640 1 Processor -- start 2024-09-21T06:13:21.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 -- start start 2024-09-21T06:13:21.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 0x7f0b10083c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 0x7f0b1007cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b10110420 0x7f0b1007d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1011a0f0 con 0x7f0b10110420 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10119f70 con 0x7f0b1010a190 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b17310640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1011a270 con 0x7f0b1010fad0 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b148ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 0x7f0b1007cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b148ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 0x7f0b1007cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49090/0 (socket says 172.21.15.31:49090) 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.231+0000 7f0b148ab640 1 -- 172.21.15.31:0/2095293006 learned_addr learned my addr 172.21.15.31:0/2095293006 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.232+0000 7f0b148ab640 1 -- 172.21.15.31:0/2095293006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 msgr2=0x7f0b10083c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.232+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 0x7f0b10083c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.232+0000 7f0b148ab640 1 -- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b10110420 msgr2=0x7f0b1007d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.232+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b10110420 0x7f0b1007d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.232+0000 7f0b148ab640 1 -- 172.21.15.31:0/2095293006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1007dc80 con 0x7f0b1010fad0 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.232+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 0x7f0b1007cf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000e970 tx=0x7f0b0000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.233+0000 7f0b067fc640 1 -- 172.21.15.31:0/2095293006 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000cd80 con 0x7f0b1010fad0 2024-09-21T06:13:21.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.233+0000 7f0b067fc640 1 -- 172.21.15.31:0/2095293006 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000035f0 con 0x7f0b1010fad0 2024-09-21T06:13:21.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.234+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b10082920 con 0x7f0b1010fad0 2024-09-21T06:13:21.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.234+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b10082e60 con 0x7f0b1010fad0 2024-09-21T06:13:21.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.234+0000 7f0b067fc640 1 -- 172.21.15.31:0/2095293006 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00010690 con 0x7f0b1010fad0 2024-09-21T06:13:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.237+0000 7f0b1d884740 1 Processor -- start 2024-09-21T06:13:21.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.237+0000 7f0b1d884740 1 -- start start 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5cb7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cb7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cb80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b1d884740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562ff5c2de50 con 0x562ff58e71b0 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b1d884740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ff5c1d280 con 0x562ff5910070 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b1d884740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562ff5c21090 con 0x562ff58ef390 2024-09-21T06:13:21.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b150ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cb7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b150ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cb7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33576/0 (socket says 172.21.15.31:33576) 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.238+0000 7f0b150ac640 1 -- 172.21.15.31:0/1362435814 learned_addr learned my addr 172.21.15.31:0/1362435814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.239+0000 7f0b150ac640 1 -- 172.21.15.31:0/1362435814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 msgr2=0x562ff5cb80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.239+0000 7f0b148ab640 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cb80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.240+0000 7f0b150ac640 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cb80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.240+0000 7f0b150ac640 1 -- 172.21.15.31:0/1362435814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 msgr2=0x562ff5cb7650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.240+0000 7f0b150ac640 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5cb7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.240+0000 7f0b067fc640 1 -- 172.21.15.31:0/2095293006 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0b00004000 con 0x7f0b1010fad0 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.240+0000 7f0b150ac640 1 -- 172.21.15.31:0/1362435814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff597b650 con 0x562ff58e71b0 2024-09-21T06:13:21.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.240+0000 7f0b148ab640 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cb80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.241+0000 7f0b150ac640 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cb7b90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c035040 tx=0x7f0b0c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ed7b5c34c7173a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.241+0000 7f0b067fc640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0afc0b6850 0x7f0afc0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.242+0000 7f0afb7fe640 1 -- 172.21.15.31:0/1362435814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c03f020 con 0x562ff58e71b0 2024-09-21T06:13:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.242+0000 7f0afb7fe640 1 -- 172.21.15.31:0/1362435814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0c030d50 con 0x562ff58e71b0 2024-09-21T06:13:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.242+0000 7f0afb7fe640 1 -- 172.21.15.31:0/1362435814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c037040 con 0x562ff58e71b0 2024-09-21T06:13:21.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.242+0000 7f0b158ad640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0afc0b6850 0x7f0afc0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.242+0000 7f0b067fc640 1 -- 172.21.15.31:0/2095293006 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0b000e7b20 con 0x7f0b1010fad0 2024-09-21T06:13:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 -- 172.21.15.31:0/1362435814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 msgr2=0x562ff5cb7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cb7b90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c035040 tx=0x7f0b0c005c90 comp rx=0 tx=0).stop 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b158ad640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0afc0b6850 0x7f0afc0b8d60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b080098d0 tx=0x7f0b08009860 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 -- 172.21.15.31:0/1362435814 shutdown_connections 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cb80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cb7b90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 --2- 172.21.15.31:0/1362435814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5cb7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 -- 172.21.15.31:0/1362435814 >> 172.21.15.31:0/1362435814 conn(0x562ff5904460 msgr2=0x562ff5cb5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.243+0000 7f0b1d884740 1 -- 172.21.15.31:0/1362435814 shutdown_connections 2024-09-21T06:13:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.244+0000 7f0b1d884740 1 -- 172.21.15.31:0/1362435814 wait complete. 2024-09-21T06:13:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.244+0000 7f0b1d884740 1 -- start start 2024-09-21T06:13:21.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 Processor -- start 2024-09-21T06:13:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5ccf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562ff5a7e700 con 0x562ff58ef390 2024-09-21T06:13:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ff5bb8110 con 0x562ff58e71b0 2024-09-21T06:13:21.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562ff5cc5dd0 con 0x562ff5910070 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.245+0000 7f0b1d884740 10 client.0 _ll_register_callbacks cb 0x562ff5bd1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41352/0 (socket says 172.21.15.31:41352) 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 -- 172.21.15.31:0/2973520053 learned_addr learned my addr 172.21.15.31:0/2973520053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b158ad640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5ccf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 -- 172.21.15.31:0/2973520053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 msgr2=0x562ff5ccf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5ccf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 -- 172.21.15.31:0/2973520053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 msgr2=0x562ff5cc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 -- 172.21.15.31:0/2973520053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff5be5c40 con 0x562ff58e71b0 2024-09-21T06:13:21.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.246+0000 7f0b150ac640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc2d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c02a860 tx=0x7f0b0c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d9f67b7024c604 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0af8ff9640 1 -- 172.21.15.31:0/2973520053 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c048070 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b0c030d50 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0af8ff9640 1 -- 172.21.15.31:0/2973520053 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0b0c004250 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0af8ff9640 1 -- 172.21.15.31:0/2973520053 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b0c054050 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0b1d884740 10 client.25589 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.247+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562ff5cd1a50 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.248+0000 7f0af8ff9640 1 -- 172.21.15.31:0/2973520053 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0c004470 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.248+0000 7f0af8ff9640 1 -- 172.21.15.31:0/2973520053 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0b0c03a030 con 0x562ff58e71b0 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.248+0000 7f0b1d884740 10 client.25589 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.248+0000 7f0b1d884740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.248+0000 7f0b1d884740 20 client.25589 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:21.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.248+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58ce9f0 0x562ff5cd4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.249+0000 7f0b1d884740 4 client.25589 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:21.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.249+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0b0c030d50 con 0x562ff58ce9f0 2024-09-21T06:13:21.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.249+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58ce9f0 0x562ff5cd4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.249+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58ce9f0 0x562ff5cd4db0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8bb3152f05d29cb server_cookie=c5917026887179c6 in_seq=0 out_seq=0 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.249+0000 7f0af8ff9640 10 client.25589 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0af8ff9640 1 -- 172.21.15.31:0/2973520053 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0b0c030d50 con 0x562ff58ce9f0 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0af8ff9640 10 client.25589 handle_command_reply: tid=0 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 client.25589 shutdown 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 msgr2=0x562ff5cc2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc2d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c02a860 tx=0x7f0b0c005850 comp rx=0 tx=0).stop 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 shutdown_connections 2024-09-21T06:13:21.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58ce9f0 0x562ff5cd4db0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5ccf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2973520053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc2d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0af8ff9640 0 client.25589 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.250+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 >> 172.21.15.31:0/2973520053 conn(0x562ff5904460 msgr2=0x562ff5cba7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.251+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 shutdown_connections 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.251+0000 7f0b1d884740 1 -- 172.21.15.31:0/2973520053 wait complete. 2024-09-21T06:13:21.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.251+0000 7f0b1d884740 20 client.25589 trim_cache size 0 max 16384 2024-09-21T06:13:21.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.286+0000 7f0b1d884740 1 Processor -- start 2024-09-21T06:13:21.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.286+0000 7f0b1d884740 1 -- start start 2024-09-21T06:13:21.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.286+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cb8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5cb4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b148ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cb8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b148ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cb8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33588/0 (socket says 172.21.15.31:33588) 2024-09-21T06:13:21.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b148ab640 1 -- 172.21.15.31:0/2785557613 learned_addr learned my addr 172.21.15.31:0/2785557613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:21.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101bc450 con 0x562ff58ef390 2024-09-21T06:13:21.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101becd0 con 0x562ff58e71b0 2024-09-21T06:13:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101bcac0 con 0x562ff5910070 2024-09-21T06:13:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.287+0000 7f0b158ad640 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5cb4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.288+0000 7f0b148ab640 1 -- 172.21.15.31:0/2785557613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 msgr2=0x562ff5cb4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.288+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5cb4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.288+0000 7f0b148ab640 1 -- 172.21.15.31:0/2785557613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 msgr2=0x562ff5cc4c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:13:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.288+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc4c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.288+0000 7f0b148ab640 1 -- 172.21.15.31:0/2785557613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff5cc1240 con 0x562ff58ef390 2024-09-21T06:13:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.288+0000 7f0b148ab640 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cb8d90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b000aabc0 tx=0x7f0b000e9fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c80abdc1551879 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.289+0000 7f0afaffd640 1 -- 172.21.15.31:0/2785557613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000a90c0 con 0x562ff58ef390 2024-09-21T06:13:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.289+0000 7f0afaffd640 1 -- 172.21.15.31:0/2785557613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000b2070 con 0x562ff58ef390 2024-09-21T06:13:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.289+0000 7f0afaffd640 1 -- 172.21.15.31:0/2785557613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000a97f0 con 0x562ff58ef390 2024-09-21T06:13:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.290+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 msgr2=0x562ff5cb8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.290+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cb8d90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b000aabc0 tx=0x7f0b000e9fb0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.291+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 shutdown_connections 2024-09-21T06:13:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.291+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff5910070 0x562ff5cb4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.291+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58ef390 0x562ff5cb8d90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.291+0000 7f0b1d884740 1 --2- 172.21.15.31:0/2785557613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff58e71b0 0x562ff5cc4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.291+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 >> 172.21.15.31:0/2785557613 conn(0x562ff5904460 msgr2=0x562ff5cb8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.292+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 shutdown_connections 2024-09-21T06:13:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.292+0000 7f0b1d884740 1 -- 172.21.15.31:0/2785557613 wait complete. 2024-09-21T06:13:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.292+0000 7f0b1d884740 1 -- start start 2024-09-21T06:13:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.293+0000 7f0b1d884740 1 Processor -- start 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.293+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cd6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cd6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b1d884740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5d3e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cd6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cd6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49128/0 (socket says 172.21.15.31:49128) 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 -- 172.21.15.31:0/4152896162 learned_addr learned my addr 172.21.15.31:0/4152896162 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b150ac640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cd6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101bd190 con 0x562ff58e71b0 2024-09-21T06:13:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101be600 con 0x562ff5910070 2024-09-21T06:13:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b101bd860 con 0x562ff58ef390 2024-09-21T06:13:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b158ad640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5d3e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 -- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 msgr2=0x562ff5d3e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5d3e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 -- 172.21.15.31:0/4152896162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 msgr2=0x562ff5cd6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cd6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.294+0000 7f0b148ab640 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff5cd6d70 con 0x562ff58ef390 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0b1d884740 10 client.0 _ll_register_callbacks cb 0x7f0b00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0b148ab640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cd6580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b000058c0 tx=0x7f0b000039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31160379f42524ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b000b2070 con 0x562ff58ef390 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0af8ff9640 10 client.25624 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0af8ff9640 1 -- 172.21.15.31:0/4152896162 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000a97f0 con 0x562ff58ef390 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0af8ff9640 1 -- 172.21.15.31:0/4152896162 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0b000e54d0 con 0x562ff58ef390 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0af8ff9640 1 -- 172.21.15.31:0/4152896162 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b000b2220 con 0x562ff58ef390 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0b1d884740 10 client.25624 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.295+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x562ff5d3f820 con 0x562ff58ef390 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0af8ff9640 1 -- 172.21.15.31:0/4152896162 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0b000c3b00 con 0x562ff58ef390 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b1d884740 10 client.25624 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b1d884740 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b1d884740 20 client.25624 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b1d884740 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58a3da0 0x562ff5d41de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b1d884740 4 client.25624 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:13:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0b000b2070 con 0x562ff58a3da0 2024-09-21T06:13:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.296+0000 7f0b150ac640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58a3da0 0x562ff5d41de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.297+0000 7f0b150ac640 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58a3da0 0x562ff5d41de0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a618afdea9c91 server_cookie=40663b9eee105e6 in_seq=0 out_seq=0 2024-09-21T06:13:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.297+0000 7f0af8ff9640 10 client.25624 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.297+0000 7f0af8ff9640 1 -- 172.21.15.31:0/4152896162 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x7f0b000b2070 con 0x562ff58a3da0 2024-09-21T06:13:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.297+0000 7f0af8ff9640 10 client.25624 handle_command_reply: tid=0 2024-09-21T06:13:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.297+0000 7f0b1d884740 1 client.25624 shutdown 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 msgr2=0x562ff5cd6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cd6580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0b000058c0 tx=0x7f0b000039d0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 shutdown_connections 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562ff58a3da0 0x562ff5d41de0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ff5910070 0x562ff5d3e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562ff58ef390 0x562ff5cd6580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 --2- 172.21.15.31:0/4152896162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ff58e71b0 0x562ff5cd6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0af8ff9640 0 client.25624 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 >> 172.21.15.31:0/4152896162 conn(0x562ff5904460 msgr2=0x562ff5cb9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 shutdown_connections 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 1 -- 172.21.15.31:0/4152896162 wait complete. 2024-09-21T06:13:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.298+0000 7f0b1d884740 20 client.25624 trim_cache size 0 max 16384 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "b82343df-281a-40b4-a471-6077c3b4ef6f", 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:13:21.303 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.304 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:13:21.305 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:21.307 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:21.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:21.308 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0afc0b6850 msgr2=0x7f0afc0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0afc0b6850 0x7f0afc0b8d60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b080098d0 tx=0x7f0b08009860 comp rx=0 tx=0).stop 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 msgr2=0x7f0b1007cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 0x7f0b1007cf20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000e970 tx=0x7f0b0000ee30 comp rx=0 tx=0).stop 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 shutdown_connections 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0afc0b6850 0x7f0afc0b8d60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b10110420 0x7f0b1007d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b1010fad0 0x7f0b1007cf20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 --2- 172.21.15.31:0/2095293006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1010a190 0x7f0b10083c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 >> 172.21.15.31:0/2095293006 conn(0x7f0b100780f0 msgr2=0x7f0b101086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.300+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 shutdown_connections 2024-09-21T06:13:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:21.301+0000 7f0b17310640 1 -- 172.21.15.31:0/2095293006 wait complete. 2024-09-21T06:13:21.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:21 smithi031 ceph-mon[31237]: pgmap v372: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 40 MiB/s wr, 1.35k op/s 2024-09-21T06:13:21.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:21 smithi110 ceph-mon[34550]: pgmap v372: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 40 MiB/s wr, 1.35k op/s 2024-09-21T06:13:21.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:21 smithi107 ceph-mon[34380]: pgmap v372: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 40 MiB/s wr, 1.35k op/s 2024-09-21T06:13:22.019 INFO:tasks.cephfs.filesystem:scrub status for tag:b82343df-281a-40b4-a471-6077c3b4ef6f - {'path': '/', 'tag': 'b82343df-281a-40b4-a471-6077c3b4ef6f', 'options': 'recursive,force'} 2024-09-21T06:13:22.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:22 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:13:22.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:22 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:13:22.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:22 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:13:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 -- 172.21.15.31:0/4134090552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108120 msgr2=0x7f7688108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 --2- 172.21.15.31:0/4134090552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108120 0x7f7688108530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f767c009860 tx=0x7f767c02a340 comp rx=0 tx=0).stop 2024-09-21T06:13:22.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 -- 172.21.15.31:0/4134090552 shutdown_connections 2024-09-21T06:13:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 --2- 172.21.15.31:0/4134090552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108a70 0x7f768810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 --2- 172.21.15.31:0/4134090552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108120 0x7f7688108530 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 --2- 172.21.15.31:0/4134090552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76881027f0 0x7f7688102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 -- 172.21.15.31:0/4134090552 >> 172.21.15.31:0/4134090552 conn(0x7f76880fe230 msgr2=0x7f7688100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 -- 172.21.15.31:0/4134090552 shutdown_connections 2024-09-21T06:13:22.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.642+0000 7f768c9e8640 1 -- 172.21.15.31:0/4134090552 wait complete. 2024-09-21T06:13:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 Processor -- start 2024-09-21T06:13:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 -- start start 2024-09-21T06:13:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76881027f0 0x7f768819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 0x7f768819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108a70 0x7f76881a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7688115a40 con 0x7f7688108120 2024-09-21T06:13:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76881158c0 con 0x7f7688108a70 2024-09-21T06:13:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f768c9e8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7688115bc0 con 0x7f76881027f0 2024-09-21T06:13:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.643+0000 7f7686ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 0x7f768819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 0x7f768819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33614/0 (socket says 172.21.15.31:33614) 2024-09-21T06:13:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 -- 172.21.15.31:0/463307190 learned_addr learned my addr 172.21.15.31:0/463307190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7687fff640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108a70 0x7f76881a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f76877fe640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76881027f0 0x7f768819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 -- 172.21.15.31:0/463307190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76881027f0 msgr2=0x7f768819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76881027f0 0x7f768819c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 -- 172.21.15.31:0/463307190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108a70 msgr2=0x7f76881a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108a70 0x7f76881a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.644+0000 7f7686ffd640 1 -- 172.21.15.31:0/463307190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76881a2390 con 0x7f7688108120 2024-09-21T06:13:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.645+0000 7f7686ffd640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 0x7f768819ca00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f767c035040 tx=0x7f767c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.645+0000 7f7684ff9640 1 -- 172.21.15.31:0/463307190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c037040 con 0x7f7688108120 2024-09-21T06:13:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.645+0000 7f7684ff9640 1 -- 172.21.15.31:0/463307190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767c004050 con 0x7f7688108120 2024-09-21T06:13:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.645+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76881a2620 con 0x7f7688108120 2024-09-21T06:13:22.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.646+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881a2ab0 con 0x7f7688108120 2024-09-21T06:13:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.647+0000 7f7684ff9640 1 -- 172.21.15.31:0/463307190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c037040 con 0x7f7688108120 2024-09-21T06:13:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 Processor -- start 2024-09-21T06:13:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 -- start start 2024-09-21T06:13:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02551bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02551c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e02551c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0254906f0 con 0x55e0251554d0 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e02547fb20 con 0x55e0251761b0 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7694c91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e025483930 con 0x55e02514d2f0 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f76877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02551c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f7687fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02551bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f76877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02551c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49162/0 (socket says 172.21.15.31:49162) 2024-09-21T06:13:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.650+0000 7f76877fe640 1 -- 172.21.15.31:0/1473258696 learned_addr learned my addr 172.21.15.31:0/1473258696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.651+0000 7f76877fe640 1 -- 172.21.15.31:0/1473258696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 msgr2=0x55e02551bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.651+0000 7f76877fe640 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02551bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.651+0000 7f76877fe640 1 -- 172.21.15.31:0/1473258696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 msgr2=0x55e02551c620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.651+0000 7f76877fe640 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e02551c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.651+0000 7f76877fe640 1 -- 172.21.15.31:0/1473258696 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e025445530 con 0x55e02514d2f0 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.651+0000 7f76877fe640 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02551c0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7680010400 tx=0x7f76800108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c441bed25fb0a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.652+0000 7f766dffb640 1 -- 172.21.15.31:0/1473258696 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680003b70 con 0x55e02514d2f0 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.652+0000 7f766dffb640 1 -- 172.21.15.31:0/1473258696 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7680003d10 con 0x55e02514d2f0 2024-09-21T06:13:22.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.652+0000 7f766dffb640 1 -- 172.21.15.31:0/1473258696 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7680011a90 con 0x55e02514d2f0 2024-09-21T06:13:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7684ff9640 1 -- 172.21.15.31:0/463307190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f767c043020 con 0x7f7688108120 2024-09-21T06:13:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 -- 172.21.15.31:0/1473258696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 msgr2=0x55e02551c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02551c0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7680010400 tx=0x7f76800108c0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 -- 172.21.15.31:0/1473258696 shutdown_connections 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e02551c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02551c0e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 --2- 172.21.15.31:0/1473258696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02551bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 -- 172.21.15.31:0/1473258696 >> 172.21.15.31:0/1473258696 conn(0x55e02516a300 msgr2=0x55e025519a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 -- 172.21.15.31:0/1473258696 shutdown_connections 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.654+0000 7f7694c91740 1 -- 172.21.15.31:0/1473258696 wait complete. 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7694c91740 1 -- start start 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7694c91740 1 Processor -- start 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7684ff9640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f76700b67f0 0x7f76700b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7687fff640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f76700b67f0 0x7f76700b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7684ff9640 1 -- 172.21.15.31:0/463307190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f767c1065a0 con 0x7f7688108120 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7694c91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025527b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.655+0000 7f7694c91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e0255280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7694c91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02552d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e0255280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e0255280f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33626/0 (socket says 172.21.15.31:33626) 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 -- 172.21.15.31:0/336089505 learned_addr learned my addr 172.21.15.31:0/336089505 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0253b7ab0 con 0x55e0251554d0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e025425730 con 0x55e0251761b0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e025522280 con 0x55e02514d2f0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7694c91740 10 client.0 _ll_register_callbacks cb 0x55e0254796b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f76877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025527b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 -- 172.21.15.31:0/336089505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 msgr2=0x55e025527b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025527b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 -- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 msgr2=0x55e02552d120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02552d120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7686ffd640 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0251e1790 con 0x55e0251554d0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f7687fff640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f76700b67f0 0x7f76700b8d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7678005fd0 tx=0x7f7678009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.656+0000 7f76877fe640 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025527b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.657+0000 7f7686ffd640 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e0255280f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f767c0cc360 tx=0x7f767c0cc1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e579e279b673c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.657+0000 7f7666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.657+0000 7f7666ffd640 1 -- 172.21.15.31:0/336089505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0cbee0 con 0x55e0251554d0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.657+0000 7f7666ffd640 1 -- 172.21.15.31:0/336089505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f767c1060d0 con 0x55e0251554d0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.657+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7680003d10 con 0x55e0251554d0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.657+0000 7f7666ffd640 1 -- 172.21.15.31:0/336089505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f767c0ccad0 con 0x55e0251554d0 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.658+0000 7f7694c91740 10 client.15870 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:22.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.658+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e02525a620 con 0x55e0251554d0 2024-09-21T06:13:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.658+0000 7f7666ffd640 1 -- 172.21.15.31:0/336089505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767c0d0eb0 con 0x55e0251554d0 2024-09-21T06:13:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.658+0000 7f7666ffd640 1 -- 172.21.15.31:0/336089505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f767c0c7d20 con 0x55e0251554d0 2024-09-21T06:13:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.658+0000 7f7694c91740 10 client.15870 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.658+0000 7f7694c91740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.659+0000 7f7694c91740 20 client.15870 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:22.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.659+0000 7f7694c91740 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e025134d80 0x55e02554a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.659+0000 7f7694c91740 4 client.15870 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.659+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7680003d10 con 0x55e025134d80 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.659+0000 7f76877fe640 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e025134d80 0x55e02554a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.661+0000 7f76877fe640 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e025134d80 0x55e02554a520 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61142ed463cdb11a server_cookie=9d478fa11531960f in_seq=0 out_seq=0 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.661+0000 7f7666ffd640 10 client.15870 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.661+0000 7f7666ffd640 1 -- 172.21.15.31:0/336089505 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7680003d10 con 0x55e025134d80 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.661+0000 7f7666ffd640 10 client.15870 handle_command_reply: tid=0 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.662+0000 7f7694c91740 1 client.15870 shutdown 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.662+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 msgr2=0x55e0255280f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.662+0000 7f7694c91740 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e0255280f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f767c0cc360 tx=0x7f767c0cc1a0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.663+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 shutdown_connections 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.664+0000 7f7694c91740 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e025134d80 0x55e02554a520 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.664+0000 7f7694c91740 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e02552d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.664+0000 7f7666ffd640 0 client.15870 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.664+0000 7f7694c91740 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251554d0 0x55e0255280f0 secure :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f767c0cc360 tx=0x7f767c0cc1a0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.664+0000 7f7694c91740 1 --2- 172.21.15.31:0/336089505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025527b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.664+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 >> 172.21.15.31:0/336089505 conn(0x55e02516a300 msgr2=0x55e025518150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.665+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 shutdown_connections 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.665+0000 7f7694c91740 1 -- 172.21.15.31:0/336089505 wait complete. 2024-09-21T06:13:22.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.665+0000 7f7694c91740 20 client.15870 trim_cache size 0 max 16384 2024-09-21T06:13:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.706+0000 7f7694c91740 1 Processor -- start 2024-09-21T06:13:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.706+0000 7f7694c91740 1 -- start start 2024-09-21T06:13:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f7694c91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025518150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f7694c91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251761b0 0x55e025518690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f7694c91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e02510a1c0 0x55e025524d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f7694c91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76881ab2e0 con 0x55e0251761b0 2024-09-21T06:13:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f7694c91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76881adb60 con 0x55e02510a1c0 2024-09-21T06:13:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f7694c91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76881ab950 con 0x55e02514d2f0 2024-09-21T06:13:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f76877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025518150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.707+0000 7f76877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025518150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49178/0 (socket says 172.21.15.31:49178) 2024-09-21T06:13:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f76877fe640 1 -- 172.21.15.31:0/4056650395 learned_addr learned my addr 172.21.15.31:0/4056650395 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7687fff640 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e02510a1c0 0x55e025524d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7686ffd640 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251761b0 0x55e025518690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7687fff640 1 -- 172.21.15.31:0/4056650395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 msgr2=0x55e025518150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7687fff640 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025518150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7687fff640 1 -- 172.21.15.31:0/4056650395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251761b0 msgr2=0x55e025518690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7687fff640 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251761b0 0x55e025518690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.708+0000 7f7687fff640 1 -- 172.21.15.31:0/4056650395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02551a420 con 0x55e02510a1c0 2024-09-21T06:13:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.709+0000 7f7687fff640 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e02510a1c0 0x55e025524d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f767801e4a0 tx=0x7f7678017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5f713c4a897237 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.709+0000 7f766d7fa640 1 -- 172.21.15.31:0/4056650395 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767802e070 con 0x55e02510a1c0 2024-09-21T06:13:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.709+0000 7f766d7fa640 1 -- 172.21.15.31:0/4056650395 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7678029420 con 0x55e02510a1c0 2024-09-21T06:13:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.710+0000 7f766d7fa640 1 -- 172.21.15.31:0/4056650395 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767800f3c0 con 0x55e02510a1c0 2024-09-21T06:13:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 -- 172.21.15.31:0/4056650395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e02510a1c0 msgr2=0x55e025524d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e02510a1c0 0x55e025524d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f767801e4a0 tx=0x7f7678017000 comp rx=0 tx=0).stop 2024-09-21T06:13:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 -- 172.21.15.31:0/4056650395 shutdown_connections 2024-09-21T06:13:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e02510a1c0 0x55e025524d00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0251761b0 0x55e025518690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 --2- 172.21.15.31:0/4056650395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e025518150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.711+0000 7f7694c91740 1 -- 172.21.15.31:0/4056650395 >> 172.21.15.31:0/4056650395 conn(0x55e02516a300 msgr2=0x55e025526180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.712+0000 7f7694c91740 1 -- 172.21.15.31:0/4056650395 shutdown_connections 2024-09-21T06:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.712+0000 7f7694c91740 1 -- 172.21.15.31:0/4056650395 wait complete. 2024-09-21T06:13:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.713+0000 7f7694c91740 1 -- start start 2024-09-21T06:13:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.713+0000 7f7694c91740 1 Processor -- start 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.714+0000 7f7694c91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e02510a1c0 0x55e02555bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.714+0000 7f7694c91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02555c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.714+0000 7f7694c91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e025556c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7694c91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f76881ac020 con 0x55e02510a1c0 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7686ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02555c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7686ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02555c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49188/0 (socket says 172.21.15.31:49188) 2024-09-21T06:13:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7686ffd640 1 -- 172.21.15.31:0/1506977819 learned_addr learned my addr 172.21.15.31:0/1506977819 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f76877fe640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e02510a1c0 0x55e02555bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f76881ad490 con 0x55e0251761b0 2024-09-21T06:13:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7687fff640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e025556c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f76881ac6f0 con 0x55e02514d2f0 2024-09-21T06:13:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.715+0000 7f7694c91740 10 client.0 _ll_register_callbacks cb 0x7f767c030450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f7686ffd640 1 -- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 msgr2=0x55e025556c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f7686ffd640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e025556c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f7686ffd640 1 -- 172.21.15.31:0/1506977819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e02510a1c0 msgr2=0x55e02555bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f7686ffd640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e02510a1c0 0x55e02555bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f7686ffd640 1 -- 172.21.15.31:0/1506977819 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02555c830 con 0x55e02514d2f0 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f7687fff640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e025556c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.716+0000 7f76877fe640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e02510a1c0 0x55e02555bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.717+0000 7f7686ffd640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02555c1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f767c0c8950 tx=0x7f767c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ea3f1bca04aaec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.717+0000 7f7666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.717+0000 7f7666ffd640 1 -- 172.21.15.31:0/1506977819 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c1062d0 con 0x55e02514d2f0 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.717+0000 7f7666ffd640 1 -- 172.21.15.31:0/1506977819 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f767c030b50 con 0x55e02514d2f0 2024-09-21T06:13:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.718+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7678029420 con 0x55e02514d2f0 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.719+0000 7f7666ffd640 1 -- 172.21.15.31:0/1506977819 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f767c0d5df0 con 0x55e02514d2f0 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.719+0000 7f7694c91740 10 client.25639 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.719+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e02555c830 con 0x55e02514d2f0 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.719+0000 7f7666ffd640 1 -- 172.21.15.31:0/1506977819 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767c030d50 con 0x55e02514d2f0 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7666ffd640 1 -- 172.21.15.31:0/1506977819 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f767c0cb0a0 con 0x55e02514d2f0 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7694c91740 10 client.25639 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7694c91740 10 client.25639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7694c91740 20 client.25639 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7694c91740 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0255c66c0 0x55e0255c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7694c91740 4 client.25639 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7678029420 con 0x55e0255c66c0 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.720+0000 7f76877fe640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0255c66c0 0x55e0255c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.721+0000 7f76877fe640 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0255c66c0 0x55e0255c8ae0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f5087ae42cc76d server_cookie=ac65fff607799d23 in_seq=0 out_seq=0 2024-09-21T06:13:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.721+0000 7f7666ffd640 10 client.25639 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7666ffd640 1 -- 172.21.15.31:0/1506977819 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7678029420 con 0x55e0255c66c0 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7666ffd640 10 client.25639 handle_command_reply: tid=0 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 client.25639 shutdown 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 msgr2=0x55e02555c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02555c1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f767c0c8950 tx=0x7f767c02a340 comp rx=0 tx=0).stop 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 shutdown_connections 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0255c66c0 0x55e0255c8ae0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0251761b0 0x55e025556c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e02514d2f0 0x55e02555c1c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 --2- 172.21.15.31:0/1506977819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e02510a1c0 0x55e02555bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 >> 172.21.15.31:0/1506977819 conn(0x55e02516a300 msgr2=0x55e025518150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7666ffd640 0 client.25639 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 shutdown_connections 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.741+0000 7f7694c91740 1 -- 172.21.15.31:0/1506977819 wait complete. 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.742+0000 7f7694c91740 20 client.25639 trim_cache size 0 max 16384 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:13:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:13:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:13:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f76700b67f0 msgr2=0x7f76700b8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f76700b67f0 0x7f76700b8d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7678005fd0 tx=0x7f7678009260 comp rx=0 tx=0).stop 2024-09-21T06:13:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 msgr2=0x7f768819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 0x7f768819ca00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f767c035040 tx=0x7f767c032d50 comp rx=0 tx=0).stop 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 shutdown_connections 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7688108a70 0x7f76881a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f76700b67f0 0x7f76700b8d00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7688108120 0x7f768819ca00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 --2- 172.21.15.31:0/463307190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76881027f0 0x7f768819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 >> 172.21.15.31:0/463307190 conn(0x7f76880fe230 msgr2=0x7f7688100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 shutdown_connections 2024-09-21T06:13:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:22.743+0000 7f768c9e8640 1 -- 172.21.15.31:0/463307190 wait complete. 2024-09-21T06:13:23.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:23 smithi110 ceph-mon[34550]: pgmap v373: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 36 MiB/s wr, 1.53k op/s 2024-09-21T06:13:23.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:23 smithi107 ceph-mon[34380]: pgmap v373: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 36 MiB/s wr, 1.53k op/s 2024-09-21T06:13:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:23 smithi031 ceph-mon[31237]: pgmap v373: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 36 MiB/s wr, 1.53k op/s 2024-09-21T06:13:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 105.0 s, 5083.9 tps, lat 19.659 ms stddev 21.316 2024-09-21T06:13:25.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:25 smithi110 ceph-mon[34550]: pgmap v374: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 9.6 MiB/s wr, 885 op/s 2024-09-21T06:13:25.538 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:25 smithi107 ceph-mon[34380]: pgmap v374: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 9.6 MiB/s wr, 885 op/s 2024-09-21T06:13:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:25 smithi031 ceph-mon[31237]: pgmap v374: 97 pgs: 97 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 9.6 MiB/s wr, 885 op/s 2024-09-21T06:13:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:27 smithi110 ceph-mon[34550]: pgmap v375: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-09-21T06:13:27.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:27 smithi107 ceph-mon[34380]: pgmap v375: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-09-21T06:13:27.625 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:27 smithi031 ceph-mon[31237]: pgmap v375: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 13 MiB/s wr, 1.19k op/s 2024-09-21T06:13:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:29 smithi110 ceph-mon[34550]: pgmap v376: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-21T06:13:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:29 smithi107 ceph-mon[34380]: pgmap v376: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-21T06:13:29.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:29 smithi031 ceph-mon[31237]: pgmap v376: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-21T06:13:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 110.0 s, 4079.7 tps, lat 24.545 ms stddev 25.742 2024-09-21T06:13:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:31 smithi110 ceph-mon[34550]: pgmap v377: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-21T06:13:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:31 smithi107 ceph-mon[34380]: pgmap v377: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-21T06:13:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:31 smithi031 ceph-mon[31237]: pgmap v377: 97 pgs: 97 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-21T06:13:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:33 smithi110 ceph-mon[34550]: pgmap v378: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-09-21T06:13:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:33 smithi107 ceph-mon[34380]: pgmap v378: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-09-21T06:13:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:33 smithi031 ceph-mon[31237]: pgmap v378: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-09-21T06:13:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 115.0 s, 5458.5 tps, lat 18.283 ms stddev 19.862 2024-09-21T06:13:35.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:35 smithi110 ceph-mon[34550]: pgmap v379: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 715 op/s 2024-09-21T06:13:35.538 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:35 smithi107 ceph-mon[34380]: pgmap v379: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 715 op/s 2024-09-21T06:13:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:35 smithi031 ceph-mon[31237]: pgmap v379: 97 pgs: 97 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 715 op/s 2024-09-21T06:13:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.892+0000 7fa504a4d640 1 -- 172.21.15.31:0/2075882026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa500110360 msgr2=0x7fa500116b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.892+0000 7fa504a4d640 1 --2- 172.21.15.31:0/2075882026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa500110360 0x7fa500116b90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009860 tx=0x7fa4f802a340 comp rx=0 tx=0).stop 2024-09-21T06:13:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 -- 172.21.15.31:0/2075882026 shutdown_connections 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 --2- 172.21.15.31:0/2075882026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa500110360 0x7fa500116b90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 --2- 172.21.15.31:0/2075882026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010fa10 0x7fa50010fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 --2- 172.21.15.31:0/2075882026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa50010a0e0 0x7fa50010a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 -- 172.21.15.31:0/2075882026 >> 172.21.15.31:0/2075882026 conn(0x7fa500079630 msgr2=0x7fa5001081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 -- 172.21.15.31:0/2075882026 shutdown_connections 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.893+0000 7fa504a4d640 1 -- 172.21.15.31:0/2075882026 wait complete. 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.894+0000 7fa504a4d640 1 Processor -- start 2024-09-21T06:13:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.894+0000 7fa504a4d640 1 -- start start 2024-09-21T06:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.895+0000 7fa504a4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 0x7fa5001a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.895+0000 7fa504a4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50010fa10 0x7fa5001a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.895+0000 7fa504a4d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa500110360 0x7fa5001a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.895+0000 7fa504a4d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa500119fd0 con 0x7fa50010a0e0 2024-09-21T06:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.895+0000 7fa504a4d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa500119e50 con 0x7fa500110360 2024-09-21T06:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.895+0000 7fa504a4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa50011a150 con 0x7fa50010fa10 2024-09-21T06:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 0x7fa5001a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 0x7fa5001a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40804/0 (socket says 172.21.15.31:40804) 2024-09-21T06:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 -- 172.21.15.31:0/1633222545 learned_addr learned my addr 172.21.15.31:0/1633222545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa500110360 0x7fa5001a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 -- 172.21.15.31:0/1633222545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50010fa10 msgr2=0x7fa5001a1010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50010fa10 0x7fa5001a1010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 -- 172.21.15.31:0/1633222545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa500110360 msgr2=0x7fa5001a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa500110360 0x7fa5001a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.896+0000 7fa4fe59c640 1 -- 172.21.15.31:0/1633222545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5001a69a0 con 0x7fa50010a0e0 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa500110360 0x7fa5001a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 0x7fa5001a0a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000eab0 tx=0x7fa4f000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa4f77fe640 1 -- 172.21.15.31:0/1633222545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000ce40 con 0x7fa50010a0e0 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5001a6c90 con 0x7fa50010a0e0 2024-09-21T06:13:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa4f77fe640 1 -- 172.21.15.31:0/1633222545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f0011920 con 0x7fa50010a0e0 2024-09-21T06:13:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa4f77fe640 1 -- 172.21.15.31:0/1633222545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0010a30 con 0x7fa50010a0e0 2024-09-21T06:13:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.897+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5001ad420 con 0x7fa50010a0e0 2024-09-21T06:13:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.902+0000 7fa4f77fe640 1 -- 172.21.15.31:0/1633222545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa4f0010bd0 con 0x7fa50010a0e0 2024-09-21T06:13:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.902+0000 7fa50b9ba740 1 Processor -- start 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 -- start start 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557ca9d436f0 con 0x557ca9a002f0 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ca9d32b20 con 0x557ca9a291b0 2024-09-21T06:13:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557ca9d36930 con 0x557ca9a084d0 2024-09-21T06:13:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa4fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa4fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49614/0 (socket says 172.21.15.31:49614) 2024-09-21T06:13:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa4fed9d640 1 -- 172.21.15.31:0/4009294130 learned_addr learned my addr 172.21.15.31:0/4009294130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.903+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4f77fe640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa4ec0b6680 0x7fa4ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4fdd9b640 1 -- 172.21.15.31:0/4009294130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 msgr2=0x557ca9dce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4fdd9b640 1 -- 172.21.15.31:0/4009294130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 msgr2=0x557ca9dceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4fdd9b640 1 -- 172.21.15.31:0/4009294130 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca9cf8530 con 0x557ca9a084d0 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dcf430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8005fd0 tx=0x7fa4e8009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7157169b53bfd7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4f4ff9640 1 -- 172.21.15.31:0/4009294130 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e800ec50 con 0x557ca9a084d0 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4f4ff9640 1 -- 172.21.15.31:0/4009294130 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e800edf0 con 0x557ca9a084d0 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.904+0000 7fa4f4ff9640 1 -- 172.21.15.31:0/4009294130 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8010430 con 0x557ca9a084d0 2024-09-21T06:13:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.905+0000 7fa4f77fe640 1 -- 172.21.15.31:0/1633222545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa4f00e7df0 con 0x7fa50010a0e0 2024-09-21T06:13:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.905+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa4ec0b6680 0x7fa4ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 -- 172.21.15.31:0/4009294130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 msgr2=0x557ca9dcf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dcf430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8005fd0 tx=0x7fa4e8009f40 comp rx=0 tx=0).stop 2024-09-21T06:13:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa4ec0b6680 0x7fa4ec0b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80095f0 tx=0x7fa4f8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 -- 172.21.15.31:0/4009294130 shutdown_connections 2024-09-21T06:13:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dcf430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/4009294130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 -- 172.21.15.31:0/4009294130 >> 172.21.15.31:0/4009294130 conn(0x557ca9a1d300 msgr2=0x557ca9dcc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.906+0000 7fa50b9ba740 1 -- 172.21.15.31:0/4009294130 shutdown_connections 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.907+0000 7fa50b9ba740 1 -- 172.21.15.31:0/4009294130 wait complete. 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.907+0000 7fa50b9ba740 1 -- start start 2024-09-21T06:13:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 Processor -- start 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dd9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dda480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dda9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557ca9c6aab0 con 0x557ca9a002f0 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ca9cd8730 con 0x557ca9a291b0 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557ca9ddd130 con 0x557ca9a084d0 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa50b9ba740 10 client.0 _ll_register_callbacks cb 0x557ca9d2c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dd9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa4fdd9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dda480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dd9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40824/0 (socket says 172.21.15.31:40824) 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa4fe59c640 1 -- 172.21.15.31:0/3305461815 learned_addr learned my addr 172.21.15.31:0/3305461815 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.908+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dda9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fdd9b640 1 -- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 msgr2=0x557ca9dda9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dda9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fdd9b640 1 -- 172.21.15.31:0/3305461815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 msgr2=0x557ca9dd9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dd9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fdd9b640 1 -- 172.21.15.31:0/3305461815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca9a94790 con 0x557ca9a084d0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dd9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dda480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8003b90 tx=0x7fa4e8005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd1562e3d9e1bc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/3305461815 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8025020 con 0x557ca9a084d0 2024-09-21T06:13:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/3305461815 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa4e8003800 con 0x557ca9a084d0 2024-09-21T06:13:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e800edf0 con 0x557ca9a084d0 2024-09-21T06:13:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/3305461815 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa4e8030050 con 0x557ca9a084d0 2024-09-21T06:13:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa50b9ba740 10 client.25648 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.909+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557ca9b0d620 con 0x557ca9a084d0 2024-09-21T06:13:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.910+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/3305461815 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e800edf0 con 0x557ca9a084d0 2024-09-21T06:13:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.911+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/3305461815 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa4e80046c0 con 0x557ca9a084d0 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.911+0000 7fa50b9ba740 10 client.25648 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.911+0000 7fa50b9ba740 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.911+0000 7fa50b9ba740 20 client.25648 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.911+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99e7d80 0x557ca9dfd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.912+0000 7fa50b9ba740 4 client.25648 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.912+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ca9d45640 con 0x557ca99e7d80 2024-09-21T06:13:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.912+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99e7d80 0x557ca9dfd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.914+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99e7d80 0x557ca9dfd400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3f7d45e776cfed server_cookie=a86bd582dd70392b in_seq=0 out_seq=0 2024-09-21T06:13:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.914+0000 7fa4e5ffb640 10 client.25648 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.914+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/3305461815 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ca9d45640 con 0x557ca99e7d80 2024-09-21T06:13:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.914+0000 7fa4e5ffb640 10 client.25648 handle_command_reply: tid=0 2024-09-21T06:13:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.914+0000 7fa50b9ba740 1 client.25648 shutdown 2024-09-21T06:13:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 msgr2=0x557ca9dda480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dda480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8003b90 tx=0x7fa4e8005e70 comp rx=0 tx=0).stop 2024-09-21T06:13:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 shutdown_connections 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99e7d80 0x557ca9dfd400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a291b0 0x557ca9dda9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a084d0 0x557ca9dda480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/3305461815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dd9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa4e5ffb640 0 client.25648 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 >> 172.21.15.31:0/3305461815 conn(0x557ca9a1d300 msgr2=0x557ca9dcf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 shutdown_connections 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.915+0000 7fa50b9ba740 1 -- 172.21.15.31:0/3305461815 wait complete. 2024-09-21T06:13:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.916+0000 7fa50b9ba740 20 client.25648 trim_cache size 0 max 16384 2024-09-21T06:13:35.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 Processor -- start 2024-09-21T06:13:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 -- start start 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dcc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9dd7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 0x557ca9dd7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa500073470 con 0x557ca9a002f0 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001addb0 con 0x557ca9a084d0 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.945+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa500073ae0 con 0x557ca9a291b0 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 0x557ca9dd7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 0x557ca9dd7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47156/0 (socket says 172.21.15.31:47156) 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 -- 172.21.15.31:0/2513417211 learned_addr learned my addr 172.21.15.31:0/2513417211 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9dd7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dcc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 -- 172.21.15.31:0/2513417211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 msgr2=0x557ca9dd7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9dd7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 -- 172.21.15.31:0/2513417211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 msgr2=0x557ca9dcc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dcc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.946+0000 7fa4fed9d640 1 -- 172.21.15.31:0/2513417211 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca9df8060 con 0x557ca9a291b0 2024-09-21T06:13:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.947+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 0x557ca9dd7650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8034000 tx=0x7fa4f803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3451fadc93121234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.947+0000 7fa4e7fff640 1 -- 172.21.15.31:0/2513417211 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8037040 con 0x557ca9a291b0 2024-09-21T06:13:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.948+0000 7fa4e7fff640 1 -- 172.21.15.31:0/2513417211 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f8042030 con 0x557ca9a291b0 2024-09-21T06:13:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.948+0000 7fa4e7fff640 1 -- 172.21.15.31:0/2513417211 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004c70 con 0x557ca9a291b0 2024-09-21T06:13:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 -- 172.21.15.31:0/2513417211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 msgr2=0x557ca9dd7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 0x557ca9dd7650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8034000 tx=0x7fa4f803e000 comp rx=0 tx=0).stop 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 -- 172.21.15.31:0/2513417211 shutdown_connections 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a291b0 0x557ca9dd7650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9dd7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/2513417211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a002f0 0x557ca9dcc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 -- 172.21.15.31:0/2513417211 >> 172.21.15.31:0/2513417211 conn(0x557ca9a1d300 msgr2=0x557ca9dcc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 -- 172.21.15.31:0/2513417211 shutdown_connections 2024-09-21T06:13:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.949+0000 7fa50b9ba740 1 -- 172.21.15.31:0/2513417211 wait complete. 2024-09-21T06:13:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.950+0000 7fa50b9ba740 1 -- start start 2024-09-21T06:13:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.950+0000 7fa50b9ba740 1 Processor -- start 2024-09-21T06:13:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 0x557ca9ddf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9ddf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a291b0 0x557ca9ddfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5000741b0 con 0x557ca9a291b0 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001ad700 con 0x557ca9a084d0 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa500074880 con 0x557ca9a002f0 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa50b9ba740 10 client.0 _ll_register_callbacks cb 0x7fa4e8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 0x557ca9ddf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 0x557ca9ddf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47166/0 (socket says 172.21.15.31:47166) 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 -- 172.21.15.31:0/795358179 learned_addr learned my addr 172.21.15.31:0/795358179 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9ddf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fed9d640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a291b0 0x557ca9ddfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 -- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 msgr2=0x557ca9ddf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9ddf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 -- 172.21.15.31:0/795358179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a291b0 msgr2=0x557ca9ddfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a291b0 0x557ca9ddfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 -- 172.21.15.31:0/795358179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca9e65eb0 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9ddf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4fe59c640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 0x557ca9ddf410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00e5630 tx=0x7fa4f00094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7220ccd7c6524df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.951+0000 7fa4e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/795358179 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00950d0 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/795358179 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa4f00b7210 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8042030 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/795358179 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa4f00ae380 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa50b9ba740 10 client.25654 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557ca9e671a0 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.952+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/795358179 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f8042030 con 0x557ca9a002f0 2024-09-21T06:13:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/795358179 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa4f000f040 con 0x557ca9a002f0 2024-09-21T06:13:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa50b9ba740 10 client.25654 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa50b9ba740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa50b9ba740 20 client.25654 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99bd1c0 0x557ca9e699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa50b9ba740 4 client.25654 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:13:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ca9d410f0 con 0x557ca99bd1c0 2024-09-21T06:13:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.953+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99bd1c0 0x557ca9e699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.954+0000 7fa4e5ffb640 10 client.25654 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.954+0000 7fa4fdd9b640 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99bd1c0 0x557ca9e699e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69e73e45f1e3610 server_cookie=cedec7f34cbacb93 in_seq=0 out_seq=0 2024-09-21T06:13:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.975+0000 7fa4e5ffb640 1 -- 172.21.15.31:0/795358179 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ca9d410f0 con 0x557ca99bd1c0 2024-09-21T06:13:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.975+0000 7fa4e5ffb640 10 client.25654 handle_command_reply: tid=0 2024-09-21T06:13:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.975+0000 7fa50b9ba740 1 client.25654 shutdown 2024-09-21T06:13:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 msgr2=0x557ca9ddf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 0x557ca9ddf410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00e5630 tx=0x7fa4f00094a0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 shutdown_connections 2024-09-21T06:13:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ca99bd1c0 0x557ca9e699e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ca9a291b0 0x557ca9ddfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ca9a084d0 0x557ca9ddf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 --2- 172.21.15.31:0/795358179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ca9a002f0 0x557ca9ddf410 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 >> 172.21.15.31:0/795358179 conn(0x557ca9a1d300 msgr2=0x557ca9dcaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.976+0000 7fa4e5ffb640 0 client.25654 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.977+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 shutdown_connections 2024-09-21T06:13:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.977+0000 7fa50b9ba740 1 -- 172.21.15.31:0/795358179 wait complete. 2024-09-21T06:13:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.977+0000 7fa50b9ba740 20 client.25654 trim_cache size 0 max 16384 2024-09-21T06:13:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:13:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:13:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:13:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa4ec0b6680 msgr2=0x7fa4ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa4ec0b6680 0x7fa4ec0b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80095f0 tx=0x7fa4f8005f90 comp rx=0 tx=0).stop 2024-09-21T06:13:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 msgr2=0x7fa5001a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 0x7fa5001a0a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000eab0 tx=0x7fa4f000ef70 comp rx=0 tx=0).stop 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 shutdown_connections 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa500110360 0x7fa5001a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa50010fa10 0x7fa5001a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa4ec0b6680 0x7fa4ec0b8b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 --2- 172.21.15.31:0/1633222545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa50010a0e0 0x7fa5001a0a70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 >> 172.21.15.31:0/1633222545 conn(0x7fa500079630 msgr2=0x7fa5001085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.978+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 shutdown_connections 2024-09-21T06:13:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:35.979+0000 7fa504a4d640 1 -- 172.21.15.31:0/1633222545 wait complete. 2024-09-21T06:13:37.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:37 smithi110 ceph-mon[34550]: pgmap v380: 97 pgs: 97 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-09-21T06:13:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:37 smithi107 ceph-mon[34380]: pgmap v380: 97 pgs: 97 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-09-21T06:13:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:37 smithi031 ceph-mon[31237]: pgmap v380: 97 pgs: 97 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-09-21T06:13:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:39 smithi110 ceph-mon[34550]: pgmap v381: 97 pgs: 97 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 802 op/s 2024-09-21T06:13:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:39 smithi107 ceph-mon[34380]: pgmap v381: 97 pgs: 97 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 802 op/s 2024-09-21T06:13:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:39 smithi031 ceph-mon[31237]: pgmap v381: 97 pgs: 97 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 802 op/s 2024-09-21T06:13:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 120.0 s, 4245.8 tps, lat 23.554 ms stddev 27.096 2024-09-21T06:13:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:41 smithi110 ceph-mon[34550]: pgmap v382: 97 pgs: 97 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.2 MiB/s wr, 714 op/s 2024-09-21T06:13:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:41 smithi107 ceph-mon[34380]: pgmap v382: 97 pgs: 97 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.2 MiB/s wr, 714 op/s 2024-09-21T06:13:41.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:41 smithi031 ceph-mon[31237]: pgmap v382: 97 pgs: 97 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.2 MiB/s wr, 714 op/s 2024-09-21T06:13:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:43 smithi110 ceph-mon[34550]: pgmap v383: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-21T06:13:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:43 smithi107 ceph-mon[34380]: pgmap v383: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-21T06:13:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:43 smithi031 ceph-mon[31237]: pgmap v383: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-21T06:13:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 125.0 s, 5559.6 tps, lat 17.978 ms stddev 20.049 2024-09-21T06:13:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:45 smithi110 ceph-mon[34550]: pgmap v384: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 836 op/s 2024-09-21T06:13:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:45 smithi107 ceph-mon[34380]: pgmap v384: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 836 op/s 2024-09-21T06:13:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:45 smithi031 ceph-mon[31237]: pgmap v384: 97 pgs: 97 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 836 op/s 2024-09-21T06:13:46.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:46 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:46.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:46 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:46.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:46 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:13:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:47 smithi110 ceph-mon[34550]: pgmap v385: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-09-21T06:13:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:47.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:13:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:47 smithi107 ceph-mon[34380]: pgmap v385: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-09-21T06:13:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:13:47.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:47 smithi031 ceph-mon[31237]: pgmap v385: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-09-21T06:13:47.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:13:47.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:13:47.637 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:13:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 -- 172.21.15.31:0/201970145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4109d80 msgr2=0x7efec41105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 --2- 172.21.15.31:0/201970145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4109d80 0x7efec41105b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efebc009be0 tx=0x7efebc02a6d0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 -- 172.21.15.31:0/201970145 shutdown_connections 2024-09-21T06:13:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 --2- 172.21.15.31:0/201970145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4109d80 0x7efec41105b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 --2- 172.21.15.31:0/201970145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4109430 0x7efec4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 --2- 172.21.15.31:0/201970145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4102060 0x7efec4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 -- 172.21.15.31:0/201970145 >> 172.21.15.31:0/201970145 conn(0x7efec4079e90 msgr2=0x7efec407a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 -- 172.21.15.31:0/201970145 shutdown_connections 2024-09-21T06:13:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.131+0000 7efec93ba640 1 -- 172.21.15.31:0/201970145 wait complete. 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.132+0000 7efec93ba640 1 Processor -- start 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 -- start start 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 0x7efec419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4109430 0x7efec419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4109d80 0x7efec419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efec4113a40 con 0x7efec4109d80 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efec41138c0 con 0x7efec4109430 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec93ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efec4113bc0 con 0x7efec4102060 2024-09-21T06:13:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 0x7efec419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 0x7efec419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41484/0 (socket says 172.21.15.31:41484) 2024-09-21T06:13:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec2ffd640 1 -- 172.21.15.31:0/1521440562 learned_addr learned my addr 172.21.15.31:0/1521440562 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec27fc640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4109430 0x7efec419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.133+0000 7efec37fe640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4109d80 0x7efec419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec2ffd640 1 -- 172.21.15.31:0/1521440562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4109430 msgr2=0x7efec419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4109430 0x7efec419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec2ffd640 1 -- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4109d80 msgr2=0x7efec419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4109d80 0x7efec419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec2ffd640 1 -- 172.21.15.31:0/1521440562 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec41a0390 con 0x7efec4102060 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 0x7efec419a460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efeb400e9c0 tx=0x7efeb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.134+0000 7efec27fc640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4109430 0x7efec419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.135+0000 7efeabfff640 1 -- 172.21.15.31:0/1521440562 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb400ccc0 con 0x7efec4102060 2024-09-21T06:13:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.135+0000 7efeabfff640 1 -- 172.21.15.31:0/1521440562 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb40054d0 con 0x7efec4102060 2024-09-21T06:13:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.135+0000 7efeabfff640 1 -- 172.21.15.31:0/1521440562 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4010430 con 0x7efec4102060 2024-09-21T06:13:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.136+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec41a0680 con 0x7efec4102060 2024-09-21T06:13:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.136+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a6e10 con 0x7efec4102060 2024-09-21T06:13:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 Processor -- start 2024-09-21T06:13:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efeabfff640 1 -- 172.21.15.31:0/1521440562 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efeb4005020 con 0x7efec4102060 2024-09-21T06:13:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 -- start start 2024-09-21T06:13:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4691b0 0x55f1fd812dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4402f0 0x55f1fd813300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd813840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f1fd7836f0 con 0x55f1fd4484d0 2024-09-21T06:13:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1fd772b20 con 0x55f1fd4402f0 2024-09-21T06:13:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.141+0000 7efed0405740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1fd776930 con 0x55f1fd4691b0 2024-09-21T06:13:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd813840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4691b0 0x55f1fd812dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4402f0 0x55f1fd813300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4691b0 0x55f1fd812dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41498/0 (socket says 172.21.15.31:41498) 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd813840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56972/0 (socket says 172.21.15.31:56972) 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec37fe640 1 -- 172.21.15.31:0/1730072503 learned_addr learned my addr 172.21.15.31:0/1730072503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec2ffd640 1 -- 172.21.15.31:0/1730072503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4691b0 msgr2=0x55f1fd812dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4691b0 0x55f1fd812dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec2ffd640 1 -- 172.21.15.31:0/1730072503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 msgr2=0x55f1fd813840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd813840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.142+0000 7efec2ffd640 1 -- 172.21.15.31:0/1730072503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1fd738530 con 0x55f1fd4402f0 2024-09-21T06:13:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.143+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4402f0 0x55f1fd813300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efeb40e5010 tx=0x7efeb40edef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9d1f1042517ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.144+0000 7efeabfff640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efeac0b67d0 0x7efeac0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.144+0000 7efea97fa640 1 -- 172.21.15.31:0/1730072503 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb40f6070 con 0x55f1fd4402f0 2024-09-21T06:13:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.144+0000 7efea97fa640 1 -- 172.21.15.31:0/1730072503 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb40ad220 con 0x55f1fd4402f0 2024-09-21T06:13:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.144+0000 7efea97fa640 1 -- 172.21.15.31:0/1730072503 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb40ade10 con 0x55f1fd4402f0 2024-09-21T06:13:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.144+0000 7efec37fe640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efeac0b67d0 0x7efeac0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.144+0000 7efeabfff640 1 -- 172.21.15.31:0/1521440562 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efeb40e78e0 con 0x7efec4102060 2024-09-21T06:13:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 -- 172.21.15.31:0/1730072503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4402f0 msgr2=0x55f1fd813300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efec37fe640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efeac0b67d0 0x7efeac0b8ce0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efebc006010 tx=0x7efebc004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4402f0 0x55f1fd813300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efeb40e5010 tx=0x7efeb40edef0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 -- 172.21.15.31:0/1730072503 shutdown_connections 2024-09-21T06:13:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd813840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4402f0 0x55f1fd813300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 --2- 172.21.15.31:0/1730072503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4691b0 0x55f1fd812dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 -- 172.21.15.31:0/1730072503 >> 172.21.15.31:0/1730072503 conn(0x55f1fd45d300 msgr2=0x55f1fd810cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 -- 172.21.15.31:0/1730072503 shutdown_connections 2024-09-21T06:13:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.145+0000 7efed0405740 1 -- 172.21.15.31:0/1730072503 wait complete. 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 -- start start 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 Processor -- start 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd80d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd80d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd82ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f1fd6aaab0 con 0x55f1fd4484d0 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1fd718730 con 0x55f1fd4691b0 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f1fd825980 con 0x55f1fd4402f0 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.146+0000 7efed0405740 10 client.0 _ll_register_callbacks cb 0x55f1fd76c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd82ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd82ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35130/0 (socket says 172.21.15.31:35130) 2024-09-21T06:13:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 -- 172.21.15.31:0/1157218697 learned_addr learned my addr 172.21.15.31:0/1157218697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd80d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 -- 172.21.15.31:0/1157218697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 msgr2=0x55f1fd80d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd80d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 -- 172.21.15.31:0/1157218697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 msgr2=0x55f1fd80d950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd80d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.147+0000 7efec37fe640 1 -- 172.21.15.31:0/1157218697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1fd4d4790 con 0x55f1fd4691b0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efec37fe640 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd82ec70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efebc02abf0 tx=0x7efebc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1671b61aad111c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efea6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efea6ff5640 1 -- 172.21.15.31:0/1157218697 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc04f070 con 0x55f1fd4691b0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efea6ff5640 1 -- 172.21.15.31:0/1157218697 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efebc030e40 con 0x55f1fd4691b0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb40ad220 con 0x55f1fd4691b0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efea6ff5640 1 -- 172.21.15.31:0/1157218697 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efebc05b050 con 0x55f1fd4691b0 2024-09-21T06:13:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efed0405740 10 client.25637 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.148+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f1fd54d620 con 0x55f1fd4691b0 2024-09-21T06:13:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.149+0000 7efea6ff5640 1 -- 172.21.15.31:0/1157218697 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb40ad220 con 0x55f1fd4691b0 2024-09-21T06:13:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efea6ff5640 1 -- 172.21.15.31:0/1157218697 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efebc004040 con 0x55f1fd4691b0 2024-09-21T06:13:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efed0405740 10 client.25637 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efed0405740 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efed0405740 20 client.25637 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efed0405740 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd427d80 0x55f1fd834db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efed0405740 4 client.25637 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1fd785640 con 0x55f1fd427d80 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.150+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd427d80 0x55f1fd834db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.151+0000 7efea6ff5640 10 client.25637 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.151+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd427d80 0x55f1fd834db0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a230057274ad9c81 server_cookie=825e77bc0696635 in_seq=0 out_seq=0 2024-09-21T06:13:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.152+0000 7efea6ff5640 1 -- 172.21.15.31:0/1157218697 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1fd785640 con 0x55f1fd427d80 2024-09-21T06:13:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.152+0000 7efea6ff5640 10 client.25637 handle_command_reply: tid=0 2024-09-21T06:13:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.152+0000 7efed0405740 1 client.25637 shutdown 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 msgr2=0x55f1fd82ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd82ec70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efebc02abf0 tx=0x7efebc009500 comp rx=0 tx=0).stop 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 shutdown_connections 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd427d80 0x55f1fd834db0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd82ec70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4484d0 0x55f1fd80d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 --2- 172.21.15.31:0/1157218697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd80d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 >> 172.21.15.31:0/1157218697 conn(0x55f1fd45d300 msgr2=0x55f1fd815f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efea6ff5640 0 client.25637 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 shutdown_connections 2024-09-21T06:13:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 1 -- 172.21.15.31:0/1157218697 wait complete. 2024-09-21T06:13:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.153+0000 7efed0405740 20 client.25637 trim_cache size 0 max 16384 2024-09-21T06:13:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.177+0000 7efed0405740 1 Processor -- start 2024-09-21T06:13:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.177+0000 7efed0405740 1 -- start start 2024-09-21T06:13:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efed0405740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd823560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efed0405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 0x55f1fd82eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efed0405740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4691b0 0x55f1fd825060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efed0405740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efec41ab4f0 con 0x55f1fd4691b0 2024-09-21T06:13:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efed0405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efec41add70 con 0x55f1fd4484d0 2024-09-21T06:13:49.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efed0405740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efec41abb60 con 0x55f1fd4402f0 2024-09-21T06:13:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efec27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 0x55f1fd82eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efec2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd823560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efec27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 0x55f1fd82eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35144/0 (socket says 172.21.15.31:35144) 2024-09-21T06:13:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.178+0000 7efec27fc640 1 -- 172.21.15.31:0/230992528 learned_addr learned my addr 172.21.15.31:0/230992528 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:49.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec37fe640 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4691b0 0x55f1fd825060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec27fc640 1 -- 172.21.15.31:0/230992528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 msgr2=0x55f1fd823560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec27fc640 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd823560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec27fc640 1 -- 172.21.15.31:0/230992528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4691b0 msgr2=0x55f1fd825060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec27fc640 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4691b0 0x55f1fd825060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec27fc640 1 -- 172.21.15.31:0/230992528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1fd80e6d0 con 0x55f1fd4484d0 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec2ffd640 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd823560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efec27fc640 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 0x55f1fd82eb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efeb8012530 tx=0x7efeb80129f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=552bb05cf9e1efd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.179+0000 7efea8ff9640 1 -- 172.21.15.31:0/230992528 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80121d0 con 0x55f1fd4484d0 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.180+0000 7efea8ff9640 1 -- 172.21.15.31:0/230992528 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb80035f0 con 0x55f1fd4484d0 2024-09-21T06:13:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.180+0000 7efea8ff9640 1 -- 172.21.15.31:0/230992528 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb800f6e0 con 0x55f1fd4484d0 2024-09-21T06:13:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 -- 172.21.15.31:0/230992528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 msgr2=0x55f1fd82eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 0x55f1fd82eb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efeb8012530 tx=0x7efeb80129f0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 -- 172.21.15.31:0/230992528 shutdown_connections 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4691b0 0x55f1fd825060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4484d0 0x55f1fd82eb70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 --2- 172.21.15.31:0/230992528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4402f0 0x55f1fd823560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 -- 172.21.15.31:0/230992528 >> 172.21.15.31:0/230992528 conn(0x55f1fd45d300 msgr2=0x55f1fd80bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 -- 172.21.15.31:0/230992528 shutdown_connections 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 -- 172.21.15.31:0/230992528 wait complete. 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.181+0000 7efed0405740 1 -- start start 2024-09-21T06:13:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.182+0000 7efed0405740 1 Processor -- start 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.182+0000 7efed0405740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4402f0 0x55f1fd82abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.182+0000 7efed0405740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4484d0 0x55f1fd82b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.182+0000 7efed0405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd839640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4484d0 0x55f1fd82b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec27fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4484d0 0x55f1fd82b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41522/0 (socket says 172.21.15.31:41522) 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec27fc640 1 -- 172.21.15.31:0/1645098381 learned_addr learned my addr 172.21.15.31:0/1645098381 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec37fe640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd839640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efed0405740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efec41ac230 con 0x55f1fd4402f0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efec41ad6a0 con 0x55f1fd4691b0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efec41ac900 con 0x55f1fd4484d0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efed0405740 10 client.0 _ll_register_callbacks cb 0x7efeb4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4402f0 0x55f1fd82abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec37fe640 1 -- 172.21.15.31:0/1645098381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4484d0 msgr2=0x55f1fd82b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec37fe640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4484d0 0x55f1fd82b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec37fe640 1 -- 172.21.15.31:0/1645098381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4402f0 msgr2=0x55f1fd82abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec37fe640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4402f0 0x55f1fd82abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.183+0000 7efec37fe640 1 -- 172.21.15.31:0/1645098381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1fd82b780 con 0x55f1fd4691b0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4402f0 0x55f1fd82abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efec37fe640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd839640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efebc030660 tx=0x7efebc009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6fd1946e91c7302 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efea6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efea6ff5640 1 -- 172.21.15.31:0/1645098381 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc04f040 con 0x55f1fd4691b0 2024-09-21T06:13:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb80035f0 con 0x55f1fd4691b0 2024-09-21T06:13:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efea6ff5640 1 -- 172.21.15.31:0/1645098381 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efebc030e40 con 0x55f1fd4691b0 2024-09-21T06:13:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efea6ff5640 1 -- 172.21.15.31:0/1645098381 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efebc06d050 con 0x55f1fd4691b0 2024-09-21T06:13:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efed0405740 10 client.25643 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.184+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f1fd89eb60 con 0x55f1fd4691b0 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efea6ff5640 1 -- 172.21.15.31:0/1645098381 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efeb80035f0 con 0x55f1fd4691b0 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efea6ff5640 1 -- 172.21.15.31:0/1645098381 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efebc031440 con 0x55f1fd4691b0 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efed0405740 10 client.25643 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efed0405740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efed0405740 20 client.25643 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efed0405740 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd3fd1c0 0x55f1fd8a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efed0405740 4 client.25643 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1fd80fd80 con 0x55f1fd3fd1c0 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.185+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd3fd1c0 0x55f1fd8a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.186+0000 7efea6ff5640 10 client.25643 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.186+0000 7efec2ffd640 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd3fd1c0 0x55f1fd8a12e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c671f0d32b2ef209 server_cookie=1f74a6e9c9225f8f in_seq=0 out_seq=0 2024-09-21T06:13:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efea6ff5640 1 -- 172.21.15.31:0/1645098381 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1fd80fd80 con 0x55f1fd3fd1c0 2024-09-21T06:13:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efea6ff5640 10 client.25643 handle_command_reply: tid=0 2024-09-21T06:13:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 client.25643 shutdown 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 msgr2=0x55f1fd839640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd839640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efebc030660 tx=0x7efebc009ba0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 shutdown_connections 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f1fd3fd1c0 0x55f1fd8a12e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1fd4691b0 0x55f1fd839640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1fd4484d0 0x55f1fd82b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 --2- 172.21.15.31:0/1645098381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f1fd4402f0 0x55f1fd82abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efea6ff5640 0 client.25643 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.213+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 >> 172.21.15.31:0/1645098381 conn(0x55f1fd45d300 msgr2=0x55f1fd80bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 shutdown_connections 2024-09-21T06:13:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efed0405740 1 -- 172.21.15.31:0/1645098381 wait complete. 2024-09-21T06:13:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efed0405740 20 client.25643 trim_cache size 0 max 16384 2024-09-21T06:13:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:13:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:13:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:13:49.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efeac0b67d0 msgr2=0x7efeac0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efec93ba640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efeac0b67d0 0x7efeac0b8ce0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efebc006010 tx=0x7efebc004a40 comp rx=0 tx=0).stop 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 msgr2=0x7efec419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.214+0000 7efec93ba640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 0x7efec419a460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efeb400e9c0 tx=0x7efeb400ee80 comp rx=0 tx=0).stop 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 shutdown_connections 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efeac0b67d0 0x7efeac0b8ce0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec4109d80 0x7efec419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec4109430 0x7efec419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 --2- 172.21.15.31:0/1521440562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efec4102060 0x7efec419a460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 >> 172.21.15.31:0/1521440562 conn(0x7efec4079e90 msgr2=0x7efec4105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 shutdown_connections 2024-09-21T06:13:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:13:49.215+0000 7efec93ba640 1 -- 172.21.15.31:0/1521440562 wait complete. 2024-09-21T06:13:49.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:49 smithi110 ceph-mon[34550]: pgmap v386: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 777 op/s 2024-09-21T06:13:49.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:49 smithi107 ceph-mon[34380]: pgmap v386: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 777 op/s 2024-09-21T06:13:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:49 smithi031 ceph-mon[31237]: pgmap v386: 97 pgs: 97 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 777 op/s 2024-09-21T06:13:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 130.0 s, 3594.4 tps, lat 27.840 ms stddev 31.841 2024-09-21T06:13:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:51 smithi110 ceph-mon[34550]: pgmap v387: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-09-21T06:13:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:51 smithi107 ceph-mon[34380]: pgmap v387: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-09-21T06:13:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:51 smithi031 ceph-mon[31237]: pgmap v387: 97 pgs: 97 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-09-21T06:13:52.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:13:52.519 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.072+0000 7fe8edea4640 1 -- 172.21.15.31:0/1963208096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 msgr2=0x7fe8e8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.072+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1963208096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 0x7fe8e8102e70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009860 tx=0x7fe8d802a360 comp rx=0 tx=0).stop 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.072+0000 7fe8edea4640 1 -- 172.21.15.31:0/1963208096 shutdown_connections 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.072+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1963208096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e8108cf0 0x7fe8e806ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.072+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1963208096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e81083a0 0x7fe8e81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.072+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1963208096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 0x7fe8e8102e70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.073+0000 7fe8edea4640 1 -- 172.21.15.31:0/1963208096 >> 172.21.15.31:0/1963208096 conn(0x7fe8e8079730 msgr2=0x7fe8e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.075+0000 7fe8edea4640 1 -- 172.21.15.31:0/1963208096 shutdown_connections 2024-09-21T06:13:53.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.075+0000 7fe8edea4640 1 -- 172.21.15.31:0/1963208096 wait complete. 2024-09-21T06:13:53.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 Processor -- start 2024-09-21T06:13:53.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 -- start start 2024-09-21T06:13:53.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 0x7fe8e806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e81083a0 0x7fe8e806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e8108cf0 0x7fe8e8072930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8115cc0 con 0x7fe8e81083a0 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8115b40 con 0x7fe8e8102a60 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.076+0000 7fe8edea4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8115e40 con 0x7fe8e8108cf0 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.077+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e8108cf0 0x7fe8e8072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.077+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e81083a0 0x7fe8e806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.077+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e8108cf0 0x7fe8e8072930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41534/0 (socket says 172.21.15.31:41534) 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.077+0000 7fe8e7fff640 1 -- 172.21.15.31:0/1293470588 learned_addr learned my addr 172.21.15.31:0/1293470588 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:53.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.077+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 0x7fe8e806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e6ffd640 1 -- 172.21.15.31:0/1293470588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e8108cf0 msgr2=0x7fe8e8072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e8108cf0 0x7fe8e8072930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e6ffd640 1 -- 172.21.15.31:0/1293470588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 msgr2=0x7fe8e806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 0x7fe8e806d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e6ffd640 1 -- 172.21.15.31:0/1293470588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e8073060 con 0x7fe8e81083a0 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e81083a0 0x7fe8e806d670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009fd0 tx=0x7fe8dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.079+0000 7fe8e4ff9640 1 -- 172.21.15.31:0/1293470588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc015020 con 0x7fe8e81083a0 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.080+0000 7fe8e4ff9640 1 -- 172.21.15.31:0/1293470588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc011ce0 con 0x7fe8e81083a0 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.080+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e80732f0 con 0x7fe8e81083a0 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.080+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e8073aa0 con 0x7fe8e81083a0 2024-09-21T06:13:53.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.080+0000 7fe8e4ff9640 1 -- 172.21.15.31:0/1293470588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc010e20 con 0x7fe8e81083a0 2024-09-21T06:13:53.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.084+0000 7fe8f4418740 1 Processor -- start 2024-09-21T06:13:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.084+0000 7fe8f4418740 1 -- start start 2024-09-21T06:13:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.085+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dacde2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.085+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacde2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.085+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dacde30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.085+0000 7fe8f4418740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557dacd58e50 con 0x557daca1a390 2024-09-21T06:13:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.085+0000 7fe8f4418740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557dacd48280 con 0x557daca3b070 2024-09-21T06:13:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.085+0000 7fe8f4418740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557dacd4c090 con 0x557daca121b0 2024-09-21T06:13:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.086+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dacde30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.086+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacde2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.086+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacde2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41544/0 (socket says 172.21.15.31:41544) 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.086+0000 7fe8e77fe640 1 -- 172.21.15.31:0/2770094538 learned_addr learned my addr 172.21.15.31:0/2770094538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.086+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dacde2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.086+0000 7fe8e77fe640 1 -- 172.21.15.31:0/2770094538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 msgr2=0x557dacde2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.087+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dacde2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.087+0000 7fe8e4ff9640 1 -- 172.21.15.31:0/1293470588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe8dc018020 con 0x7fe8e81083a0 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.087+0000 7fe8e77fe640 1 -- 172.21.15.31:0/2770094538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 msgr2=0x557dacde30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.087+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dacde30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.087+0000 7fe8e77fe640 1 -- 172.21.15.31:0/2770094538 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dacaa6650 con 0x557daca121b0 2024-09-21T06:13:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.087+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dacde2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.088+0000 7fe8e4ff9640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8d00b6780 0x7fe8d00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.088+0000 7fe8e4ff9640 1 -- 172.21.15.31:0/1293470588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe8dc0e6be0 con 0x7fe8e81083a0 2024-09-21T06:13:53.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.089+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacde2b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802a950 tx=0x7fe8d8005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94cec630d365c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.089+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8d00b6780 0x7fe8d00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.090+0000 7fe8cdffb640 1 -- 172.21.15.31:0/2770094538 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8043020 con 0x557daca121b0 2024-09-21T06:13:53.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.090+0000 7fe8cdffb640 1 -- 172.21.15.31:0/2770094538 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d8038040 con 0x557daca121b0 2024-09-21T06:13:53.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.090+0000 7fe8cdffb640 1 -- 172.21.15.31:0/2770094538 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8037040 con 0x557daca121b0 2024-09-21T06:13:53.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.091+0000 7fe8f4418740 1 -- 172.21.15.31:0/2770094538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 msgr2=0x557dacde2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.091+0000 7fe8f4418740 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacde2b90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802a950 tx=0x7fe8d8005da0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8d00b6780 0x7fe8d00b8c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002f580 tx=0x7fe8e00095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 -- 172.21.15.31:0/2770094538 shutdown_connections 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dacde30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacde2b90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 --2- 172.21.15.31:0/2770094538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dacde2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 -- 172.21.15.31:0/2770094538 >> 172.21.15.31:0/2770094538 conn(0x557daca2f460 msgr2=0x557dacde0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 -- 172.21.15.31:0/2770094538 shutdown_connections 2024-09-21T06:13:53.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 -- 172.21.15.31:0/2770094538 wait complete. 2024-09-21T06:13:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.092+0000 7fe8f4418740 1 -- start start 2024-09-21T06:13:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.093+0000 7fe8f4418740 1 Processor -- start 2024-09-21T06:13:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.093+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacdedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca3b070 0x557dacdee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8f4418740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557dacba9700 con 0x557daca3b070 2024-09-21T06:13:53.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8f4418740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557dacce3110 con 0x557daca1a390 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8f4418740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557dace01de0 con 0x557daca121b0 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8f4418740 10 client.0 _ll_register_callbacks cb 0x557daccfc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacdedcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacdedcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41550/0 (socket says 172.21.15.31:41550) 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e77fe640 1 -- 172.21.15.31:0/3499057774 learned_addr learned my addr 172.21.15.31:0/3499057774 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e77fe640 1 -- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 msgr2=0x557dacdee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.095+0000 7fe8e77fe640 1 -- 172.21.15.31:0/3499057774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca3b070 msgr2=0x557dacdee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.094+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca3b070 0x557dacdee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.095+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca3b070 0x557dacdee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.095+0000 7fe8e77fe640 1 -- 172.21.15.31:0/3499057774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dacd10c40 con 0x557daca121b0 2024-09-21T06:13:53.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.095+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca3b070 0x557dacdee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:13:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.095+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d8038040 con 0x557daca121b0 2024-09-21T06:13:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.096+0000 7fe8c6ffd640 10 client.25681 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.096+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacdedcf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8005eb0 tx=0x7fe8d8030fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1becdca2c75f8e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.096+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/3499057774 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8037040 con 0x557daca121b0 2024-09-21T06:13:53.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.097+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/3499057774 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe8d8004030 con 0x557daca121b0 2024-09-21T06:13:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.097+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/3499057774 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe8d8037040 con 0x557daca121b0 2024-09-21T06:13:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.098+0000 7fe8f4418740 10 client.25681 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.098+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x557dace0d9c0 con 0x557daca121b0 2024-09-21T06:13:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.098+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/3499057774 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe8d8004230 con 0x557daca121b0 2024-09-21T06:13:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8f4418740 10 client.25681 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8f4418740 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8f4418740 20 client.25681 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9f99f0 0x557dace10d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8f4418740 4 client.25681 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dacd5ada0 con 0x557dac9f99f0 2024-09-21T06:13:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.099+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9f99f0 0x557dace10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.100+0000 7fe8c6ffd640 10 client.25681 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.100+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9f99f0 0x557dace10d20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9029ebfb201a300 server_cookie=6a3314ff568443e3 in_seq=0 out_seq=0 2024-09-21T06:13:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.100+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/3499057774 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557dacd5ada0 con 0x557dac9f99f0 2024-09-21T06:13:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8c6ffd640 10 client.25681 handle_command_reply: tid=0 2024-09-21T06:13:53.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 client.25681 shutdown 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 msgr2=0x557dacdedcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacdedcf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8005eb0 tx=0x7fe8d8030fb0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 shutdown_connections 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9f99f0 0x557dace10d20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca3b070 0x557dacdee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3499057774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dacdedcf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 >> 172.21.15.31:0/3499057774 conn(0x557daca2f460 msgr2=0x557dacde57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.101+0000 7fe8c6ffd640 0 client.25681 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.102+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 shutdown_connections 2024-09-21T06:13:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.102+0000 7fe8f4418740 1 -- 172.21.15.31:0/3499057774 wait complete. 2024-09-21T06:13:53.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.102+0000 7fe8f4418740 20 client.25681 trim_cache size 0 max 16384 2024-09-21T06:13:53.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.139+0000 7fe8f4418740 1 Processor -- start 2024-09-21T06:13:53.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.141+0000 7fe8f4418740 1 -- start start 2024-09-21T06:13:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca121b0 0x557dace0a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdeaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca3b070 0x557dacdeb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8f4418740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d400c720 con 0x557daca121b0 2024-09-21T06:13:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8f4418740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d400f000 con 0x557daca1a390 2024-09-21T06:13:53.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8f4418740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d400cdf0 con 0x557daca3b070 2024-09-21T06:13:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdeaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca121b0 0x557dace0a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca121b0 0x557dace0a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57024/0 (socket says 172.21.15.31:57024) 2024-09-21T06:13:53.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.142+0000 7fe8e77fe640 1 -- 172.21.15.31:0/3082690859 learned_addr learned my addr 172.21.15.31:0/3082690859 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.143+0000 7fe8e6ffd640 1 -- 172.21.15.31:0/3082690859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca3b070 msgr2=0x557dacdeb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.143+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca3b070 0x557dacdeb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.143+0000 7fe8e6ffd640 1 -- 172.21.15.31:0/3082690859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca121b0 msgr2=0x557dace0a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.143+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca121b0 0x557dace0a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.143+0000 7fe8e6ffd640 1 -- 172.21.15.31:0/3082690859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dacde4940 con 0x557daca1a390 2024-09-21T06:13:53.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.143+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdeaee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0e4310 tx=0x7fe8dc010980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce2d8ffaea18f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.144+0000 7fe8cd7fa640 1 -- 172.21.15.31:0/3082690859 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0a8180 con 0x557daca1a390 2024-09-21T06:13:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.144+0000 7fe8cd7fa640 1 -- 172.21.15.31:0/3082690859 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc0ac520 con 0x557daca1a390 2024-09-21T06:13:53.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.144+0000 7fe8cd7fa640 1 -- 172.21.15.31:0/3082690859 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0e4590 con 0x557daca1a390 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 -- 172.21.15.31:0/3082690859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 msgr2=0x557dacdeaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdeaee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0e4310 tx=0x7fe8dc010980 comp rx=0 tx=0).stop 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 -- 172.21.15.31:0/3082690859 shutdown_connections 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca3b070 0x557dacdeb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca1a390 0x557dacdeaee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 --2- 172.21.15.31:0/3082690859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca121b0 0x557dace0a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.145+0000 7fe8f4418740 1 -- 172.21.15.31:0/3082690859 >> 172.21.15.31:0/3082690859 conn(0x557daca2f460 msgr2=0x557dace0a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.146+0000 7fe8f4418740 1 -- 172.21.15.31:0/3082690859 shutdown_connections 2024-09-21T06:13:53.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.146+0000 7fe8f4418740 1 -- 172.21.15.31:0/3082690859 wait complete. 2024-09-21T06:13:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.146+0000 7fe8f4418740 1 -- start start 2024-09-21T06:13:53.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.147+0000 7fe8f4418740 1 Processor -- start 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.147+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dace144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8f4418740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dace11980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dace144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dace144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41574/0 (socket says 172.21.15.31:41574) 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8e77fe640 1 -- 172.21.15.31:0/33943583 learned_addr learned my addr 172.21.15.31:0/33943583 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dace11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d400d4c0 con 0x557daca1a390 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.148+0000 7fe8e7fff640 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dace11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d400e930 con 0x557daca3b070 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d400db90 con 0x557daca121b0 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8f4418740 10 client.0 _ll_register_callbacks cb 0x7fe8d80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8e77fe640 1 -- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 msgr2=0x557dace11ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dace11ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8e77fe640 1 -- 172.21.15.31:0/33943583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 msgr2=0x557dace11980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dace11980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.149+0000 7fe8e77fe640 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dace12600 con 0x557daca121b0 2024-09-21T06:13:53.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.150+0000 7fe8e77fe640 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dace144b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8047420 tx=0x7fe8d80056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e6af88ac82e334 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.152+0000 7fe8c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.152+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/33943583 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8043040 con 0x557daca121b0 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.152+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/33943583 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe8d803ed10 con 0x557daca121b0 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.153+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc0ac520 con 0x557daca121b0 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.153+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/33943583 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe8d8048410 con 0x557daca121b0 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.153+0000 7fe8f4418740 10 client.25684 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.153+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557dace7b320 con 0x557daca121b0 2024-09-21T06:13:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.154+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/33943583 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8dc0ac520 con 0x557daca121b0 2024-09-21T06:13:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.154+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/33943583 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe8d8004230 con 0x557daca121b0 2024-09-21T06:13:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8f4418740 10 client.25684 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8f4418740 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8f4418740 20 client.25684 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9ceda0 0x557dace7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8f4418740 4 client.25684 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:13:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557dacd56850 con 0x557dac9ceda0 2024-09-21T06:13:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.155+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9ceda0 0x557dace7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.157+0000 7fe8e6ffd640 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9ceda0 0x557dace7da20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b81d2c392612ff2b server_cookie=3bbb04767758ae52 in_seq=0 out_seq=0 2024-09-21T06:13:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.157+0000 7fe8c6ffd640 10 client.25684 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.157+0000 7fe8c6ffd640 1 -- 172.21.15.31:0/33943583 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x557dacd56850 con 0x557dac9ceda0 2024-09-21T06:13:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.157+0000 7fe8c6ffd640 10 client.25684 handle_command_reply: tid=0 2024-09-21T06:13:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.157+0000 7fe8f4418740 1 client.25684 shutdown 2024-09-21T06:13:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 msgr2=0x557dace144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dace144b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8047420 tx=0x7fe8d80056d0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 shutdown_connections 2024-09-21T06:13:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557dac9ceda0 0x557dace7da20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557daca3b070 0x557dace11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557daca1a390 0x557dace11980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 --2- 172.21.15.31:0/33943583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557daca121b0 0x557dace144b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 >> 172.21.15.31:0/33943583 conn(0x557daca2f460 msgr2=0x557dace0a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8c6ffd640 0 client.25684 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 shutdown_connections 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 1 -- 172.21.15.31:0/33943583 wait complete. 2024-09-21T06:13:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.158+0000 7fe8f4418740 20 client.25684 trim_cache size 0 max 16384 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.163 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:53.164 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.165 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:13:53.166 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:13:53.167 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8d00b6780 msgr2=0x7fe8d00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8d00b6780 0x7fe8d00b8c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002f580 tx=0x7fe8e00095c0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e81083a0 msgr2=0x7fe8e806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e81083a0 0x7fe8e806d670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009fd0 tx=0x7fe8dc00b870 comp rx=0 tx=0).stop 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 shutdown_connections 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e8108cf0 0x7fe8e8072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe8d00b6780 0x7fe8d00b8c90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8e81083a0 0x7fe8e806d670 secure :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009fd0 tx=0x7fe8dc00b870 comp rx=0 tx=0).stop 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.160+0000 7fe8edea4640 1 --2- 172.21.15.31:0/1293470588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8e8102a60 0x7fe8e806d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.161+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 >> 172.21.15.31:0/1293470588 conn(0x7fe8e8079730 msgr2=0x7fe8e8106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:53.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.162+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 shutdown_connections 2024-09-21T06:13:53.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:53.163+0000 7fe8edea4640 1 -- 172.21.15.31:0/1293470588 wait complete. 2024-09-21T06:13:53.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:53 smithi110 ceph-mon[34550]: pgmap v388: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-21T06:13:53.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:53 smithi107 ceph-mon[34380]: pgmap v388: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-21T06:13:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:53 smithi031 ceph-mon[31237]: pgmap v388: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 1.08k op/s 2024-09-21T06:13:54.095 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:13:54.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:13:54.477 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:55.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 135.0 s, 4067.2 tps, lat 24.544 ms stddev 31.469 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.312+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2580766007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 msgr2=0x7f03b0118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.312+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2580766007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b0118b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009be0 tx=0x7f03a802a6c0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2580766007 shutdown_connections 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2580766007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b0118b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2580766007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 0x7f03b0115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2580766007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b010a590 0x7f03b010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2580766007 >> 172.21.15.31:0/2580766007 conn(0x7f03b0070ec0 msgr2=0x7f03b00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2580766007 shutdown_connections 2024-09-21T06:13:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.313+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2580766007 wait complete. 2024-09-21T06:13:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.314+0000 7f03b4e4d640 1 Processor -- start 2024-09-21T06:13:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.314+0000 7f03b4e4d640 1 -- start start 2024-09-21T06:13:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03b4e4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b010a590 0x7f03b01a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03b4e4d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 0x7f03b01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03b4e4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b01a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03b4e4d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03b011a150 con 0x7f03b010a590 2024-09-21T06:13:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03b4e4d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03b0119fd0 con 0x7f03b010aee0 2024-09-21T06:13:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03b4e4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03b011a2d0 con 0x7f03b0116640 2024-09-21T06:13:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.316+0000 7f03aed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b01a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.317+0000 7f03aed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b01a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41602/0 (socket says 172.21.15.31:41602) 2024-09-21T06:13:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.317+0000 7f03aed9d640 1 -- 172.21.15.31:0/2571768376 learned_addr learned my addr 172.21.15.31:0/2571768376 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.317+0000 7f03ae59c640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b010a590 0x7f03b01a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.317+0000 7f03add9b640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 0x7f03b01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03aed9d640 1 -- 172.21.15.31:0/2571768376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 msgr2=0x7f03b01a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03aed9d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 0x7f03b01a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03aed9d640 1 -- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b010a590 msgr2=0x7f03b01a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03aed9d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b010a590 0x7f03b01a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03aed9d640 1 -- 172.21.15.31:0/2571768376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b01a6a10 con 0x7f03b0116640 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03add9b640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 0x7f03b01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03aed9d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b01a62e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009b30 tx=0x7f03a8003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8040020 con 0x7f03b0116640 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b01a6ca0 con 0x7f03b0116640 2024-09-21T06:13:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.318+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b01b5c70 con 0x7f03b0116640 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.319+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03a8030970 con 0x7f03b0116640 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.319+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8036040 con 0x7f03b0116640 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.322+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03900052c0 con 0x7f03b0116640 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.323+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f03a8043020 con 0x7f03b0116640 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.324+0000 7f039f7fe640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03980b6780 0x7f03980b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.324+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f03a8108360 con 0x7f03b0116640 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.324+0000 7f03ae59c640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03980b6780 0x7f03980b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.325+0000 7f03ae59c640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03980b6780 0x7f03980b8c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009890 tx=0x7f03a0008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:55.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.329+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03a804a020 con 0x7f03b0116640 2024-09-21T06:13:55.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:55 smithi110 ceph-mon[34550]: pgmap v389: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-21T06:13:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:55 smithi107 ceph-mon[34380]: pgmap v389: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-21T06:13:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:55 smithi031 ceph-mon[31237]: pgmap v389: 97 pgs: 97 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-21T06:13:55.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.730+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0390005880 con 0x7f03b0116640 2024-09-21T06:13:55.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.731+0000 7f039f7fe640 1 -- 172.21.15.31:0/2571768376 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f03a80ce890 con 0x7f03b0116640 2024-09-21T06:13:55.733 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:13:55.733 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:13:55.734 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:13:55.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03980b6780 msgr2=0x7f03980b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03980b6780 0x7f03980b8c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009890 tx=0x7f03a0008040 comp rx=0 tx=0).stop 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 msgr2=0x7f03b01a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b01a62e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009b30 tx=0x7f03a8003cc0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 shutdown_connections 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b0116640 0x7f03b01a62e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b010aee0 0x7f03b01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03980b6780 0x7f03980b8c90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 --2- 172.21.15.31:0/2571768376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03b010a590 0x7f03b01a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:55.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.735+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 >> 172.21.15.31:0/2571768376 conn(0x7f03b0070ec0 msgr2=0x7f03b0071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:55.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.736+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 shutdown_connections 2024-09-21T06:13:55.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:55.736+0000 7f03b4e4d640 1 -- 172.21.15.31:0/2571768376 wait complete. 2024-09-21T06:13:56.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:56 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2571768376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:56.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:56 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2571768376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:56 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2571768376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:13:56.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:13:57.186 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:13:57.489 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:57 smithi031 ceph-mon[31237]: pgmap v390: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 1.00k op/s 2024-09-21T06:13:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:57 smithi110 ceph-mon[34550]: pgmap v390: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 1.00k op/s 2024-09-21T06:13:57.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:57 smithi107 ceph-mon[34380]: pgmap v390: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 1.00k op/s 2024-09-21T06:13:57.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.856+0000 7f4fdb770640 1 -- 172.21.15.31:0/2639866620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd4108cf0 msgr2=0x7f4fd410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.856+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2639866620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd4108cf0 0x7f4fd410f520 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009ca0 tx=0x7f4fcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.857+0000 7f4fdb770640 1 -- 172.21.15.31:0/2639866620 shutdown_connections 2024-09-21T06:13:57.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.857+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2639866620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd4108cf0 0x7f4fd410f520 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.857+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2639866620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fd41083a0 0x7f4fd41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.857+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2639866620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.857+0000 7f4fdb770640 1 -- 172.21.15.31:0/2639866620 >> 172.21.15.31:0/2639866620 conn(0x7f4fd4079730 msgr2=0x7f4fd4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.858+0000 7f4fdb770640 1 -- 172.21.15.31:0/2639866620 shutdown_connections 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.858+0000 7f4fdb770640 1 -- 172.21.15.31:0/2639866620 wait complete. 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 Processor -- start 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 -- start start 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 0x7f4fd4110240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.861 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fd4108cf0 0x7f4fd4110960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4115c70 con 0x7f4fd4102a60 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4115af0 con 0x7f4fd4108cf0 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fdb770640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4115df0 con 0x7f4fd41083a0 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fd8d0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 0x7f4fd4110240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fd950c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fd8d0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 0x7f4fd4110240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41612/0 (socket says 172.21.15.31:41612) 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fd950c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57062/0 (socket says 172.21.15.31:57062) 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/2172958835 learned_addr learned my addr 172.21.15.31:0/2172958835 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.859+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fd4108cf0 0x7f4fd4110960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/2172958835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fd4108cf0 msgr2=0x7f4fd4110960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fd4108cf0 0x7f4fd4110960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 msgr2=0x7f4fd4113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/2172958835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd41acbd0 con 0x7f4fd41083a0 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd950c640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:57.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 0x7f4fd4110240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000e9a0 tx=0x7f4fd000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fca7fc640 1 -- 172.21.15.31:0/2172958835 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000cd80 con 0x7f4fd41083a0 2024-09-21T06:13:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fca7fc640 1 -- 172.21.15.31:0/2172958835 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd00035f0 con 0x7f4fd41083a0 2024-09-21T06:13:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fca7fc640 1 -- 172.21.15.31:0/2172958835 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0010690 con 0x7f4fd41083a0 2024-09-21T06:13:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd41ace00 con 0x7f4fd41083a0 2024-09-21T06:13:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.860+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41ad1a0 con 0x7f4fd41083a0 2024-09-21T06:13:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.865+0000 7f4fe1ce4740 1 Processor -- start 2024-09-21T06:13:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.865+0000 7f4fe1ce4740 1 -- start start 2024-09-21T06:13:57.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 0x561cb9df26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9df2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 0x561cb9df3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561cb9d68ea0 con 0x561cb9a2a390 2024-09-21T06:13:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cb9d582d0 con 0x561cb9a221b0 2024-09-21T06:13:57.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561cb9d5c0e0 con 0x561cb9a4b070 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fd9d0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 0x561cb9df26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fd9d0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 0x561cb9df26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41618/0 (socket says 172.21.15.31:41618) 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/1118772775 learned_addr learned my addr 172.21.15.31:0/1118772775 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.866+0000 7f4fd950c640 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9df2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/1118772775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 msgr2=0x561cb9df2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9df2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/1118772775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 msgr2=0x561cb9df3120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:13:57.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 0x561cb9df3120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/1118772775 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb9ab6650 con 0x561cb9a4b070 2024-09-21T06:13:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd950c640 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9df2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:13:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 0x561cb9df26a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02f410 tx=0x7f4fcc02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87386851f3b273f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fca7fc640 1 -- 172.21.15.31:0/2172958835 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4fd0004020 con 0x7f4fd41083a0 2024-09-21T06:13:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1118772775 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc031cf0 con 0x561cb9a4b070 2024-09-21T06:13:57.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1118772775 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc036030 con 0x561cb9a4b070 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.867+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1118772775 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc0042a0 con 0x561cb9a4b070 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.868+0000 7f4fca7fc640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4fc00b6700 0x7f4fc00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1118772775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 msgr2=0x561cb9df26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 0x561cb9df26a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc02f410 tx=0x7f4fcc02f440 comp rx=0 tx=0).stop 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fd950c640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4fc00b6700 0x7f4fc00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1118772775 shutdown_connections 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 0x561cb9df3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9df2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1118772775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a4b070 0x561cb9df26a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1118772775 >> 172.21.15.31:0/1118772775 conn(0x561cb9a3f460 msgr2=0x561cb9df0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fca7fc640 1 -- 172.21.15.31:0/2172958835 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4fd00e7900 con 0x7f4fd41083a0 2024-09-21T06:13:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1118772775 shutdown_connections 2024-09-21T06:13:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.869+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1118772775 wait complete. 2024-09-21T06:13:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.870+0000 7f4fd950c640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4fc00b6700 0x7f4fc00b8c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x561cb9df1320 tx=0x7f4fc400d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.870+0000 7f4fe1ce4740 1 -- start start 2024-09-21T06:13:57.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.870+0000 7f4fe1ce4740 1 Processor -- start 2024-09-21T06:13:57.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9dfde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a2a390 0x561cb9dfe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 0x561cb9e09f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561cb9bb9700 con 0x561cb9a4b070 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cb9cf3160 con 0x561cb9a2a390 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561cb9e00ca0 con 0x561cb9a221b0 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fe1ce4740 10 client.0 _ll_register_callbacks cb 0x561cb9d0c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fd950c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9dfde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fd9d0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 0x561cb9e09f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fd9d0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 0x561cb9e09f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57070/0 (socket says 172.21.15.31:57070) 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.872+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/3356864636 learned_addr learned my addr 172.21.15.31:0/3356864636 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.873+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/3356864636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 msgr2=0x561cb9dfde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.873+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9dfde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.873+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a2a390 msgr2=0x561cb9dfe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.873+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a2a390 0x561cb9dfe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.873+0000 7f4fd9d0d640 1 -- 172.21.15.31:0/3356864636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb9d20c90 con 0x561cb9a4b070 2024-09-21T06:13:57.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fd9d0d640 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 0x561cb9e09f90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005c30 tx=0x7f4fcc0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1426d839b40a51d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:13:57.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fbcff9640 1 -- 172.21.15.31:0/3356864636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc04a040 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fbcff9640 1 -- 172.21.15.31:0/3356864636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4fcc031650 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fcc036030 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fbcff9640 1 -- 172.21.15.31:0/3356864636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4fcc031850 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fe1ce4740 10 client.15933 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.874+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561cb9e0cb00 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fbcff9640 1 -- 172.21.15.31:0/3356864636 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc036030 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fbcff9640 1 -- 172.21.15.31:0/3356864636 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4fcc04d030 con 0x561cb9a4b070 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fe1ce4740 10 client.15933 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fe1ce4740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:57.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fe1ce4740 20 client.15933 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a099f0 0x561cb9e0fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fe1ce4740 4 client.15933 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:13:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb9d6adf0 con 0x561cb9a099f0 2024-09-21T06:13:57.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.875+0000 7f4fd950c640 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a099f0 0x561cb9e0fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.876+0000 7f4fd950c640 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a099f0 0x561cb9e0fe60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9a77e4a403a7e6 server_cookie=77158caaf337ecd3 in_seq=0 out_seq=0 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.876+0000 7f4fbcff9640 10 client.15933 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.877+0000 7f4fbcff9640 1 -- 172.21.15.31:0/3356864636 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561cb9d6adf0 con 0x561cb9a099f0 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.877+0000 7f4fbcff9640 10 client.15933 handle_command_reply: tid=0 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.877+0000 7f4fe1ce4740 1 client.15933 shutdown 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.877+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 msgr2=0x561cb9e09f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.877+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 0x561cb9e09f90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005c30 tx=0x7f4fcc0099c0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 shutdown_connections 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a099f0 0x561cb9e0fe60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a4b070 0x561cb9e09f90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fbcff9640 0 client.15933 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a2a390 0x561cb9dfe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/3356864636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9dfde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 >> 172.21.15.31:0/3356864636 conn(0x561cb9a3f460 msgr2=0x561cb9df5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 shutdown_connections 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/3356864636 wait complete. 2024-09-21T06:13:57.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.878+0000 7f4fe1ce4740 20 client.15933 trim_cache size 0 max 16384 2024-09-21T06:13:57.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 Processor -- start 2024-09-21T06:13:57.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 -- start start 2024-09-21T06:13:57.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a099f0 0x561cb9dffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9df3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 0x561cb9e09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41a2a20 con 0x561cb9a2a390 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41adc00 con 0x561cb9a099f0 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.901+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41a3090 con 0x561cb9a221b0 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9df3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9df3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:41642/0 (socket says 172.21.15.31:41642) 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/204268998 learned_addr learned my addr 172.21.15.31:0/204268998 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd950c640 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a099f0 0x561cb9dffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/204268998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a099f0 msgr2=0x561cb9dffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a099f0 0x561cb9dffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/204268998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 msgr2=0x561cb9e09e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 0x561cb9e09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/204268998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb9df8b30 con 0x561cb9a221b0 2024-09-21T06:13:57.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.902+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9df3de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c6a0 tx=0x7f4fd00a9630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0123ea03b7dc3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.903+0000 7f4fbeffd640 1 -- 172.21.15.31:0/204268998 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00ad0e0 con 0x561cb9a221b0 2024-09-21T06:13:57.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.903+0000 7f4fbeffd640 1 -- 172.21.15.31:0/204268998 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0004310 con 0x561cb9a221b0 2024-09-21T06:13:57.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.903+0000 7f4fbeffd640 1 -- 172.21.15.31:0/204268998 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00acf00 con 0x561cb9a221b0 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.904+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/204268998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 msgr2=0x561cb9df3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.904+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9df3de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000c6a0 tx=0x7f4fd00a9630 comp rx=0 tx=0).stop 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.905+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/204268998 shutdown_connections 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.905+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a2a390 0x561cb9e09e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.905+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a221b0 0x561cb9df3de0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.905+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/204268998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a099f0 0x561cb9dffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.905+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/204268998 >> 172.21.15.31:0/204268998 conn(0x561cb9a3f460 msgr2=0x561cb9df36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.906+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/204268998 shutdown_connections 2024-09-21T06:13:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.906+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/204268998 wait complete. 2024-09-21T06:13:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.906+0000 7f4fe1ce4740 1 -- start start 2024-09-21T06:13:57.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.906+0000 7f4fe1ce4740 1 Processor -- start 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a099f0 0x561cb9e065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9e04a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a2a390 0x561cb9e04fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41a3760 con 0x561cb9a099f0 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41ad530 con 0x561cb9a221b0 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41a3e30 con 0x561cb9a2a390 2024-09-21T06:13:57.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fe1ce4740 10 client.0 _ll_register_callbacks cb 0x7f4fc400b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fd950c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a099f0 0x561cb9e065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fd8d0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9e04a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fd950c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a099f0 0x561cb9e065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57082/0 (socket says 172.21.15.31:57082) 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fd950c640 1 -- 172.21.15.31:0/1153275930 learned_addr learned my addr 172.21.15.31:0/1153275930 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fd9d0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a2a390 0x561cb9e04fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.907+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9e04a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35292/0 (socket says 172.21.15.31:35292) 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/1153275930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a2a390 msgr2=0x561cb9e04fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a2a390 0x561cb9e04fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/1153275930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a099f0 msgr2=0x561cb9e065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a099f0 0x561cb9e065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fd8d0b640 1 -- 172.21.15.31:0/1153275930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb9e05630 con 0x561cb9a221b0 2024-09-21T06:13:57.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fd8d0b640 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9e04a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00e5140 tx=0x7f4fd0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e7463ce1ce3e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:13:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:13:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fbcff9640 1 -- 172.21.15.31:0/1153275930 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00acf00 con 0x561cb9a221b0 2024-09-21T06:13:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fbcff9640 1 -- 172.21.15.31:0/1153275930 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4fd00a92e0 con 0x561cb9a221b0 2024-09-21T06:13:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.908+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561cb9d668a0 con 0x561cb9a221b0 2024-09-21T06:13:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.910+0000 7f4fbcff9640 1 -- 172.21.15.31:0/1153275930 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4fd00ad0e0 con 0x561cb9a221b0 2024-09-21T06:13:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.910+0000 7f4fe1ce4740 10 client.25679 fetch_fsmap learned FSMap version 19 2024-09-21T06:13:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.910+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561cb9e7b450 con 0x561cb9a221b0 2024-09-21T06:13:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fbcff9640 1 -- 172.21.15.31:0/1153275930 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561cb9d668a0 con 0x561cb9a221b0 2024-09-21T06:13:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fbcff9640 1 -- 172.21.15.31:0/1153275930 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4fd00cf2f0 con 0x561cb9a221b0 2024-09-21T06:13:57.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fe1ce4740 10 client.25679 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:13:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fe1ce4740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:13:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fe1ce4740 20 client.25679 populate_metadata read hostname 'smithi031' 2024-09-21T06:13:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a4b070 0x561cb9e7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:13:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fe1ce4740 4 client.25679 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:13:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.911+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cb9d63cd0 con 0x561cb9a4b070 2024-09-21T06:13:57.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.912+0000 7f4fd950c640 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a4b070 0x561cb9e7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:13:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.912+0000 7f4fbcff9640 10 client.25679 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.912+0000 7f4fd950c640 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a4b070 0x561cb9e7db50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564c01f856fc1a73 server_cookie=2328aa4ff8e8d27d in_seq=0 out_seq=0 2024-09-21T06:13:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fbcff9640 1 -- 172.21.15.31:0/1153275930 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cb9d63cd0 con 0x561cb9a4b070 2024-09-21T06:13:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fbcff9640 10 client.25679 handle_command_reply: tid=0 2024-09-21T06:13:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 client.25679 shutdown 2024-09-21T06:13:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 msgr2=0x561cb9e04a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9e04a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00e5140 tx=0x7f4fd0000bc0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 shutdown_connections 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561cb9a4b070 0x561cb9e7db50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561cb9a2a390 0x561cb9e04fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cb9a221b0 0x561cb9e04a80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 --2- 172.21.15.31:0/1153275930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561cb9a099f0 0x561cb9e065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.913+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 >> 172.21.15.31:0/1153275930 conn(0x561cb9a3f460 msgr2=0x561cb9dff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.914+0000 7f4fbcff9640 0 client.25679 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.914+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 shutdown_connections 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.914+0000 7f4fe1ce4740 1 -- 172.21.15.31:0/1153275930 wait complete. 2024-09-21T06:13:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.914+0000 7f4fe1ce4740 20 client.25679 trim_cache size 0 max 16384 2024-09-21T06:13:57.918 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4fc00b6700 msgr2=0x7f4fc00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4fc00b6700 0x7f4fc00b8c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x561cb9df1320 tx=0x7f4fc400d040 comp rx=0 tx=0).stop 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 msgr2=0x7f4fd4110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 0x7f4fd4110240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000e9a0 tx=0x7f4fd000ee60 comp rx=0 tx=0).stop 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 shutdown_connections 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fd4108cf0 0x7f4fd4110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4fd41083a0 0x7f4fd4110240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4fc00b6700 0x7f4fc00b8c10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 --2- 172.21.15.31:0/2172958835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fd4102a60 0x7f4fd4113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:13:57.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 >> 172.21.15.31:0/2172958835 conn(0x7f4fd4079730 msgr2=0x7f4fd4104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:13:57.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.915+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 shutdown_connections 2024-09-21T06:13:57.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:13:57.916+0000 7f4fdb770640 1 -- 172.21.15.31:0/2172958835 wait complete. 2024-09-21T06:13:59.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:13:59 smithi110 ceph-mon[34550]: pgmap v391: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 825 op/s 2024-09-21T06:13:59.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:13:59 smithi107 ceph-mon[34380]: pgmap v391: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 825 op/s 2024-09-21T06:13:59.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:13:59 smithi031 ceph-mon[31237]: pgmap v391: 97 pgs: 97 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 825 op/s 2024-09-21T06:13:59.641 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:13:59.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:14:00.048 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 140.0 s, 3071.1 tps, lat 32.628 ms stddev 41.206 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 -- 172.21.15.31:0/3852865527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109ec0 msgr2=0x7f93681106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 --2- 172.21.15.31:0/3852865527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109ec0 0x7f93681106f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9360009be0 tx=0x7f936002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 -- 172.21.15.31:0/3852865527 shutdown_connections 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 --2- 172.21.15.31:0/3852865527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109ec0 0x7f93681106f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 --2- 172.21.15.31:0/3852865527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368109570 0x7f9368109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 --2- 172.21.15.31:0/3852865527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368103c30 0x7f9368104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 -- 172.21.15.31:0/3852865527 >> 172.21.15.31:0/3852865527 conn(0x7f936807b8d0 msgr2=0x7f936807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 -- 172.21.15.31:0/3852865527 shutdown_connections 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.084+0000 7f936d81e640 1 -- 172.21.15.31:0/3852865527 wait complete. 2024-09-21T06:14:01.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.085+0000 7f936d81e640 1 Processor -- start 2024-09-21T06:14:01.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.085+0000 7f936d81e640 1 -- start start 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f936d81e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 0x7f936819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f936d81e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109570 0x7f936819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f936d81e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368109ec0 0x7f936819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f936d81e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9368113ae0 con 0x7f9368109ec0 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f936d81e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9368113960 con 0x7f9368109570 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f936d81e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9368113c60 con 0x7f9368103c30 2024-09-21T06:14:01.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f93677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368109ec0 0x7f936819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 0x7f936819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 0x7f936819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48084/0 (socket says 172.21.15.31:48084) 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f93677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368109ec0 0x7f936819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56130/0 (socket says 172.21.15.31:56130) 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 -- 172.21.15.31:0/1890967163 learned_addr learned my addr 172.21.15.31:0/1890967163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f93667fc640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109570 0x7f936819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 -- 172.21.15.31:0/1890967163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109570 msgr2=0x7f936819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109570 0x7f936819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 -- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368109ec0 msgr2=0x7f936819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368109ec0 0x7f936819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 -- 172.21.15.31:0/1890967163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93681a03c0 con 0x7f9368103c30 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.086+0000 7f9366ffd640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 0x7f936819a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f935800e9c0 tx=0x7f935800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.087+0000 7f936c81c640 1 -- 172.21.15.31:0/1890967163 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935800ccc0 con 0x7f9368103c30 2024-09-21T06:14:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.087+0000 7f936c81c640 1 -- 172.21.15.31:0/1890967163 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93580054d0 con 0x7f9368103c30 2024-09-21T06:14:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.087+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93681a06b0 con 0x7f9368103c30 2024-09-21T06:14:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.087+0000 7f936c81c640 1 -- 172.21.15.31:0/1890967163 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358010430 con 0x7f9368103c30 2024-09-21T06:14:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.087+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93681a6e10 con 0x7f9368103c30 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.089+0000 7f9373d92740 1 Processor -- start 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 -- start start 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f936c81c640 1 -- 172.21.15.31:0/1890967163 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9358004f80 con 0x7f9368103c30 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 0x5574d1ed8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b08490 0x5574d1ed8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ed8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5574d1e4ee60 con 0x5574d1b31350 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574d1e3e290 con 0x5574d1b10670 2024-09-21T06:14:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9373d92740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574d1e420a0 con 0x5574d1b08490 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 0x5574d1ed8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 0x5574d1ed8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56136/0 (socket says 172.21.15.31:56136) 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 -- 172.21.15.31:0/3629384980 learned_addr learned my addr 172.21.15.31:0/3629384980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93667fc640 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ed8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f9366ffd640 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b08490 0x5574d1ed8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 -- 172.21.15.31:0/3629384980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b08490 msgr2=0x5574d1ed8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b08490 0x5574d1ed8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 -- 172.21.15.31:0/3629384980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 msgr2=0x5574d1ed8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.090+0000 7f93677fe640 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ed8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f93677fe640 1 -- 172.21.15.31:0/3629384980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d1b9c930 con 0x5574d1b31350 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f93667fc640 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ed8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f93677fe640 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 0x5574d1ed8430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f936002acb0 tx=0x7f9360030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a42d0f528401f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f936c81c640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93500b6680 0x7f93500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f934dffb640 1 -- 172.21.15.31:0/3629384980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360041030 con 0x5574d1b31350 2024-09-21T06:14:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f934dffb640 1 -- 172.21.15.31:0/3629384980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9360003e20 con 0x5574d1b31350 2024-09-21T06:14:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f934dffb640 1 -- 172.21.15.31:0/3629384980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360036040 con 0x5574d1b31350 2024-09-21T06:14:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f9366ffd640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93500b6680 0x7f93500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.091+0000 7f936c81c640 1 -- 172.21.15.31:0/1890967163 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f93580e7b30 con 0x7f9368103c30 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9366ffd640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93500b6680 0x7f93500b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93580e5370 tx=0x7f935801fad0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 -- 172.21.15.31:0/3629384980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 msgr2=0x5574d1ed8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 0x5574d1ed8430 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f936002acb0 tx=0x7f9360030420 comp rx=0 tx=0).stop 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 -- 172.21.15.31:0/3629384980 shutdown_connections 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ed8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b08490 0x5574d1ed8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 --2- 172.21.15.31:0/3629384980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b31350 0x5574d1ed8430 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 -- 172.21.15.31:0/3629384980 >> 172.21.15.31:0/3629384980 conn(0x5574d1b25740 msgr2=0x5574d1ed6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 -- 172.21.15.31:0/3629384980 shutdown_connections 2024-09-21T06:14:01.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.092+0000 7f9373d92740 1 -- 172.21.15.31:0/3629384980 wait complete. 2024-09-21T06:14:01.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.093+0000 7f9373d92740 1 -- start start 2024-09-21T06:14:01.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.093+0000 7f9373d92740 1 Processor -- start 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ee8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5574d1c9f890 con 0x5574d1b10670 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574d1dd8d40 con 0x5574d1b08490 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574d1eeaf10 con 0x5574d1b31350 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f9373d92740 10 client.0 _ll_register_callbacks cb 0x5574d1e3cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f93677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ee8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f93677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ee8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54420/0 (socket says 172.21.15.31:54420) 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f93677fe640 1 -- 172.21.15.31:0/247927772 learned_addr learned my addr 172.21.15.31:0/247927772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:01.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f93667fc640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.095+0000 7f93677fe640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93677fe640 1 -- 172.21.15.31:0/247927772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 msgr2=0x5574d1ee1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93677fe640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93677fe640 1 -- 172.21.15.31:0/247927772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 msgr2=0x5574d1ee8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93677fe640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93677fe640 1 -- 172.21.15.31:0/247927772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d1ee1d50 con 0x5574d1b08490 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93667fc640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f93677fe640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ee8410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9360006010 tx=0x7f936002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c3fe44d5c4c0588 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f9346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f9346ffd640 1 -- 172.21.15.31:0/247927772 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360052060 con 0x5574d1b08490 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f9346ffd640 1 -- 172.21.15.31:0/247927772 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f9360003b70 con 0x5574d1b08490 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9360003e20 con 0x5574d1b08490 2024-09-21T06:14:01.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.096+0000 7f9346ffd640 1 -- 172.21.15.31:0/247927772 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9360041050 con 0x5574d1b08490 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 10 client.25691 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5574d1efb530 con 0x5574d1b08490 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9346ffd640 1 -- 172.21.15.31:0/247927772 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9360003e20 con 0x5574d1b08490 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9346ffd640 1 -- 172.21.15.31:0/247927772 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f936003d4f0 con 0x5574d1b08490 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 10 client.25691 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 20 client.25691 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1aefcd0 0x5574d1efea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 4 client.25691 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574d1e50db0 con 0x5574d1aefcd0 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.097+0000 7f93667fc640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1aefcd0 0x5574d1efea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.098+0000 7f9346ffd640 10 client.25691 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:01.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.098+0000 7f93667fc640 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1aefcd0 0x5574d1efea50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513577bb55bae899 server_cookie=1a5ebe029a9ed292 in_seq=0 out_seq=0 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9346ffd640 1 -- 172.21.15.31:0/247927772 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5574d1e50db0 con 0x5574d1aefcd0 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9346ffd640 10 client.25691 handle_command_reply: tid=0 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 client.25691 shutdown 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 msgr2=0x5574d1ee8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ee8410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9360006010 tx=0x7f936002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 shutdown_connections 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1aefcd0 0x5574d1efea50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9346ffd640 0 client.25691 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 --2- 172.21.15.31:0/247927772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ee8410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 >> 172.21.15.31:0/247927772 conn(0x5574d1b25740 msgr2=0x5574d1ed9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 shutdown_connections 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 1 -- 172.21.15.31:0/247927772 wait complete. 2024-09-21T06:14:01.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.099+0000 7f9373d92740 20 client.25691 trim_cache size 0 max 16384 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.119+0000 7f9373d92740 1 Processor -- start 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 -- start start 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ed6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93681ab390 con 0x5574d1b10670 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93681adc10 con 0x5574d1b08490 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f9373d92740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93681aba00 con 0x5574d1b31350 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56144/0 (socket says 172.21.15.31:56144) 2024-09-21T06:14:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 -- 172.21.15.31:0/997897497 learned_addr learned my addr 172.21.15.31:0/997897497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93677fe640 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93677fe640 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ed6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 -- 172.21.15.31:0/997897497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 msgr2=0x5574d1ee16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 -- 172.21.15.31:0/997897497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 msgr2=0x5574d1ed6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ed6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.120+0000 7f93667fc640 1 -- 172.21.15.31:0/997897497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574d1ee94d0 con 0x5574d1b10670 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.121+0000 7f93667fc640 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee11a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f935c009fd0 tx=0x7f935c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f5f688a2438045 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.121+0000 7f934d7fa640 1 -- 172.21.15.31:0/997897497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c004230 con 0x5574d1b10670 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.121+0000 7f934d7fa640 1 -- 172.21.15.31:0/997897497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c0043d0 con 0x5574d1b10670 2024-09-21T06:14:01.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.121+0000 7f934d7fa640 1 -- 172.21.15.31:0/997897497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c025920 con 0x5574d1b10670 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 -- 172.21.15.31:0/997897497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 msgr2=0x5574d1ee11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee11a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f935c009fd0 tx=0x7f935c005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 -- 172.21.15.31:0/997897497 shutdown_connections 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1ee16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b10670 0x5574d1ee11a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 --2- 172.21.15.31:0/997897497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b08490 0x5574d1ed6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 -- 172.21.15.31:0/997897497 >> 172.21.15.31:0/997897497 conn(0x5574d1b25740 msgr2=0x5574d1ed5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 -- 172.21.15.31:0/997897497 shutdown_connections 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.122+0000 7f9373d92740 1 -- 172.21.15.31:0/997897497 wait complete. 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 -- start start 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 Processor -- start 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 0x5574d1ef5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ef5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1f68410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93681ac0d0 con 0x5574d1b08490 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93681ad540 con 0x5574d1b10670 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93681ac7a0 con 0x5574d1b31350 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.123+0000 7f9373d92740 10 client.0 _ll_register_callbacks cb 0x7f935c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 0x5574d1ef5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 0x5574d1ef5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56156/0 (socket says 172.21.15.31:56156) 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 -- 172.21.15.31:0/2490711017 learned_addr learned my addr 172.21.15.31:0/2490711017 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93667fc640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ef5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1f68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 -- 172.21.15.31:0/2490711017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 msgr2=0x5574d1f68410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1f68410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 -- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 msgr2=0x5574d1ef5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ef5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 -- 172.21.15.31:0/2490711017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574d1f68950 con 0x5574d1b08490 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93667fc640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ef5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.124+0000 7f93677fe640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 0x5574d1ef5130 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f936003f040 tx=0x7f936002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22984c6d6f22854a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9346ffd640 1 -- 172.21.15.31:0/2490711017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360052040 con 0x5574d1b08490 2024-09-21T06:14:01.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9346ffd640 1 -- 172.21.15.31:0/2490711017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f9360003b70 con 0x5574d1b08490 2024-09-21T06:14:01.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935c0043d0 con 0x5574d1b08490 2024-09-21T06:14:01.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9346ffd640 1 -- 172.21.15.31:0/2490711017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f936005c080 con 0x5574d1b08490 2024-09-21T06:14:01.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9373d92740 10 client.15954 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:01.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5574d1f69b80 con 0x5574d1b08490 2024-09-21T06:14:01.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.125+0000 7f9346ffd640 1 -- 172.21.15.31:0/2490711017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c0043d0 con 0x5574d1b08490 2024-09-21T06:14:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9346ffd640 1 -- 172.21.15.31:0/2490711017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f93600046b0 con 0x5574d1b08490 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9373d92740 10 client.15954 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9373d92740 10 client.15954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9373d92740 20 client.15954 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9373d92740 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1ac5080 0x5574d1f6c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9373d92740 4 client.15954 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5574d1e4c860 con 0x5574d1ac5080 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.126+0000 7f93667fc640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1ac5080 0x5574d1f6c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.127+0000 7f93667fc640 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1ac5080 0x5574d1f6c310 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c98fb83cd105e9 server_cookie=ba66d0847e6ee031 in_seq=0 out_seq=0 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.127+0000 7f9346ffd640 10 client.15954 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.128+0000 7f9346ffd640 1 -- 172.21.15.31:0/2490711017 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5574d1e4c860 con 0x5574d1ac5080 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.128+0000 7f9346ffd640 10 client.15954 handle_command_reply: tid=0 2024-09-21T06:14:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.128+0000 7f9373d92740 1 client.15954 shutdown 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 msgr2=0x5574d1ef5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 0x5574d1ef5130 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f936003f040 tx=0x7f936002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 shutdown_connections 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5574d1ac5080 0x5574d1f6c310 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574d1b31350 0x5574d1f68410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574d1b10670 0x5574d1ef5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 --2- 172.21.15.31:0/2490711017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574d1b08490 0x5574d1ef5130 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9346ffd640 0 client.15954 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 >> 172.21.15.31:0/2490711017 conn(0x5574d1b25740 msgr2=0x5574d1ed9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 shutdown_connections 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 1 -- 172.21.15.31:0/2490711017 wait complete. 2024-09-21T06:14:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.129+0000 7f9373d92740 20 client.15954 trim_cache size 0 max 16384 2024-09-21T06:14:01.134 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:14:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "34367b1c-c534-440d-962a-b6cbdeb827e6", 2024-09-21T06:14:01.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:14:01.134 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93500b6680 msgr2=0x7f93500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93500b6680 0x7f93500b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93580e5370 tx=0x7f935801fad0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 msgr2=0x7f936819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 0x7f936819a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f935800e9c0 tx=0x7f935800ee80 comp rx=0 tx=0).stop 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 shutdown_connections 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93500b6680 0x7f93500b8b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9368109ec0 0x7f936819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9368109570 0x7f936819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 --2- 172.21.15.31:0/1890967163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9368103c30 0x7f936819a450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 >> 172.21.15.31:0/1890967163 conn(0x7f936807b8d0 msgr2=0x7f9368107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 shutdown_connections 2024-09-21T06:14:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:01.130+0000 7f936d81e640 1 -- 172.21.15.31:0/1890967163 wait complete. 2024-09-21T06:14:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:01 smithi110 ceph-mon[34550]: pgmap v392: 97 pgs: 97 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 832 op/s 2024-09-21T06:14:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:01 smithi107 ceph-mon[34380]: pgmap v392: 97 pgs: 97 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 832 op/s 2024-09-21T06:14:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:01 smithi031 ceph-mon[31237]: pgmap v392: 97 pgs: 97 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 19 MiB/s wr, 832 op/s 2024-09-21T06:14:02.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:14:02.347 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 -- 172.21.15.31:0/2590056218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc10dd80 msgr2=0x7f34fc110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 --2- 172.21.15.31:0/2590056218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc10dd80 0x7f34fc110260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009ca0 tx=0x7f34f402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 -- 172.21.15.31:0/2590056218 shutdown_connections 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 --2- 172.21.15.31:0/2590056218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc10dd80 0x7f34fc110260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 --2- 172.21.15.31:0/2590056218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc1052b0 0x7f34fc10d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 --2- 172.21.15.31:0/2590056218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc104960 0x7f34fc104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 -- 172.21.15.31:0/2590056218 >> 172.21.15.31:0/2590056218 conn(0x7f34fc07a740 msgr2=0x7f34fc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 -- 172.21.15.31:0/2590056218 shutdown_connections 2024-09-21T06:14:02.359 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.357+0000 7f3503c86640 1 -- 172.21.15.31:0/2590056218 wait complete. 2024-09-21T06:14:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 Processor -- start 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 -- start start 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc104960 0x7f34fc19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc1052b0 0x7f34fc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 0x7f34fc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1139d0 con 0x7f34fc10dd80 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc113850 con 0x7f34fc104960 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3503c86640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc113b50 con 0x7f34fc1052b0 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3502223640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 0x7f34fc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3501a22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc104960 0x7f34fc19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3502223640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 0x7f34fc19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56174/0 (socket says 172.21.15.31:56174) 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3501221640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc1052b0 0x7f34fc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.358+0000 7f3502223640 1 -- 172.21.15.31:0/1800285476 learned_addr learned my addr 172.21.15.31:0/1800285476 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3502223640 1 -- 172.21.15.31:0/1800285476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc1052b0 msgr2=0x7f34fc19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3502223640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc1052b0 0x7f34fc19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3502223640 1 -- 172.21.15.31:0/1800285476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc104960 msgr2=0x7f34fc19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3502223640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc104960 0x7f34fc19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3502223640 1 -- 172.21.15.31:0/1800285476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc1a0300 con 0x7f34fc10dd80 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3501a22640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc104960 0x7f34fc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:02.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3502223640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 0x7f34fc19fc00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005cc0 tx=0x7f34f4031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f34f2ffd640 1 -- 172.21.15.31:0/1800285476 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4003b70 con 0x7f34fc10dd80 2024-09-21T06:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f34f2ffd640 1 -- 172.21.15.31:0/1800285476 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f40318f0 con 0x7f34fc10dd80 2024-09-21T06:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f34f2ffd640 1 -- 172.21.15.31:0/1800285476 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f402f950 con 0x7f34fc10dd80 2024-09-21T06:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.359+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc1a0590 con 0x7f34fc10dd80 2024-09-21T06:14:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.360+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc1a6d80 con 0x7f34fc10dd80 2024-09-21T06:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.363+0000 7f350abed740 1 Processor -- start 2024-09-21T06:14:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.363+0000 7f34f2ffd640 1 -- 172.21.15.31:0/1800285476 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f34f402fb50 con 0x7f34fc10dd80 2024-09-21T06:14:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 -- start start 2024-09-21T06:14:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441092f0 0x5652444d7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56524444c6f0 con 0x5652441092f0 2024-09-21T06:14:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56524443bb20 con 0x5652441114d0 2024-09-21T06:14:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f350abed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56524443f930 con 0x5652441321b0 2024-09-21T06:14:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f3501221640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f3501221640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444d79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48160/0 (socket says 172.21.15.31:48160) 2024-09-21T06:14:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f3501a22640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441092f0 0x5652444d7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f3501221640 1 -- 172.21.15.31:0/1138511443 learned_addr learned my addr 172.21.15.31:0/1138511443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.364+0000 7f3501221640 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501221640 1 -- 172.21.15.31:0/1138511443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 msgr2=0x5652444d8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501221640 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444d8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501221640 1 -- 172.21.15.31:0/1138511443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441092f0 msgr2=0x5652444d7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501221640 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441092f0 0x5652444d7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501221640 1 -- 172.21.15.31:0/1138511443 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565244401530 con 0x5652441321b0 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f34f2ffd640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34e80b6680 0x7f34e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501221640 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444d79b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34ec011740 tx=0x7f34ec005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87da1a0b60617f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f34e7fff640 1 -- 172.21.15.31:0/1138511443 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00f840 con 0x5652441321b0 2024-09-21T06:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f34f2ffd640 1 -- 172.21.15.31:0/1800285476 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f34f41087d0 con 0x7f34fc10dd80 2024-09-21T06:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f34e7fff640 1 -- 172.21.15.31:0/1138511443 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec00fe90 con 0x5652441321b0 2024-09-21T06:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f34e7fff640 1 -- 172.21.15.31:0/1138511443 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec01fca0 con 0x5652441321b0 2024-09-21T06:14:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.365+0000 7f3501a22640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34e80b6680 0x7f34e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.366+0000 7f350abed740 1 -- 172.21.15.31:0/1138511443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 msgr2=0x5652444d79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.366+0000 7f3501a22640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34e80b6680 0x7f34e80b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5652444d6630 tx=0x7f34f8015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.366+0000 7f350abed740 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444d79b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34ec011740 tx=0x7f34ec005850 comp rx=0 tx=0).stop 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 -- 172.21.15.31:0/1138511443 shutdown_connections 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441092f0 0x5652444d7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 --2- 172.21.15.31:0/1138511443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444d79b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 -- 172.21.15.31:0/1138511443 >> 172.21.15.31:0/1138511443 conn(0x565244126300 msgr2=0x5652444d58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 -- 172.21.15.31:0/1138511443 shutdown_connections 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 -- 172.21.15.31:0/1138511443 wait complete. 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 -- start start 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.367+0000 7f350abed740 1 Processor -- start 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 0x5652444e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x565244373ab0 con 0x5652441114d0 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5652443e1730 con 0x5652441092f0 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5652444e6130 con 0x5652441321b0 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f3501221640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 0x5652444e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f3502223640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f3501221640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 0x5652444e3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56184/0 (socket says 172.21.15.31:56184) 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f3502223640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444ef420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48162/0 (socket says 172.21.15.31:48162) 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f3501221640 1 -- 172.21.15.31:0/2575985155 learned_addr learned my addr 172.21.15.31:0/2575985155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f350abed740 10 client.0 _ll_register_callbacks cb 0x5652444356b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.368+0000 7f3501a22640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501221640 1 -- 172.21.15.31:0/2575985155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 msgr2=0x5652444ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501221640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501221640 1 -- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 msgr2=0x5652444e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501221640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444e30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501221640 1 -- 172.21.15.31:0/2575985155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524419d790 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501a22640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f3501221640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 0x5652444e3600 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34ec01f7e0 tx=0x7f34ec009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caab07341a28b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f34e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f34e57fa640 1 -- 172.21.15.31:0/2575985155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec004350 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f34e57fa640 1 -- 172.21.15.31:0/2575985155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f34ec019070 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.369+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ec00fe90 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f34e57fa640 1 -- 172.21.15.31:0/2575985155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f34ec011020 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 10 client.15966 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x565244216620 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f34e57fa640 1 -- 172.21.15.31:0/2575985155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec00fe90 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f34e57fa640 1 -- 172.21.15.31:0/2575985155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f34ec01d020 con 0x5652441114d0 2024-09-21T06:14:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 10 client.15966 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 10 client.15966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 20 client.15966 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652440f0d80 0x5652444f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 4 client.15966 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.370+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524444e640 con 0x5652440f0d80 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.371+0000 7f3501a22640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652440f0d80 0x5652444f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.371+0000 7f34e57fa640 10 client.15966 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.371+0000 7f3501a22640 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652440f0d80 0x5652444f5070 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1334ad4681e57f3b server_cookie=b6b6fe2cf4f9489e in_seq=0 out_seq=0 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.372+0000 7f34e57fa640 1 -- 172.21.15.31:0/2575985155 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56524444e640 con 0x5652440f0d80 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.372+0000 7f34e57fa640 10 client.15966 handle_command_reply: tid=0 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.372+0000 7f350abed740 1 client.15966 shutdown 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.372+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 msgr2=0x5652444e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.372+0000 7f350abed740 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 0x5652444e3600 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f34ec01f7e0 tx=0x7f34ec009d20 comp rx=0 tx=0).stop 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 shutdown_connections 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f350abed740 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652440f0d80 0x5652444f5070 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f34e57fa640 0 client.15966 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f350abed740 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441321b0 0x5652444ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f350abed740 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652441114d0 0x5652444e3600 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f350abed740 1 --2- 172.21.15.31:0/2575985155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.373+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 >> 172.21.15.31:0/2575985155 conn(0x565244126300 msgr2=0x5652444d89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.374+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 shutdown_connections 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.374+0000 7f350abed740 1 -- 172.21.15.31:0/2575985155 wait complete. 2024-09-21T06:14:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.374+0000 7f350abed740 20 client.15966 trim_cache size 0 max 16384 2024-09-21T06:14:02.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:02 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:14:02.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:02 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:02.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:02 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:14:02.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:02 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:02.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:02 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:14:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.395+0000 7f350abed740 1 Processor -- start 2024-09-21T06:14:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.395+0000 7f350abed740 1 -- start start 2024-09-21T06:14:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f350abed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f350abed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f350abed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441114d0 0x5652444ef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f350abed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1ab330 con 0x5652440f0d80 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f350abed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1adbb0 con 0x5652441092f0 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f350abed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1ab9a0 con 0x5652441114d0 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444dd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56196/0 (socket says 172.21.15.31:56196) 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 -- 172.21.15.31:0/2245853039 learned_addr learned my addr 172.21.15.31:0/2245853039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501221640 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3502223640 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441114d0 0x5652444ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 -- 172.21.15.31:0/2245853039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441114d0 msgr2=0x5652444ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441114d0 0x5652444ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 -- 172.21.15.31:0/2245853039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 msgr2=0x5652444d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 -- 172.21.15.31:0/2245853039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652444cfb40 con 0x5652440f0d80 2024-09-21T06:14:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.396+0000 7f3501a22640 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444dd190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009f70 tx=0x7f34f801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfccf93294df28e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.397+0000 7f34e77fe640 1 -- 172.21.15.31:0/2245853039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8003c40 con 0x5652440f0d80 2024-09-21T06:14:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.397+0000 7f34e77fe640 1 -- 172.21.15.31:0/2245853039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8026040 con 0x5652440f0d80 2024-09-21T06:14:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.397+0000 7f34e77fe640 1 -- 172.21.15.31:0/2245853039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004cc0 con 0x5652440f0d80 2024-09-21T06:14:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 -- 172.21.15.31:0/2245853039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 msgr2=0x5652444dd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444dd190 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009f70 tx=0x7f34f801b000 comp rx=0 tx=0).stop 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 -- 172.21.15.31:0/2245853039 shutdown_connections 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441114d0 0x5652444ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441092f0 0x5652444d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 --2- 172.21.15.31:0/2245853039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444dd190 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 -- 172.21.15.31:0/2245853039 >> 172.21.15.31:0/2245853039 conn(0x565244126300 msgr2=0x5652444d89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 -- 172.21.15.31:0/2245853039 shutdown_connections 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 -- 172.21.15.31:0/2245853039 wait complete. 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 -- start start 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.398+0000 7f350abed740 1 Processor -- start 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441092f0 0x5652444e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1ac070 con 0x5652440f0d80 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1ad4e0 con 0x5652441114d0 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1ac740 con 0x5652441092f0 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f350abed740 10 client.0 _ll_register_callbacks cb 0x7f34f8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501221640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441092f0 0x5652444e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56198/0 (socket says 172.21.15.31:56198) 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 -- 172.21.15.31:0/4014663900 learned_addr learned my addr 172.21.15.31:0/4014663900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3502223640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 -- 172.21.15.31:0/4014663900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441092f0 msgr2=0x5652444e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441092f0 0x5652444e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 -- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 msgr2=0x5652444e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 -- 172.21.15.31:0/4014663900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524455d910 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3502223640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f3501a22640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444e63d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f34f800e980 tx=0x7f34f8004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295cbe8fea53735a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.399+0000 7f34e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f34e57fa640 1 -- 172.21.15.31:0/4014663900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8040020 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f34e57fa640 1 -- 172.21.15.31:0/4014663900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f34f8003e50 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f8026040 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f34e57fa640 1 -- 172.21.15.31:0/4014663900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f34f803c470 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 10 client.15972 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56524455eb80 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f34e57fa640 1 -- 172.21.15.31:0/4014663900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8026040 con 0x5652440f0d80 2024-09-21T06:14:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f34e57fa640 1 -- 172.21.15.31:0/4014663900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f34f802d740 con 0x5652440f0d80 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 10 client.15972 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 20 client.15972 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652441321b0 0x565244561360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 4 client.15972 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.400+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56524444a0f0 con 0x5652441321b0 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.401+0000 7f3501221640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652441321b0 0x565244561360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.401+0000 7f34e57fa640 10 client.15972 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.401+0000 7f3501221640 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652441321b0 0x565244561360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75027c0a1832b928 server_cookie=b2fcd2849cd16c96 in_seq=0 out_seq=0 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.423+0000 7f34e57fa640 1 -- 172.21.15.31:0/4014663900 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56524444a0f0 con 0x5652441321b0 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.423+0000 7f34e57fa640 10 client.15972 handle_command_reply: tid=0 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.423+0000 7f350abed740 1 client.15972 shutdown 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.423+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 msgr2=0x5652444e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.423+0000 7f350abed740 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444e63d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f34f800e980 tx=0x7f34f8004560 comp rx=0 tx=0).stop 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 shutdown_connections 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5652441321b0 0x565244561360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5652441114d0 0x5652444e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5652441092f0 0x5652444e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 --2- 172.21.15.31:0/4014663900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5652440f0d80 0x5652444e63d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f34e57fa640 0 client.15972 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 >> 172.21.15.31:0/4014663900 conn(0x565244126300 msgr2=0x5652444dcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 shutdown_connections 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 1 -- 172.21.15.31:0/4014663900 wait complete. 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f350abed740 20 client.15972 trim_cache size 0 max 16384 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:14:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:14:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34e80b6680 msgr2=0x7f34e80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f3503c86640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34e80b6680 0x7f34e80b8b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5652444d6630 tx=0x7f34f8015040 comp rx=0 tx=0).stop 2024-09-21T06:14:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 msgr2=0x7f34fc19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.424+0000 7f3503c86640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 0x7f34fc19fc00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005cc0 tx=0x7f34f4031700 comp rx=0 tx=0).stop 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 shutdown_connections 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34e80b6680 0x7f34e80b8b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34fc10dd80 0x7f34fc19fc00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34fc1052b0 0x7f34fc19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 --2- 172.21.15.31:0/1800285476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34fc104960 0x7f34fc19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 >> 172.21.15.31:0/1800285476 conn(0x7f34fc07a740 msgr2=0x7f34fc105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 shutdown_connections 2024-09-21T06:14:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:02.425+0000 7f3503c86640 1 -- 172.21.15.31:0/1800285476 wait complete. 2024-09-21T06:14:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:02 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:14:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:02 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:02 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:14:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:02 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:02 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:14:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:02 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:14:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:02 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:02 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:14:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:02 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:02 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:14:02.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 -- 172.21.15.31:0/582428796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec108d00 msgr2=0x7f88ec10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 --2- 172.21.15.31:0/582428796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec108d00 0x7f88ec10f530 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009ca0 tx=0x7f88e402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 -- 172.21.15.31:0/582428796 shutdown_connections 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 --2- 172.21.15.31:0/582428796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec108d00 0x7f88ec10f530 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 --2- 172.21.15.31:0/582428796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec1083b0 0x7f88ec1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 --2- 172.21.15.31:0/582428796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88ec102ac0 0x7f88ec102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 -- 172.21.15.31:0/582428796 >> 172.21.15.31:0/582428796 conn(0x7f88ec079790 msgr2=0x7f88ec079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 -- 172.21.15.31:0/582428796 shutdown_connections 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.878+0000 7f88f207f640 1 -- 172.21.15.31:0/582428796 wait complete. 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 Processor -- start 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 -- start start 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 0x7f88ec19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec1083b0 0x7f88ec19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88ec108d00 0x7f88ec1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1129c0 con 0x7f88ec1083b0 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec112840 con 0x7f88ec102ac0 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88f207f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec112b40 con 0x7f88ec108d00 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec1083b0 0x7f88ec19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 0x7f88ec19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 0x7f88ec19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54522/0 (socket says 172.21.15.31:54522) 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3175636046 learned_addr learned my addr 172.21.15.31:0/3175636046 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec1083b0 0x7f88ec19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56216/0 (socket says 172.21.15.31:56216) 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.879+0000 7f88ebfff640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88ec108d00 0x7f88ec1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3175636046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88ec108d00 msgr2=0x7f88ec1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88ec108d00 0x7f88ec1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec1083b0 msgr2=0x7f88ec19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec1083b0 0x7f88ec19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3175636046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1a2690 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 0x7f88ec19c720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00e9f0 tx=0x7f88dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88e97fa640 1 -- 172.21.15.31:0/3175636046 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00cd80 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1a2980 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88e97fa640 1 -- 172.21.15.31:0/3175636046 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc0054d0 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88e97fa640 1 -- 172.21.15.31:0/3175636046 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc010690 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.880+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1a90e0 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.883+0000 7f88e97fa640 1 -- 172.21.15.31:0/3175636046 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f88dc010830 con 0x7f88ec102ac0 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.883+0000 7f88f85f3740 1 Processor -- start 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.883+0000 7f88f85f3740 1 -- start start 2024-09-21T06:14:02.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44a0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88e97fa640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88d80b6680 0x7f88d80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88f85f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ff4416e50 con 0x564ff40d8390 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88f85f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ff4406280 con 0x564ff40d01b0 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88f85f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ff440a090 con 0x564ff40f9070 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56218/0 (socket says 172.21.15.31:56218) 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88e2ffd640 1 -- 172.21.15.31:0/1816013067 learned_addr learned my addr 172.21.15.31:0/1816013067 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88ebfff640 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88ebfff640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88d80b6680 0x7f88d80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88e97fa640 1 -- 172.21.15.31:0/3175636046 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f88dc0e7b30 con 0x7f88ec102ac0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88eb7fe640 1 -- 172.21.15.31:0/1816013067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 msgr2=0x564ff44a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.884+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88eb7fe640 1 -- 172.21.15.31:0/1816013067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 msgr2=0x564ff44a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88eb7fe640 1 -- 172.21.15.31:0/1816013067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff4164650 con 0x564ff40d01b0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44a0b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0e5260 tx=0x7f88dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a2c7670519052b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88ebfff640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88d80b6680 0x7f88d80b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009a60 tx=0x7f88e40095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88e1ffb640 1 -- 172.21.15.31:0/1816013067 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0047a0 con 0x564ff40d01b0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88e1ffb640 1 -- 172.21.15.31:0/1816013067 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc0ad5c0 con 0x564ff40d01b0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.885+0000 7f88e1ffb640 1 -- 172.21.15.31:0/1816013067 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0e54e0 con 0x564ff40d01b0 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 -- 172.21.15.31:0/1816013067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 msgr2=0x564ff44a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44a0b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0e5260 tx=0x7f88dc004730 comp rx=0 tx=0).stop 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 -- 172.21.15.31:0/1816013067 shutdown_connections 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44a0b90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 --2- 172.21.15.31:0/1816013067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 -- 172.21.15.31:0/1816013067 >> 172.21.15.31:0/1816013067 conn(0x564ff40ed460 msgr2=0x564ff449e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 -- 172.21.15.31:0/1816013067 shutdown_connections 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.886+0000 7f88f85f3740 1 -- 172.21.15.31:0/1816013067 wait complete. 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.887+0000 7f88f85f3740 1 -- start start 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.887+0000 7f88f85f3740 1 Processor -- start 2024-09-21T06:14:02.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ff4267700 con 0x564ff40d8390 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ff43a1110 con 0x564ff40d01b0 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ff44aedd0 con 0x564ff40f9070 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88f85f3740 10 client.0 _ll_register_callbacks cb 0x564ff43ba990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54546/0 (socket says 172.21.15.31:54546) 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3183254086 learned_addr learned my addr 172.21.15.31:0/3183254086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88ebfff640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3183254086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 msgr2=0x564ff44ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.888+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3183254086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 msgr2=0x564ff44b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88eb7fe640 1 -- 172.21.15.31:0/3183254086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff43cec40 con 0x564ff40d01b0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88ebfff640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44b4dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00c6a0 tx=0x7f88dc004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d5778ad9e26b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88caffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88caffd640 1 -- 172.21.15.31:0/3183254086 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0acf10 con 0x564ff40d01b0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88caffd640 1 -- 172.21.15.31:0/3183254086 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f88dc010b20 con 0x564ff40d01b0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc0ad5c0 con 0x564ff40d01b0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88caffd640 1 -- 172.21.15.31:0/3183254086 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f88dc0047a0 con 0x564ff40d01b0 2024-09-21T06:14:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88f85f3740 10 client.25727 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.889+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564ff44c3310 con 0x564ff40d01b0 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88caffd640 1 -- 172.21.15.31:0/3183254086 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc010da0 con 0x564ff40d01b0 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88caffd640 1 -- 172.21.15.31:0/3183254086 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f88dc0a9490 con 0x564ff40d01b0 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88f85f3740 10 client.25727 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88f85f3740 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88f85f3740 20 client.25727 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff40b79f0 0x564ff44c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88f85f3740 4 client.25727 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f88dc0ad5c0 con 0x564ff40b79f0 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.890+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff40b79f0 0x564ff44c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.891+0000 7f88caffd640 10 client.25727 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.891+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff40b79f0 0x564ff44c6670 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a512aee2474f8b server_cookie=8c9420bf6f95d199 in_seq=0 out_seq=0 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88caffd640 1 -- 172.21.15.31:0/3183254086 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f88dc0ad5c0 con 0x564ff40b79f0 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88caffd640 10 client.25727 handle_command_reply: tid=0 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 client.25727 shutdown 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 msgr2=0x564ff44b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44b4dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00c6a0 tx=0x7f88dc004a90 comp rx=0 tx=0).stop 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 shutdown_connections 2024-09-21T06:14:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff40b79f0 0x564ff44c6670 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40f9070 0x564ff44ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3183254086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d01b0 0x564ff44b4dc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88caffd640 0 client.25727 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 >> 172.21.15.31:0/3183254086 conn(0x564ff40ed460 msgr2=0x564ff44a37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 shutdown_connections 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 1 -- 172.21.15.31:0/3183254086 wait complete. 2024-09-21T06:14:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.892+0000 7f88f85f3740 20 client.25727 trim_cache size 0 max 16384 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 Processor -- start 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 -- start start 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 0x564ff44afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40f9070 0x564ff44b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1ab4d0 con 0x564ff40f9070 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1adcd0 con 0x564ff40d8390 2024-09-21T06:14:02.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88f85f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1abb20 con 0x564ff40d01b0 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40f9070 0x564ff44b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 0x564ff44afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 0x564ff44afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54560/0 (socket says 172.21.15.31:54560) 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40f9070 0x564ff44b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56240/0 (socket says 172.21.15.31:56240) 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.914+0000 7f88e2ffd640 1 -- 172.21.15.31:0/3799919413 learned_addr learned my addr 172.21.15.31:0/3799919413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e2ffd640 1 -- 172.21.15.31:0/3799919413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 msgr2=0x564ff44ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e2ffd640 1 -- 172.21.15.31:0/3799919413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40f9070 msgr2=0x564ff44b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40f9070 0x564ff44b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e2ffd640 1 -- 172.21.15.31:0/3799919413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff44a2940 con 0x564ff40d8390 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 0x564ff44afd70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f88d40129b0 tx=0x7f88d4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a87c45424cefa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e17fa640 1 -- 172.21.15.31:0/3799919413 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d402b070 con 0x564ff40d8390 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e17fa640 1 -- 172.21.15.31:0/3799919413 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d40122d0 con 0x564ff40d8390 2024-09-21T06:14:02.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.915+0000 7f88e17fa640 1 -- 172.21.15.31:0/3799919413 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4010b70 con 0x564ff40d8390 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 -- 172.21.15.31:0/3799919413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 msgr2=0x564ff44afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 0x564ff44afd70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f88d40129b0 tx=0x7f88d4005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 -- 172.21.15.31:0/3799919413 shutdown_connections 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40f9070 0x564ff44b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40d8390 0x564ff44afd70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 --2- 172.21.15.31:0/3799919413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 -- 172.21.15.31:0/3799919413 >> 172.21.15.31:0/3799919413 conn(0x564ff40ed460 msgr2=0x564ff44992a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 -- 172.21.15.31:0/3799919413 shutdown_connections 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.916+0000 7f88f85f3740 1 -- 172.21.15.31:0/3799919413 wait complete. 2024-09-21T06:14:02.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 -- start start 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 Processor -- start 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40f9070 0x564ff4527da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1ac190 con 0x564ff40d8390 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1ad600 con 0x564ff40f9070 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1ac860 con 0x564ff40d01b0 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88f85f3740 10 client.0 _ll_register_callbacks cb 0x7f88d40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40f9070 0x564ff4527da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44be820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56252/0 (socket says 172.21.15.31:56252) 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.917+0000 7f88e2ffd640 1 -- 172.21.15.31:0/2379864082 learned_addr learned my addr 172.21.15.31:0/2379864082 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88e2ffd640 1 -- 172.21.15.31:0/2379864082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 msgr2=0x564ff44be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88e2ffd640 1 -- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40f9070 msgr2=0x564ff4527da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40f9070 0x564ff4527da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88e2ffd640 1 -- 172.21.15.31:0/2379864082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff4528410 con 0x564ff40d8390 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88e2ffd640 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44be820 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba3995274c35d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88caffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88caffd640 1 -- 172.21.15.31:0/2379864082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d402b040 con 0x564ff40d8390 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88caffd640 1 -- 172.21.15.31:0/2379864082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f88d40108f0 con 0x564ff40d8390 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d40122d0 con 0x564ff40d8390 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88caffd640 1 -- 172.21.15.31:0/2379864082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f88d4010b30 con 0x564ff40d8390 2024-09-21T06:14:02.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88f85f3740 10 client.15990 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.918+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564ff45308b0 con 0x564ff40d8390 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88caffd640 1 -- 172.21.15.31:0/2379864082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d40122d0 con 0x564ff40d8390 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88caffd640 1 -- 172.21.15.31:0/2379864082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f88d400fa90 con 0x564ff40d8390 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88f85f3740 10 client.15990 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88f85f3740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88f85f3740 20 client.15990 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88f85f3740 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff408cda0 0x564ff4533090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88f85f3740 4 client.15990 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ff4418da0 con 0x564ff408cda0 2024-09-21T06:14:02.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.919+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff408cda0 0x564ff4533090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:02.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88eb7fe640 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff408cda0 0x564ff4533090 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9a2fc27705747 server_cookie=25309b791f9c856c in_seq=0 out_seq=0 2024-09-21T06:14:02.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88caffd640 10 client.15990 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88caffd640 1 -- 172.21.15.31:0/2379864082 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x564ff4418da0 con 0x564ff408cda0 2024-09-21T06:14:02.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88caffd640 10 client.15990 handle_command_reply: tid=0 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 client.15990 shutdown 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 msgr2=0x564ff44be820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44be820 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4012500 comp rx=0 tx=0).stop 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 shutdown_connections 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ff408cda0 0x564ff4533090 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ff40f9070 0x564ff4527da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ff40d8390 0x564ff44be820 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 --2- 172.21.15.31:0/2379864082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ff40d01b0 0x564ff44be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88caffd640 0 client.15990 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 >> 172.21.15.31:0/2379864082 conn(0x564ff40ed460 msgr2=0x564ff4499280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 shutdown_connections 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.920+0000 7f88f85f3740 1 -- 172.21.15.31:0/2379864082 wait complete. 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f85f3740 20 client.15990 trim_cache size 0 max 16384 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:14:02.927 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.928 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.928 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.928 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.928 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.929 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.930 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:14:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:02.933 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:02.934 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:02.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:02.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:02.934 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88d80b6680 msgr2=0x7f88d80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88d80b6680 0x7f88d80b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009a60 tx=0x7f88e40095c0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 msgr2=0x7f88ec19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 0x7f88ec19c720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00e9f0 tx=0x7f88dc00edf0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 shutdown_connections 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88ec108d00 0x7f88ec1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88d80b6680 0x7f88d80b8b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88ec1083b0 0x7f88ec19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 --2- 172.21.15.31:0/3175636046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88ec102ac0 0x7f88ec19c720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:02.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 >> 172.21.15.31:0/3175636046 conn(0x7f88ec079790 msgr2=0x7f88ec06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:02.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 shutdown_connections 2024-09-21T06:14:02.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:02.921+0000 7f88f207f640 1 -- 172.21.15.31:0/3175636046 wait complete. 2024-09-21T06:14:03.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:03 smithi031 ceph-mon[31237]: pgmap v393: 97 pgs: 97 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2024-09-21T06:14:03.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:03 smithi110 ceph-mon[34550]: pgmap v393: 97 pgs: 97 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2024-09-21T06:14:03.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:03 smithi107 ceph-mon[34380]: pgmap v393: 97 pgs: 97 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 34 MiB/s wr, 1.07k op/s 2024-09-21T06:14:03.832 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:14:03.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:04.132 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:04.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 -- 172.21.15.31:0/4098692905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81901101e0 msgr2=0x7f8190112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 --2- 172.21.15.31:0/4098692905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81901101e0 0x7f8190112600 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8188009ca0 tx=0x7f818802a950 comp rx=0 tx=0).stop 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 -- 172.21.15.31:0/4098692905 shutdown_connections 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 --2- 172.21.15.31:0/4098692905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81901101e0 0x7f8190112600 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 --2- 172.21.15.31:0/4098692905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8190102f80 0x7f819010fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 --2- 172.21.15.31:0/4098692905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 0x7f8190102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 -- 172.21.15.31:0/4098692905 >> 172.21.15.31:0/4098692905 conn(0x7f8190079730 msgr2=0x7f8190079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 -- 172.21.15.31:0/4098692905 shutdown_connections 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.639+0000 7f8194e6f640 1 -- 172.21.15.31:0/4098692905 wait complete. 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.640+0000 7f8194e6f640 1 Processor -- start 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 -- start start 2024-09-21T06:14:04.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 0x7f819019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 0x7f819019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81901101e0 0x7f81901a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8190115c50 con 0x7f8190102f80 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8190115ad0 con 0x7f81901101e0 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f8194e6f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8190115dd0 con 0x7f8190102630 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f818dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 0x7f819019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f818dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 0x7f819019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56268/0 (socket says 172.21.15.31:56268) 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f818dd9b640 1 -- 172.21.15.31:0/1561095813 learned_addr learned my addr 172.21.15.31:0/1561095813 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f818e59c640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 0x7f819019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.641+0000 7f818ed9d640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81901101e0 0x7f81901a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818e59c640 1 -- 172.21.15.31:0/1561095813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81901101e0 msgr2=0x7f81901a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818e59c640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81901101e0 0x7f81901a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818e59c640 1 -- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 msgr2=0x7f819019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818e59c640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 0x7f819019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818e59c640 1 -- 172.21.15.31:0/1561095813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81901a2510 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818dd9b640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 0x7f819019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f818e59c640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 0x7f819019c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000ccc0 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81800108e0 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180018880 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81901a2800 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.642+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901a8f60 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.643+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8190102a40 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.645+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8180010430 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.646+0000 7f817f7fe640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81780b6700 0x7f81780b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.646+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f81800e7e40 con 0x7f8190102630 2024-09-21T06:14:04.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.646+0000 7f818dd9b640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81780b6700 0x7f81780b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.646+0000 7f818dd9b640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81780b6700 0x7f81780b8c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8190111ef0 tx=0x7f8184005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.649+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81800b27b0 con 0x7f8190102630 2024-09-21T06:14:04.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.915+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f819019d6d0 con 0x7f8190102630 2024-09-21T06:14:04.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.916+0000 7f817f7fe640 1 -- 172.21.15.31:0/1561095813 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f81800b76f0 con 0x7f8190102630 2024-09-21T06:14:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:14:04.918 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:14:04.918 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81780b6700 msgr2=0x7f81780b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81780b6700 0x7f81780b8c10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8190111ef0 tx=0x7f8184005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 msgr2=0x7f819019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 0x7f819019c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 shutdown_connections 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81901101e0 0x7f81901a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81780b6700 0x7f81780b8c10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8190102f80 0x7f819019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 --2- 172.21.15.31:0/1561095813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8190102630 0x7f819019c5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 >> 172.21.15.31:0/1561095813 conn(0x7f8190079730 msgr2=0x7f8190108c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:04.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 shutdown_connections 2024-09-21T06:14:04.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:04.920+0000 7f8194e6f640 1 -- 172.21.15.31:0/1561095813 wait complete. 2024-09-21T06:14:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 145.0 s, 348.6 tps, lat 262.520 ms stddev 223.158 2024-09-21T06:14:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:05 smithi110 ceph-mon[34550]: pgmap v394: 97 pgs: 97 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 31 MiB/s wr, 796 op/s 2024-09-21T06:14:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:05 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1561095813' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:05 smithi107 ceph-mon[34380]: pgmap v394: 97 pgs: 97 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 31 MiB/s wr, 796 op/s 2024-09-21T06:14:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:05 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1561095813' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:05.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:05 smithi031 ceph-mon[31237]: pgmap v394: 97 pgs: 97 active+clean; 2.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 31 MiB/s wr, 796 op/s 2024-09-21T06:14:05.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:05 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1561095813' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:05.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:14:06.126 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:06.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/2887691575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38108d10 msgr2=0x7f0b3810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/2887691575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38108d10 0x7f0b3810f540 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009900 tx=0x7f0b3002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:06.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/2887691575 shutdown_connections 2024-09-21T06:14:06.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/2887691575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38108d10 0x7f0b3810f540 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/2887691575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 0x7f0b381087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/2887691575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38102ad0 0x7f0b38102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/2887691575 >> 172.21.15.31:0/2887691575 conn(0x7f0b380796d0 msgr2=0x7f0b38079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/2887691575 shutdown_connections 2024-09-21T06:14:06.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.619+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/2887691575 wait complete. 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 Processor -- start 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 -- start start 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 0x7f0b3819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 0x7f0b3819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108d10 0x7f0b3819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381129d0 con 0x7f0b38102ad0 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38112850 con 0x7f0b381083c0 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3cdfa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38112b50 con 0x7f0b38108d10 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.620+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 0x7f0b3819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 0x7f0b3819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56294/0 (socket says 172.21.15.31:56294) 2024-09-21T06:14:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 -- 172.21.15.31:0/894528549 learned_addr learned my addr 172.21.15.31:0/894528549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 0x7f0b3819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108d10 0x7f0b3819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 -- 172.21.15.31:0/894528549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108d10 msgr2=0x7f0b3819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108d10 0x7f0b3819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 -- 172.21.15.31:0/894528549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 msgr2=0x7f0b3819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 0x7f0b3819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 -- 172.21.15.31:0/894528549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b381a6510 con 0x7f0b38102ad0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 0x7f0b3819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3659c640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 0x7f0b3819b280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800eb90 tx=0x7f0b2800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b277fe640 1 -- 172.21.15.31:0/894528549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28004d20 con 0x7f0b38102ad0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381a6800 con 0x7f0b38102ad0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b277fe640 1 -- 172.21.15.31:0/894528549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2800ee90 con 0x7f0b38102ad0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b277fe640 1 -- 172.21.15.31:0/894528549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28010670 con 0x7f0b38102ad0 2024-09-21T06:14:06.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.621+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a6d40 con 0x7f0b38102ad0 2024-09-21T06:14:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.624+0000 7f0b4336e740 1 Processor -- start 2024-09-21T06:14:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.624+0000 7f0b4336e740 1 -- start start 2024-09-21T06:14:06.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e17516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13811b0 0x5596e1751be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 0x5596e1752120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b4336e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5596e16c7ea0 con 0x5596e1389390 2024-09-21T06:14:06.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b4336e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5596e16b72d0 con 0x5596e13811b0 2024-09-21T06:14:06.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b4336e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5596e16bb0e0 con 0x5596e13aa070 2024-09-21T06:14:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b277fe640 1 -- 172.21.15.31:0/894528549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0b28005020 con 0x7f0b38102ad0 2024-09-21T06:14:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 0x5596e1752120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b36d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e17516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b36d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e17516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48288/0 (socket says 172.21.15.31:48288) 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 0x5596e1752120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56300/0 (socket says 172.21.15.31:56300) 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b36d9d640 1 -- 172.21.15.31:0/1033507970 learned_addr learned my addr 172.21.15.31:0/1033507970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b3659c640 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13811b0 0x5596e1751be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 -- 172.21.15.31:0/1033507970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 msgr2=0x5596e17516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e17516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 -- 172.21.15.31:0/1033507970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13811b0 msgr2=0x5596e1751be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13811b0 0x5596e1751be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b35d9b640 1 -- 172.21.15.31:0/1033507970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e1415650 con 0x5596e1389390 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e17516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:06.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.625+0000 7f0b3659c640 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13811b0 0x5596e1751be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 0x5596e1752120 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0b3806b6c0 tx=0x7f0b2c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a590612a3d774b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b24ff9640 1 -- 172.21.15.31:0/1033507970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c004230 con 0x5596e1389390 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b277fe640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b200b66b0 0x7f0b200b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b24ff9640 1 -- 172.21.15.31:0/1033507970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2c0043d0 con 0x5596e1389390 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b24ff9640 1 -- 172.21.15.31:0/1033507970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0035f0 con 0x5596e1389390 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b200b66b0 0x7f0b200b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.626+0000 7f0b277fe640 1 -- 172.21.15.31:0/894528549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0b280e7d40 con 0x7f0b38102ad0 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 -- 172.21.15.31:0/1033507970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 msgr2=0x5596e1752120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b200b66b0 0x7f0b200b8bc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b30031440 tx=0x7f0b300058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 0x5596e1752120 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0b3806b6c0 tx=0x7f0b2c0118d0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 -- 172.21.15.31:0/1033507970 shutdown_connections 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e1389390 0x5596e1752120 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13811b0 0x5596e1751be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1033507970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e17516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 -- 172.21.15.31:0/1033507970 >> 172.21.15.31:0/1033507970 conn(0x5596e139e460 msgr2=0x5596e174f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 -- 172.21.15.31:0/1033507970 shutdown_connections 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.627+0000 7f0b4336e740 1 -- 172.21.15.31:0/1033507970 wait complete. 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 -- start start 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 Processor -- start 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1761100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e1389390 0x5596e1761640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e1761ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5596e1518700 con 0x5596e13811b0 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5596e1652160 con 0x5596e1389390 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5596e17641e0 con 0x5596e13aa070 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1761100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b4336e740 10 client.0 _ll_register_callbacks cb 0x5596e166b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1761100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56312/0 (socket says 172.21.15.31:56312) 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b3659c640 1 -- 172.21.15.31:0/2473416039 learned_addr learned my addr 172.21.15.31:0/2473416039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e1389390 0x5596e1761640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.628+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e1761ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b3659c640 1 -- 172.21.15.31:0/2473416039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 msgr2=0x5596e1761ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b3659c640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e1761ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b3659c640 1 -- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e1389390 msgr2=0x5596e1761640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b3659c640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e1389390 0x5596e1761640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b3659c640 1 -- 172.21.15.31:0/2473416039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e167fc90 con 0x5596e13811b0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b3659c640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1761100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0b280049d0 tx=0x7f0b280b7520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc85a511021ccc31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2473416039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280a9a70 con 0x5596e13811b0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2473416039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0b280cfde0 con 0x5596e13811b0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b2c0043d0 con 0x5596e13811b0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2473416039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b280ad340 con 0x5596e13811b0 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b4336e740 10 client.16008 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:06.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.629+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5596e176ff30 con 0x5596e13811b0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2473416039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2c0043d0 con 0x5596e13811b0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2473416039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0b280ace30 con 0x5596e13811b0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b4336e740 10 client.16008 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b4336e740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b4336e740 20 client.16008 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e13689f0 0x5596e1773400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b4336e740 4 client.16008 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e16c9df0 con 0x5596e13689f0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.630+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e13689f0 0x5596e1773400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.631+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e13689f0 0x5596e1773400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee18aeb974cc3cd server_cookie=aef994b6cb416292 in_seq=0 out_seq=0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.631+0000 7f0b1dffb640 10 client.16008 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.631+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2473416039 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596e16c9df0 con 0x5596e13689f0 2024-09-21T06:14:06.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.631+0000 7f0b1dffb640 10 client.16008 handle_command_reply: tid=0 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 client.16008 shutdown 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 msgr2=0x5596e1761100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1761100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0b280049d0 tx=0x7f0b280b7520 comp rx=0 tx=0).stop 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 shutdown_connections 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e13689f0 0x5596e1773400 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e13aa070 0x5596e1761ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e1389390 0x5596e1761640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2473416039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1761100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 >> 172.21.15.31:0/2473416039 conn(0x5596e139e460 msgr2=0x5596e174dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b1dffb640 0 client.16008 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 shutdown_connections 2024-09-21T06:14:06.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 1 -- 172.21.15.31:0/2473416039 wait complete. 2024-09-21T06:14:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.632+0000 7f0b4336e740 20 client.16008 trim_cache size 0 max 16384 2024-09-21T06:14:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 Processor -- start 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 -- start start 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1760eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e17613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e175e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381ab440 con 0x5596e13811b0 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381adcc0 con 0x5596e13aa070 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b4336e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381abab0 con 0x5596e1389390 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1760eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b36d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e175e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1760eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56322/0 (socket says 172.21.15.31:56322) 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 -- 172.21.15.31:0/1582433542 learned_addr learned my addr 172.21.15.31:0/1582433542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e17613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 -- 172.21.15.31:0/1582433542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 msgr2=0x5596e17613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e17613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 -- 172.21.15.31:0/1582433542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 msgr2=0x5596e175e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e175e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b3659c640 1 -- 172.21.15.31:0/1582433542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e17544d0 con 0x5596e13811b0 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e17613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.646+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e175e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.647+0000 7f0b3659c640 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1760eb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800cba0 tx=0x7f0b280a9410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49e523c99db2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.647+0000 7f0b1ffff640 1 -- 172.21.15.31:0/1582433542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280ad520 con 0x5596e13811b0 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.647+0000 7f0b1ffff640 1 -- 172.21.15.31:0/1582433542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b280c7070 con 0x5596e13811b0 2024-09-21T06:14:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.647+0000 7f0b1ffff640 1 -- 172.21.15.31:0/1582433542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280ad9a0 con 0x5596e13811b0 2024-09-21T06:14:06.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 -- 172.21.15.31:0/1582433542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 msgr2=0x5596e1760eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1760eb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800cba0 tx=0x7f0b280a9410 comp rx=0 tx=0).stop 2024-09-21T06:14:06.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 -- 172.21.15.31:0/1582433542 shutdown_connections 2024-09-21T06:14:06.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e175e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e17613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 --2- 172.21.15.31:0/1582433542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1760eb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 -- 172.21.15.31:0/1582433542 >> 172.21.15.31:0/1582433542 conn(0x5596e139e460 msgr2=0x5596e175f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 -- 172.21.15.31:0/1582433542 shutdown_connections 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 -- 172.21.15.31:0/1582433542 wait complete. 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 -- start start 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.648+0000 7f0b4336e740 1 Processor -- start 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1765640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e1765b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e17660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381ac180 con 0x5596e13811b0 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381ad5f0 con 0x5596e13aa070 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b381ac850 con 0x5596e1389390 2024-09-21T06:14:06.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b4336e740 10 client.0 _ll_register_callbacks cb 0x7f0b280aa070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1765640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1765640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56334/0 (socket says 172.21.15.31:56334) 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b36d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e17660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 -- 172.21.15.31:0/2289003670 learned_addr learned my addr 172.21.15.31:0/2289003670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e1765b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 -- 172.21.15.31:0/2289003670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 msgr2=0x5596e1765b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e1765b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 -- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 msgr2=0x5596e17660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e17660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 -- 172.21.15.31:0/2289003670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e176d440 con 0x5596e13811b0 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b36d9d640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e17660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:06.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b3659c640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1765640 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0b280e5ad0 tx=0x7f0b280b7520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907c5ddce7fd98a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.649+0000 7f0b1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2289003670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b280ad9a0 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2289003670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0b280b21c0 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b280c7070 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2289003670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b280a9a70 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 10 client.16014 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5596e17ddf70 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2289003670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b280c7070 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2289003670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0b28005310 con 0x5596e13811b0 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 10 client.16014 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 20 client.16014 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:06.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e133dda0 0x5596e17e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 4 client.16014 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.650+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596e16c58a0 con 0x5596e133dda0 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e133dda0 0x5596e17e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b35d9b640 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e133dda0 0x5596e17e0470 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3fe30435a2fe7f4 server_cookie=b75ffb259f4592a0 in_seq=0 out_seq=0 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b1dffb640 10 client.16014 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b1dffb640 1 -- 172.21.15.31:0/2289003670 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596e16c58a0 con 0x5596e133dda0 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b1dffb640 10 client.16014 handle_command_reply: tid=0 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 client.16014 shutdown 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 msgr2=0x5596e1765640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1765640 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0b280e5ad0 tx=0x7f0b280b7520 comp rx=0 tx=0).stop 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 shutdown_connections 2024-09-21T06:14:06.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5596e133dda0 0x5596e17e0470 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5596e13aa070 0x5596e17660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596e1389390 0x5596e1765b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b1dffb640 0 client.16014 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 --2- 172.21.15.31:0/2289003670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596e13811b0 0x5596e1765640 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 >> 172.21.15.31:0/2289003670 conn(0x5596e139e460 msgr2=0x5596e17526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 shutdown_connections 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.651+0000 7f0b4336e740 1 -- 172.21.15.31:0/2289003670 wait complete. 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b4336e740 20 client.16014 trim_cache size 0 max 16384 2024-09-21T06:14:06.659 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b200b66b0 msgr2=0x7f0b200b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b200b66b0 0x7f0b200b8bc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b30031440 tx=0x7f0b300058d0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 msgr2=0x7f0b3819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 0x7f0b3819b280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800eb90 tx=0x7f0b2800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 shutdown_connections 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b38108d10 0x7f0b3819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b381083c0 0x7f0b3819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b200b66b0 0x7f0b200b8bc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 --2- 172.21.15.31:0/894528549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b38102ad0 0x7f0b3819b280 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 >> 172.21.15.31:0/894528549 conn(0x7f0b380796d0 msgr2=0x7f0b38104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 shutdown_connections 2024-09-21T06:14:06.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:06.652+0000 7f0b3cdfa640 1 -- 172.21.15.31:0/894528549 wait complete. 2024-09-21T06:14:07.346 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:07 smithi031 ceph-mon[31237]: pgmap v395: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 33 MiB/s wr, 1.01k op/s 2024-09-21T06:14:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:07 smithi110 ceph-mon[34550]: pgmap v395: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 33 MiB/s wr, 1.01k op/s 2024-09-21T06:14:07.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:07 smithi107 ceph-mon[34380]: pgmap v395: 97 pgs: 97 active+clean; 2.0 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 33 MiB/s wr, 1.01k op/s 2024-09-21T06:14:08.374 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:14:08.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:14:08.800 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:09.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 -- 172.21.15.31:0/3276849625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc628111f40 msgr2=0x7fc62810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 --2- 172.21.15.31:0/3276849625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc628111f40 0x7fc62810a3c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc618009900 tx=0x7fc61802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 -- 172.21.15.31:0/3276849625 shutdown_connections 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 --2- 172.21.15.31:0/3276849625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc62811cc30 0x7fc62811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 --2- 172.21.15.31:0/3276849625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62810a9e0 0x7fc62810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 --2- 172.21.15.31:0/3276849625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc628111f40 0x7fc62810a3c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.390+0000 7fc62de39640 1 -- 172.21.15.31:0/3276849625 >> 172.21.15.31:0/3276849625 conn(0x7fc628070ec0 msgr2=0x7fc6280712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.391+0000 7fc62de39640 1 -- 172.21.15.31:0/3276849625 shutdown_connections 2024-09-21T06:14:09.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.391+0000 7fc62de39640 1 -- 172.21.15.31:0/3276849625 wait complete. 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.391+0000 7fc62de39640 1 Processor -- start 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.391+0000 7fc62de39640 1 -- start start 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc62de39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 0x7fc6281a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc62de39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc628111f40 0x7fc6281a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc62de39640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62811cc30 0x7fc6281aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc62de39640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc628122900 con 0x7fc62810a9e0 2024-09-21T06:14:09.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc62de39640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc628122780 con 0x7fc62811cc30 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc62de39640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc628122a80 con 0x7fc628111f40 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 0x7fc6281a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 0x7fc6281a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37406/0 (socket says 172.21.15.31:37406) 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 -- 172.21.15.31:0/893316362 learned_addr learned my addr 172.21.15.31:0/893316362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc61ffff640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc628111f40 0x7fc6281a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc627fff640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62811cc30 0x7fc6281aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 -- 172.21.15.31:0/893316362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc628111f40 msgr2=0x7fc6281a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc628111f40 0x7fc6281a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 -- 172.21.15.31:0/893316362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62811cc30 msgr2=0x7fc6281aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62811cc30 0x7fc6281aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc6277fe640 1 -- 172.21.15.31:0/893316362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6281af2a0 con 0x7fc62810a9e0 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc627fff640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62811cc30 0x7fc6281aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:09.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.392+0000 7fc61ffff640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc628111f40 0x7fc6281a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:09.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.393+0000 7fc6277fe640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 0x7fc6281a9330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc618005ab0 tx=0x7fc618003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.393+0000 7fc6257fa640 1 -- 172.21.15.31:0/893316362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61803f070 con 0x7fc62810a9e0 2024-09-21T06:14:09.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.393+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6281af530 con 0x7fc62810a9e0 2024-09-21T06:14:09.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.393+0000 7fc6257fa640 1 -- 172.21.15.31:0/893316362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc618031d60 con 0x7fc62810a9e0 2024-09-21T06:14:09.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.393+0000 7fc6257fa640 1 -- 172.21.15.31:0/893316362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618035030 con 0x7fc62810a9e0 2024-09-21T06:14:09.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.393+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6281b5cf0 con 0x7fc62810a9e0 2024-09-21T06:14:09.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.396+0000 7fc6343ad740 1 Processor -- start 2024-09-21T06:14:09.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 -- start start 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e49f350 0x55dc8e846430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e846970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 0x55dc8e846eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dc8e7bce60 con 0x55dc8e49f350 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc8e7ac290 con 0x55dc8e47e670 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc6343ad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc8e7b00a0 con 0x55dc8e476490 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 0x55dc8e846eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 0x55dc8e846eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43320/0 (socket says 172.21.15.31:43320) 2024-09-21T06:14:09.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 -- 172.21.15.31:0/2630530727 learned_addr learned my addr 172.21.15.31:0/2630530727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 -- 172.21.15.31:0/2630530727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 msgr2=0x55dc8e846970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e846970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 -- 172.21.15.31:0/2630530727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e49f350 msgr2=0x55dc8e846430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e49f350 0x55dc8e846430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.397+0000 7fc61ffff640 1 -- 172.21.15.31:0/2630530727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8e50a930 con 0x55dc8e47e670 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.398+0000 7fc61ffff640 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 0x55dc8e846eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc610011470 tx=0x7fc610011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77359278dfacbe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.398+0000 7fc6257fa640 1 -- 172.21.15.31:0/893316362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc618042070 con 0x7fc62810a9e0 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.398+0000 7fc61dffb640 1 -- 172.21.15.31:0/2630530727 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610004310 con 0x55dc8e47e670 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.398+0000 7fc61dffb640 1 -- 172.21.15.31:0/2630530727 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6100044b0 con 0x55dc8e47e670 2024-09-21T06:14:09.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.398+0000 7fc61dffb640 1 -- 172.21.15.31:0/2630530727 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100035f0 con 0x55dc8e47e670 2024-09-21T06:14:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6257fa640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6140b6880 0x7fc6140b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6257fa640 1 -- 172.21.15.31:0/893316362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc61803bb40 con 0x7fc62810a9e0 2024-09-21T06:14:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc627fff640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6140b6880 0x7fc6140b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6343ad740 1 -- 172.21.15.31:0/2630530727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 msgr2=0x55dc8e846eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 0x55dc8e846eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc610011470 tx=0x7fc610011930 comp rx=0 tx=0).stop 2024-09-21T06:14:09.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc627fff640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6140b6880 0x7fc6140b8d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc6200094a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6343ad740 1 -- 172.21.15.31:0/2630530727 shutdown_connections 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e47e670 0x55dc8e846eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e846970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.399+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2630530727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e49f350 0x55dc8e846430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.400+0000 7fc6343ad740 1 -- 172.21.15.31:0/2630530727 >> 172.21.15.31:0/2630530727 conn(0x55dc8e493740 msgr2=0x55dc8e844320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.400+0000 7fc6343ad740 1 -- 172.21.15.31:0/2630530727 shutdown_connections 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.400+0000 7fc6343ad740 1 -- 172.21.15.31:0/2630530727 wait complete. 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.400+0000 7fc6343ad740 1 -- start start 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 Processor -- start 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e8520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 0x55dc8e8526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e49f350 0x55dc8e85df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dc8e60d890 con 0x55dc8e476490 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dc8e746d40 con 0x55dc8e49f350 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc8e854c10 con 0x55dc8e47e670 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6343ad740 10 client.0 _ll_register_callbacks cb 0x55dc8e7aac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 0x55dc8e8526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e8520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 0x55dc8e8526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47318/0 (socket says 172.21.15.31:47318) 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc61ffff640 1 -- 172.21.15.31:0/2592252107 learned_addr learned my addr 172.21.15.31:0/2592252107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc6277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e8520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37432/0 (socket says 172.21.15.31:37432) 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.401+0000 7fc627fff640 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e49f350 0x55dc8e85df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc61ffff640 1 -- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e49f350 msgr2=0x55dc8e85df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc61ffff640 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e49f350 0x55dc8e85df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc61ffff640 1 -- 172.21.15.31:0/2592252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 msgr2=0x55dc8e8520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc61ffff640 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e8520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc61ffff640 1 -- 172.21.15.31:0/2592252107 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8e85e5a0 con 0x55dc8e47e670 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc61ffff640 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 0x55dc8e8526f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc610009350 tx=0x7fc6100048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c54a78ff2374d192 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc606ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc606ffd640 1 -- 172.21.15.31:0/2592252107 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610029070 con 0x55dc8e47e670 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6100044b0 con 0x55dc8e47e670 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc606ffd640 1 -- 172.21.15.31:0/2592252107 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc610003a80 con 0x55dc8e47e670 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc606ffd640 1 -- 172.21.15.31:0/2592252107 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc610035050 con 0x55dc8e47e670 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc6343ad740 10 client.25795 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.402+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dc8e860ed0 con 0x55dc8e47e670 2024-09-21T06:14:09.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc606ffd640 1 -- 172.21.15.31:0/2592252107 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6100044b0 con 0x55dc8e47e670 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc606ffd640 1 -- 172.21.15.31:0/2592252107 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc610004c60 con 0x55dc8e47e670 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6343ad740 10 client.25795 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6343ad740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6343ad740 20 client.25795 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e45dcd0 0x55dc8e864290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6343ad740 4 client.25795 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc8e7bedb0 con 0x55dc8e45dcd0 2024-09-21T06:14:09.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.403+0000 7fc6277fe640 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e45dcd0 0x55dc8e864290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.404+0000 7fc606ffd640 10 client.25795 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.404+0000 7fc6277fe640 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e45dcd0 0x55dc8e864290 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da8ac1fc2d735dc server_cookie=12556766009969f1 in_seq=0 out_seq=0 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.404+0000 7fc606ffd640 1 -- 172.21.15.31:0/2592252107 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc8e7bedb0 con 0x55dc8e45dcd0 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.404+0000 7fc606ffd640 10 client.25795 handle_command_reply: tid=0 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.405+0000 7fc6343ad740 1 client.25795 shutdown 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.405+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 msgr2=0x55dc8e8526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.405+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 0x55dc8e8526f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc610009350 tx=0x7fc6100048d0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 shutdown_connections 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e45dcd0 0x55dc8e864290 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc606ffd640 0 client.25795 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e49f350 0x55dc8e85df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e47e670 0x55dc8e8526f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 --2- 172.21.15.31:0/2592252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e8520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 >> 172.21.15.31:0/2592252107 conn(0x55dc8e493740 msgr2=0x55dc8e849610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 shutdown_connections 2024-09-21T06:14:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.406+0000 7fc6343ad740 1 -- 172.21.15.31:0/2592252107 wait complete. 2024-09-21T06:14:09.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.407+0000 7fc6343ad740 20 client.25795 trim_cache size 0 max 16384 2024-09-21T06:14:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 Processor -- start 2024-09-21T06:14:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 -- start start 2024-09-21T06:14:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e493740 0x55dc8e85de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 0x55dc8e844ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e845420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281bc4a0 con 0x55dc8e45dcd0 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281bed20 con 0x55dc8e493740 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.428+0000 7fc6343ad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281bcb10 con 0x55dc8e476490 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 0x55dc8e844ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 0x55dc8e844ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37446/0 (socket says 172.21.15.31:37446) 2024-09-21T06:14:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 -- 172.21.15.31:0/400310099 learned_addr learned my addr 172.21.15.31:0/400310099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc627fff640 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e845420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 -- 172.21.15.31:0/400310099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 msgr2=0x55dc8e845420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e845420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 -- 172.21.15.31:0/400310099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e493740 msgr2=0x55dc8e85de30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e493740 0x55dc8e85de30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 -- 172.21.15.31:0/400310099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8e848b20 con 0x55dc8e45dcd0 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.429+0000 7fc61ffff640 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 0x55dc8e844ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc610005f90 tx=0x7fc610005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e86ce4a0e898b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.430+0000 7fc61d7fa640 1 -- 172.21.15.31:0/400310099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610029070 con 0x55dc8e45dcd0 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.430+0000 7fc61d7fa640 1 -- 172.21.15.31:0/400310099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc610003d30 con 0x55dc8e45dcd0 2024-09-21T06:14:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.430+0000 7fc61d7fa640 1 -- 172.21.15.31:0/400310099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61000f7a0 con 0x55dc8e45dcd0 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 -- 172.21.15.31:0/400310099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 msgr2=0x55dc8e844ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 0x55dc8e844ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc610005f90 tx=0x7fc610005fc0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 -- 172.21.15.31:0/400310099 shutdown_connections 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e476490 0x55dc8e845420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e45dcd0 0x55dc8e844ee0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 --2- 172.21.15.31:0/400310099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e493740 0x55dc8e85de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 -- 172.21.15.31:0/400310099 >> 172.21.15.31:0/400310099 conn(0x55dc8e844360 msgr2=0x55dc8e850770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 -- 172.21.15.31:0/400310099 shutdown_connections 2024-09-21T06:14:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.431+0000 7fc6343ad740 1 -- 172.21.15.31:0/400310099 wait complete. 2024-09-21T06:14:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 -- start start 2024-09-21T06:14:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 Processor -- start 2024-09-21T06:14:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e45dcd0 0x55dc8e85a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e856a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e493740 0x55dc8e856fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281bd1e0 con 0x55dc8e476490 2024-09-21T06:14:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281be650 con 0x55dc8e45dcd0 2024-09-21T06:14:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281bd8b0 con 0x55dc8e493740 2024-09-21T06:14:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.432+0000 7fc6343ad740 10 client.0 _ll_register_callbacks cb 0x7fc610009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e856a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e856a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37462/0 (socket says 172.21.15.31:37462) 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 -- 172.21.15.31:0/3911877707 learned_addr learned my addr 172.21.15.31:0/3911877707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc627fff640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e493740 0x55dc8e856fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc6277fe640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e45dcd0 0x55dc8e85a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 -- 172.21.15.31:0/3911877707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e493740 msgr2=0x55dc8e856fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e493740 0x55dc8e856fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 -- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e45dcd0 msgr2=0x55dc8e85a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e45dcd0 0x55dc8e85a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 -- 172.21.15.31:0/3911877707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8e8576c0 con 0x55dc8e476490 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc61ffff640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e856a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc610004fb0 tx=0x7fc6100049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c75f0bb146b097 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:09.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.433+0000 7fc606ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc606ffd640 1 -- 172.21.15.31:0/3911877707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610029040 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc606ffd640 1 -- 172.21.15.31:0/3911877707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc610003a90 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc610003d30 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc606ffd640 1 -- 172.21.15.31:0/3911877707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc61000f540 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc6343ad740 10 client.16029 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dc8e8cf9e0 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc606ffd640 1 -- 172.21.15.31:0/3911877707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc610003d30 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc606ffd640 1 -- 172.21.15.31:0/3911877707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc610042480 con 0x55dc8e476490 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc6343ad740 10 client.16029 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc6343ad740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.434+0000 7fc6343ad740 20 client.16029 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:09.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.435+0000 7fc6343ad740 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e47e670 0x55dc8e8d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:09.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.435+0000 7fc6343ad740 4 client.16029 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:14:09.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.435+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dc8e7ba860 con 0x55dc8e47e670 2024-09-21T06:14:09.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.435+0000 7fc6277fe640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e47e670 0x55dc8e8d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:09.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.435+0000 7fc6277fe640 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e47e670 0x55dc8e8d2100 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12250f77b5a0f34e server_cookie=aaabc570bbb26046 in_seq=0 out_seq=0 2024-09-21T06:14:09.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.435+0000 7fc606ffd640 10 client.16029 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:09.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc606ffd640 1 -- 172.21.15.31:0/3911877707 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dc8e7ba860 con 0x55dc8e47e670 2024-09-21T06:14:09.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc606ffd640 10 client.16029 handle_command_reply: tid=0 2024-09-21T06:14:09.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 client.16029 shutdown 2024-09-21T06:14:09.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 msgr2=0x55dc8e856a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e856a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc610004fb0 tx=0x7fc6100049c0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 shutdown_connections 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dc8e47e670 0x55dc8e8d2100 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc8e493740 0x55dc8e856fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc8e476490 0x55dc8e856a90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 --2- 172.21.15.31:0/3911877707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc8e45dcd0 0x55dc8e85a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc606ffd640 0 client.16029 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 >> 172.21.15.31:0/3911877707 conn(0x55dc8e844360 msgr2=0x55dc8e847460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.440+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 shutdown_connections 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.441+0000 7fc6343ad740 1 -- 172.21.15.31:0/3911877707 wait complete. 2024-09-21T06:14:09.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.441+0000 7fc6343ad740 20 client.16029 trim_cache size 0 max 16384 2024-09-21T06:14:09.444 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:14:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "57de4f22-440b-403d-a8c2-c8f5c072e14d", 2024-09-21T06:14:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:14:09.444 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.441+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6140b6880 msgr2=0x7fc6140b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.441+0000 7fc62de39640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6140b6880 0x7fc6140b8d90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc620005fd0 tx=0x7fc6200094a0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.441+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 msgr2=0x7fc6281a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.441+0000 7fc62de39640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 0x7fc6281a9330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc618005ab0 tx=0x7fc618003c80 comp rx=0 tx=0).stop 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 shutdown_connections 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62811cc30 0x7fc6281aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc628111f40 0x7fc6281a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc6140b6880 0x7fc6140b8d90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 --2- 172.21.15.31:0/893316362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc62810a9e0 0x7fc6281a9330 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 >> 172.21.15.31:0/893316362 conn(0x7fc628070ec0 msgr2=0x7fc628071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 shutdown_connections 2024-09-21T06:14:09.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:09.442+0000 7fc62de39640 1 -- 172.21.15.31:0/893316362 wait complete. 2024-09-21T06:14:09.502 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:09 smithi031 ceph-mon[31237]: pgmap v396: 97 pgs: 97 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 27 MiB/s wr, 813 op/s 2024-09-21T06:14:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:09 smithi110 ceph-mon[34550]: pgmap v396: 97 pgs: 97 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 27 MiB/s wr, 813 op/s 2024-09-21T06:14:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:09 smithi107 ceph-mon[34380]: pgmap v396: 97 pgs: 97 active+clean; 2.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 27 MiB/s wr, 813 op/s 2024-09-21T06:14:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 150.0 s, 1329.6 tps, lat 81.365 ms stddev 361.648 2024-09-21T06:14:10.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:14:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:10 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:14:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:10 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:10 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:14:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:10 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:14:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:10 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:10 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:14:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:10 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:14:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:10 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:10.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:10 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:14:10.678 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 -- 172.21.15.31:0/3705511005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 msgr2=0x7f13b4107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 --2- 172.21.15.31:0/3705511005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b4107640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009860 tx=0x7f13a402a340 comp rx=0 tx=0).stop 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 -- 172.21.15.31:0/3705511005 shutdown_connections 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 --2- 172.21.15.31:0/3705511005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b4107c00 0x7f13b41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 --2- 172.21.15.31:0/3705511005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b4107640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 --2- 172.21.15.31:0/3705511005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b41027c0 0x7f13b4102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 -- 172.21.15.31:0/3705511005 >> 172.21.15.31:0/3705511005 conn(0x7f13b4079740 msgr2=0x7f13b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 -- 172.21.15.31:0/3705511005 shutdown_connections 2024-09-21T06:14:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.271+0000 7f13bbc44640 1 -- 172.21.15.31:0/3705511005 wait complete. 2024-09-21T06:14:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.272+0000 7f13bbc44640 1 Processor -- start 2024-09-21T06:14:11.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.272+0000 7f13bbc44640 1 -- start start 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13bbc44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b41027c0 0x7f13b41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13bbc44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13bbc44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b4107c00 0x7f13b41a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13bbc44640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13b411a060 con 0x7f13b41027c0 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13bbc44640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4119ee0 con 0x7f13b4107c00 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13bbc44640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13b411a1e0 con 0x7f13b4103110 2024-09-21T06:14:11.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b99e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b41027c0 0x7f13b41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b99e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b41027c0 0x7f13b41a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37482/0 (socket says 172.21.15.31:37482) 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b91df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b99e0640 1 -- 172.21.15.31:0/1734834124 learned_addr learned my addr 172.21.15.31:0/1734834124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b4107c00 0x7f13b41a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b91df640 1 -- 172.21.15.31:0/1734834124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b4107c00 msgr2=0x7f13b41a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b91df640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b4107c00 0x7f13b41a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b91df640 1 -- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b41027c0 msgr2=0x7f13b41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b91df640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b41027c0 0x7f13b41a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.273+0000 7f13b91df640 1 -- 172.21.15.31:0/1734834124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b41a6a20 con 0x7f13b4103110 2024-09-21T06:14:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.274+0000 7f13b91df640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b41a1050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009780 tx=0x7f13a4032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.274+0000 7f13aaffd640 1 -- 172.21.15.31:0/1734834124 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4037040 con 0x7f13b4103110 2024-09-21T06:14:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.274+0000 7f13aaffd640 1 -- 172.21.15.31:0/1734834124 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4030c00 con 0x7f13b4103110 2024-09-21T06:14:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.274+0000 7f13aaffd640 1 -- 172.21.15.31:0/1734834124 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4003850 con 0x7f13b4103110 2024-09-21T06:14:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.275+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b41a6cb0 con 0x7f13b4103110 2024-09-21T06:14:11.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.275+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41a70b0 con 0x7f13b4103110 2024-09-21T06:14:11.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.278+0000 7f13c21b8740 1 Processor -- start 2024-09-21T06:14:11.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.279+0000 7f13c21b8740 1 -- start start 2024-09-21T06:14:11.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.279+0000 7f13aaffd640 1 -- 172.21.15.31:0/1734834124 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f13a4043020 con 0x7f13b4103110 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.279+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff6f070 0x561240316650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.279+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240316b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.279+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x5612403170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13c21b8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56124028ce50 con 0x56123ff461b0 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13c21b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56124027c280 con 0x56123ff6f070 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13c21b8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561240280090 con 0x56123ff4e390 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b99e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240316b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b91df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x5612403170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b99e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240316b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37492/0 (socket says 172.21.15.31:37492) 2024-09-21T06:14:11.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b99e0640 1 -- 172.21.15.31:0/4078917176 learned_addr learned my addr 172.21.15.31:0/4078917176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:11.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff6f070 0x561240316650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b91df640 1 -- 172.21.15.31:0/4078917176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff6f070 msgr2=0x561240316650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b91df640 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff6f070 0x561240316650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b91df640 1 -- 172.21.15.31:0/4078917176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 msgr2=0x561240316b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b91df640 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240316b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.280+0000 7f13b91df640 1 -- 172.21.15.31:0/4078917176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123ffda650 con 0x56123ff4e390 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.281+0000 7f13b91df640 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x5612403170d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13a4103d20 tx=0x7f13a410c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb1b963513fef96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.281+0000 7f139ffff640 1 -- 172.21.15.31:0/4078917176 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a410ce60 con 0x56123ff4e390 2024-09-21T06:14:11.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.281+0000 7f139ffff640 1 -- 172.21.15.31:0/4078917176 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4113030 con 0x56123ff4e390 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.281+0000 7f139ffff640 1 -- 172.21.15.31:0/4078917176 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4118880 con 0x56123ff4e390 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.281+0000 7f13aaffd640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f13a00b6700 0x7f13a00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.282+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f13a00b6700 0x7f13a00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.282+0000 7f13aaffd640 1 -- 172.21.15.31:0/1734834124 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f13a41065f0 con 0x7f13b4103110 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 -- 172.21.15.31:0/4078917176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 msgr2=0x5612403170d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x5612403170d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13a4103d20 tx=0x7f13a410c660 comp rx=0 tx=0).stop 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f13a00b6700 0x7f13a00b8c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13ac005fd0 tx=0x7f13ac010040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 -- 172.21.15.31:0/4078917176 shutdown_connections 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x5612403170d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240316b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 --2- 172.21.15.31:0/4078917176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff6f070 0x561240316650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 -- 172.21.15.31:0/4078917176 >> 172.21.15.31:0/4078917176 conn(0x56123ff63460 msgr2=0x561240314540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 -- 172.21.15.31:0/4078917176 shutdown_connections 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.283+0000 7f13c21b8740 1 -- 172.21.15.31:0/4078917176 wait complete. 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.284+0000 7f13c21b8740 1 -- start start 2024-09-21T06:14:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.284+0000 7f13c21b8740 1 Processor -- start 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff461b0 0x561240321d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240322260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 0x56124032de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5612400dd700 con 0x56123ff6f070 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561240217110 con 0x56123ff4e390 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561240324bf0 con 0x56123ff461b0 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13c21b8740 10 client.0 _ll_register_callbacks cb 0x561240230990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 0x56124032de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 0x56124032de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37502/0 (socket says 172.21.15.31:37502) 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1383527563 learned_addr learned my addr 172.21.15.31:0/1383527563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13b99e0640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff461b0 0x561240321d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13b91df640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240322260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1383527563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff461b0 msgr2=0x561240321d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff461b0 0x561240321d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 msgr2=0x561240322260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240322260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1383527563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561240244c40 con 0x56123ff6f070 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13b99e0640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff461b0 0x561240321d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.285+0000 7f13b91df640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240322260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.286+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 0x56124032de50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0045c0 tx=0x7f13ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20752cbf57c550cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.286+0000 7f139d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.286+0000 7f139d7fa640 1 -- 172.21.15.31:0/1383527563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac020040 con 0x56123ff6f070 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.286+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a4113030 con 0x56123ff6f070 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.286+0000 7f139d7fa640 1 -- 172.21.15.31:0/1383527563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f13ac00dce0 con 0x56123ff6f070 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f139d7fa640 1 -- 172.21.15.31:0/1383527563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f13ac032050 con 0x56123ff6f070 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 10 client.16041 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561240330890 con 0x56123ff6f070 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f139d7fa640 1 -- 172.21.15.31:0/1383527563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4113030 con 0x56123ff6f070 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f139d7fa640 1 -- 172.21.15.31:0/1383527563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f13ac00b040 con 0x56123ff6f070 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 10 client.16041 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 20 client.16041 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff2d9f0 0x561240333eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 4 client.16041 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.287+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56124028eda0 con 0x56123ff2d9f0 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.288+0000 7f13b99e0640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff2d9f0 0x561240333eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.289+0000 7f139d7fa640 10 client.16041 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.289+0000 7f13b99e0640 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff2d9f0 0x561240333eb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2173c8e06b43565c server_cookie=2b81e1e40a23a7d5 in_seq=0 out_seq=0 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f139d7fa640 1 -- 172.21.15.31:0/1383527563 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56124028eda0 con 0x56123ff2d9f0 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f139d7fa640 10 client.16041 handle_command_reply: tid=0 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f13c21b8740 1 client.16041 shutdown 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 msgr2=0x56124032de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 0x56124032de50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0045c0 tx=0x7f13ac004ab0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 shutdown_connections 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.290+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff2d9f0 0x561240333eb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.291+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff6f070 0x56124032de50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.291+0000 7f139d7fa640 0 client.16041 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.291+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240322260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.291+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1383527563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff461b0 0x561240321d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.291+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 >> 172.21.15.31:0/1383527563 conn(0x56123ff63460 msgr2=0x561240319830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.291+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 shutdown_connections 2024-09-21T06:14:11.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.292+0000 7f13c21b8740 1 -- 172.21.15.31:0/1383527563 wait complete. 2024-09-21T06:14:11.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.292+0000 7f13c21b8740 20 client.16041 trim_cache size 0 max 16384 2024-09-21T06:14:11.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 Processor -- start 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 -- start start 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff2d9f0 0x561240323c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240317d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x561240313b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b3c20 con 0x56123ff461b0 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b64a0 con 0x56123ff2d9f0 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13c21b8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b4290 con 0x56123ff4e390 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13ba1e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x561240313b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13ba1e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x561240313b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47374/0 (socket says 172.21.15.31:47374) 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1443633648 learned_addr learned my addr 172.21.15.31:0/1443633648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.323+0000 7f13b91df640 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240317d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13b99e0640 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff2d9f0 0x561240323c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1443633648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff2d9f0 msgr2=0x561240323c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff2d9f0 0x561240323c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1443633648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 msgr2=0x561240317d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240317d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13ba1e1640 1 -- 172.21.15.31:0/1443633648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561240318970 con 0x56123ff4e390 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13ba1e1640 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x561240313b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f13ac024700 tx=0x7f13ac009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b741d4bca8e6fecd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.324+0000 7f13b99e0640 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff2d9f0 0x561240323c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.325+0000 7f139f7fe640 1 -- 172.21.15.31:0/1443633648 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac020040 con 0x56123ff4e390 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.325+0000 7f139f7fe640 1 -- 172.21.15.31:0/1443633648 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac029070 con 0x56123ff4e390 2024-09-21T06:14:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.325+0000 7f139f7fe640 1 -- 172.21.15.31:0/1443633648 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac01ad80 con 0x56123ff4e390 2024-09-21T06:14:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 -- 172.21.15.31:0/1443633648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 msgr2=0x561240313b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x561240313b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f13ac024700 tx=0x7f13ac009ab0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 -- 172.21.15.31:0/1443633648 shutdown_connections 2024-09-21T06:14:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff4e390 0x561240313b70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240317d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 --2- 172.21.15.31:0/1443633648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff2d9f0 0x561240323c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 -- 172.21.15.31:0/1443633648 >> 172.21.15.31:0/1443633648 conn(0x56123ff63460 msgr2=0x561240317680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 -- 172.21.15.31:0/1443633648 shutdown_connections 2024-09-21T06:14:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.326+0000 7f13c21b8740 1 -- 172.21.15.31:0/1443633648 wait complete. 2024-09-21T06:14:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.327+0000 7f13c21b8740 1 -- start start 2024-09-21T06:14:11.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.328+0000 7f13c21b8740 1 Processor -- start 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 0x56124032a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240325950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240325e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b4960 con 0x56123ff461b0 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b5dd0 con 0x56123ff4e390 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41b5030 con 0x56123ff2d9f0 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b91df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240325950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13c21b8740 10 client.0 _ll_register_callbacks cb 0x7f13ac01d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b99e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 0x56124032a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b99e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 0x56124032a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47384/0 (socket says 172.21.15.31:47384) 2024-09-21T06:14:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b99e0640 1 -- 172.21.15.31:0/2518092007 learned_addr learned my addr 172.21.15.31:0/2518092007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:11.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b91df640 1 -- 172.21.15.31:0/2518092007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 msgr2=0x56124032a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:11.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b91df640 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 0x56124032a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b91df640 1 -- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 msgr2=0x561240325e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:14:11.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b91df640 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240325e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.329+0000 7f13b91df640 1 -- 172.21.15.31:0/2518092007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561240326530 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f13b99e0640 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 0x56124032a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f13b91df640 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240325950 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009830 tx=0x7f13a4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c85dc07de6f0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f139d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f139d7fa640 1 -- 172.21.15.31:0/2518092007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40c8b30 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac029070 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f139d7fa640 1 -- 172.21.15.31:0/2518092007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f13a40328a0 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f139d7fa640 1 -- 172.21.15.31:0/2518092007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f13a41041b0 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f13c21b8740 10 client.16047 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.330+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56124039e760 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f139d7fa640 1 -- 172.21.15.31:0/2518092007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac029070 con 0x56123ff461b0 2024-09-21T06:14:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f139d7fa640 1 -- 172.21.15.31:0/2518092007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f13a40c6be0 con 0x56123ff461b0 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13c21b8740 10 client.16047 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13c21b8740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13c21b8740 20 client.16047 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13c21b8740 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff6f070 0x5612403a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13c21b8740 4 client.16047 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56124028a850 con 0x56123ff6f070 2024-09-21T06:14:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.331+0000 7f13b99e0640 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff6f070 0x5612403a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.332+0000 7f139d7fa640 10 client.16047 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13b99e0640 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff6f070 0x5612403a0f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855f71e364e068f7 server_cookie=60712ef74bc1a391 in_seq=0 out_seq=0 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f139d7fa640 1 -- 172.21.15.31:0/2518092007 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x56124028a850 con 0x56123ff6f070 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f139d7fa640 10 client.16047 handle_command_reply: tid=0 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 client.16047 shutdown 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 msgr2=0x561240325950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240325950 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009830 tx=0x7f13a4005760 comp rx=0 tx=0).stop 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 shutdown_connections 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56123ff6f070 0x5612403a0f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56123ff4e390 0x561240325e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56123ff461b0 0x561240325950 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 --2- 172.21.15.31:0/2518092007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56123ff2d9f0 0x56124032a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f139d7fa640 0 client.16047 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 >> 172.21.15.31:0/2518092007 conn(0x56123ff63460 msgr2=0x561240318040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 shutdown_connections 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.333+0000 7f13c21b8740 1 -- 172.21.15.31:0/2518092007 wait complete. 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.334+0000 7f13c21b8740 20 client.16047 trim_cache size 0 max 16384 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:14:11.339 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "57de4f22-440b-403d-a8c2-c8f5c072e14d", 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.342 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:14:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f13a00b6700 msgr2=0x7f13a00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f13a00b6700 0x7f13a00b8c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13ac005fd0 tx=0x7f13ac010040 comp rx=0 tx=0).stop 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 msgr2=0x7f13b41a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b41a1050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009780 tx=0x7f13a4032d50 comp rx=0 tx=0).stop 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 shutdown_connections 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b4107c00 0x7f13b41a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b4103110 0x7f13b41a1050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f13a00b6700 0x7f13a00b8c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 --2- 172.21.15.31:0/1734834124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b41027c0 0x7f13b41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.335+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 >> 172.21.15.31:0/1734834124 conn(0x7f13b4079740 msgr2=0x7f13b4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.336+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 shutdown_connections 2024-09-21T06:14:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:11.336+0000 7f13bbc44640 1 -- 172.21.15.31:0/1734834124 wait complete. 2024-09-21T06:14:11.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:11 smithi031 ceph-mon[31237]: pgmap v397: 97 pgs: 97 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 34 MiB/s wr, 993 op/s 2024-09-21T06:14:11.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:11 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:11 smithi110 ceph-mon[34550]: pgmap v397: 97 pgs: 97 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 34 MiB/s wr, 993 op/s 2024-09-21T06:14:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:11 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:11 smithi107 ceph-mon[34380]: pgmap v397: 97 pgs: 97 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 34 MiB/s wr, 993 op/s 2024-09-21T06:14:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:11 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:12.410 INFO:tasks.cephfs.filesystem:scrub status for tag:57de4f22-440b-403d-a8c2-c8f5c072e14d - {'path': '/', 'tag': '57de4f22-440b-403d-a8c2-c8f5c072e14d', 'options': 'recursive,force'} 2024-09-21T06:14:12.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:12 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:14:12.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:12 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:14:12.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:12 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:14:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:13 smithi107 ceph-mon[34380]: pgmap v398: 97 pgs: 97 active+clean; 2.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 61 MiB/s wr, 1.78k op/s 2024-09-21T06:14:13.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:13 smithi031 ceph-mon[31237]: pgmap v398: 97 pgs: 97 active+clean; 2.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 61 MiB/s wr, 1.78k op/s 2024-09-21T06:14:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:13 smithi110 ceph-mon[34550]: pgmap v398: 97 pgs: 97 active+clean; 2.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 61 MiB/s wr, 1.78k op/s 2024-09-21T06:14:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 155.0 s, 3644.0 tps, lat 27.507 ms stddev 32.997 2024-09-21T06:14:15.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:15 smithi107 ceph-mon[34380]: pgmap v399: 97 pgs: 97 active+clean; 2.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 48 MiB/s wr, 1.53k op/s 2024-09-21T06:14:15.579 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:15 smithi110 ceph-mon[34550]: pgmap v399: 97 pgs: 97 active+clean; 2.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 48 MiB/s wr, 1.53k op/s 2024-09-21T06:14:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 -- 172.21.15.31:0/1377876252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c1027f0 msgr2=0x7ff80c102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 --2- 172.21.15.31:0/1377876252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c1027f0 0x7ff80c102c00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff808009ca0 tx=0x7ff80802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 -- 172.21.15.31:0/1377876252 shutdown_connections 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 --2- 172.21.15.31:0/1377876252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108a70 0x7ff80c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 --2- 172.21.15.31:0/1377876252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c108120 0x7ff80c108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 --2- 172.21.15.31:0/1377876252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c1027f0 0x7ff80c102c00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 -- 172.21.15.31:0/1377876252 >> 172.21.15.31:0/1377876252 conn(0x7ff80c0fe210 msgr2=0x7ff80c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 -- 172.21.15.31:0/1377876252 shutdown_connections 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.577+0000 7ff81226a640 1 -- 172.21.15.31:0/1377876252 wait complete. 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 Processor -- start 2024-09-21T06:14:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 -- start start 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c1027f0 0x7ff80c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108120 0x7ff80c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c108a70 0x7ff80c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c113890 con 0x7ff80c108a70 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c113710 con 0x7ff80c108120 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff811268640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c1027f0 0x7ff80c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff81226a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c113a10 con 0x7ff80c1027f0 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff811268640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c1027f0 0x7ff80c19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47394/0 (socket says 172.21.15.31:47394) 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff811268640 1 -- 172.21.15.31:0/3777145182 learned_addr learned my addr 172.21.15.31:0/3777145182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff810a67640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108120 0x7ff80c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.578+0000 7ff811a69640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c108a70 0x7ff80c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff810a67640 1 -- 172.21.15.31:0/3777145182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c1027f0 msgr2=0x7ff80c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff810a67640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c1027f0 0x7ff80c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff810a67640 1 -- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c108a70 msgr2=0x7ff80c19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff810a67640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c108a70 0x7ff80c19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff810a67640 1 -- 172.21.15.31:0/3777145182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80c1a01c0 con 0x7ff80c108120 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff810a67640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108120 0x7ff80c19a830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff80c06a370 tx=0x7ff7fc00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff8027fc640 1 -- 172.21.15.31:0/3777145182 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc017070 con 0x7ff80c108120 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff8027fc640 1 -- 172.21.15.31:0/3777145182 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc0099a0 con 0x7ff80c108120 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80c1a04b0 con 0x7ff80c108120 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff8027fc640 1 -- 172.21.15.31:0/3777145182 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc005630 con 0x7ff80c108120 2024-09-21T06:14:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.579+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff80c1a6c40 con 0x7ff80c108120 2024-09-21T06:14:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.583+0000 7ff8027fc640 1 -- 172.21.15.31:0/3777145182 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff7fc0047a0 con 0x7ff80c108120 2024-09-21T06:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.585+0000 7ff8027fc640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7f80b6680 0x7ff7f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.585+0000 7ff8027fc640 1 -- 172.21.15.31:0/3777145182 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff7fc009b50 con 0x7ff80c108120 2024-09-21T06:14:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.586+0000 7ff811268640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7f80b6680 0x7ff7f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.586+0000 7ff81a43c740 1 Processor -- start 2024-09-21T06:14:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.586+0000 7ff81a43c740 1 -- start start 2024-09-21T06:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff811268640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7f80b6680 0x7ff7f80b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff808009a60 tx=0x7ff808005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c0e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff81a43c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b92c0586f0 con 0x55b92bd3e1b0 2024-09-21T06:14:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff81a43c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b92c047b20 con 0x55b92bd1d4d0 2024-09-21T06:14:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff81a43c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b92c04b930 con 0x55b92bd152f0 2024-09-21T06:14:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff810a67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff810a67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0e40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47406/0 (socket says 172.21.15.31:47406) 2024-09-21T06:14:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff810a67640 1 -- 172.21.15.31:0/1613627929 learned_addr learned my addr 172.21.15.31:0/1613627929 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff811268640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.587+0000 7ff811a69640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c0e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff810a67640 1 -- 172.21.15.31:0/1613627929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 msgr2=0x55b92c0e4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff810a67640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c0e4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff810a67640 1 -- 172.21.15.31:0/1613627929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 msgr2=0x55b92c0e3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff810a67640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff810a67640 1 -- 172.21.15.31:0/1613627929 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b92c00d530 con 0x55b92bd152f0 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff811a69640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c0e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff811268640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff810a67640 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0e40e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0e4a00 tx=0x7ff7fc0b4f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1413becce8586481 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff7f77fe640 1 -- 172.21.15.31:0/1613627929 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0a9950 con 0x55b92bd152f0 2024-09-21T06:14:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff7f77fe640 1 -- 172.21.15.31:0/1613627929 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc019ec0 con 0x55b92bd152f0 2024-09-21T06:14:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.588+0000 7ff7f77fe640 1 -- 172.21.15.31:0/1613627929 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0a9af0 con 0x55b92bd152f0 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.590+0000 7ff81a43c740 1 -- 172.21.15.31:0/1613627929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 msgr2=0x55b92c0e40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.590+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0e40e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0e4a00 tx=0x7ff7fc0b4f50 comp rx=0 tx=0).stop 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 -- 172.21.15.31:0/1613627929 shutdown_connections 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c0e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0e40e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1613627929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 -- 172.21.15.31:0/1613627929 >> 172.21.15.31:0/1613627929 conn(0x55b92bd32300 msgr2=0x55b92c0e1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 -- 172.21.15.31:0/1613627929 shutdown_connections 2024-09-21T06:14:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.591+0000 7ff81a43c740 1 -- 172.21.15.31:0/1613627929 wait complete. 2024-09-21T06:14:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.592+0000 7ff81a43c740 1 -- start start 2024-09-21T06:14:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.592+0000 7ff81a43c740 1 Processor -- start 2024-09-21T06:14:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.593+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd152f0 0x55b92c0ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0ff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff811a69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff811a69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37562/0 (socket says 172.21.15.31:37562) 2024-09-21T06:14:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff810a67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff811a69640 1 -- 172.21.15.31:0/943483809 learned_addr learned my addr 172.21.15.31:0/943483809 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.594+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b92bf7fab0 con 0x55b92bd3e1b0 2024-09-21T06:14:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b92bfed730 con 0x55b92bd152f0 2024-09-21T06:14:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b92c0f6640 con 0x55b92bd1d4d0 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff810a67640 1 -- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd152f0 msgr2=0x55b92c0ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff810a67640 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd152f0 0x55b92c0ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff810a67640 1 -- 172.21.15.31:0/943483809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 msgr2=0x55b92c0ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff810a67640 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff81a43c740 10 client.0 _ll_register_callbacks cb 0x55b92c0416b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff810a67640 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92bda9790 con 0x55b92bd1d4d0 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc019ec0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.595+0000 7ff811a69640 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff810a67640 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ef890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0aa000 tx=0x7ff7fc0e4f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3e2f2d3c21c087 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff7f4ff9640 10 client.25831 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/943483809 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0d0070 con 0x55b92bd1d4d0 2024-09-21T06:14:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/943483809 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff7fc0ac5d0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/943483809 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff7fc0a9cc0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff81a43c740 10 client.25831 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:15.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.596+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55b92be22620 con 0x55b92bd1d4d0 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/943483809 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff7fc0b1ce0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff81a43c740 10 client.25831 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff81a43c740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff81a43c740 20 client.25831 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff81a43c740 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcfcd80 0x55b92c105370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff81a43c740 4 client.25831 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.597+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b92c05a640 con 0x55b92bcfcd80 2024-09-21T06:14:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.598+0000 7ff811268640 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcfcd80 0x55b92c105370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.601+0000 7ff811268640 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcfcd80 0x55b92c105370 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afffa76cb295ac95 server_cookie=78ba3ac231e847ce in_seq=0 out_seq=0 2024-09-21T06:14:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.601+0000 7ff7f4ff9640 10 client.25831 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.602+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/943483809 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b92c05a640 con 0x55b92bcfcd80 2024-09-21T06:14:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.602+0000 7ff7f4ff9640 10 client.25831 handle_command_reply: tid=0 2024-09-21T06:14:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.602+0000 7ff81a43c740 1 client.25831 shutdown 2024-09-21T06:14:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.603+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd1d4d0 msgr2=0x55b92c0ef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.603+0000 7ff81a43c740 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ef890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0aa000 tx=0x7ff7fc0e4f10 comp rx=0 tx=0).stop 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 shutdown_connections 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcfcd80 0x55b92c105370 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd3e1b0 0x55b92c0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ef890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 --2- 172.21.15.31:0/943483809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd152f0 0x55b92c0ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 >> 172.21.15.31:0/943483809 conn(0x55b92bd32300 msgr2=0x55b92c0e4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff7f4ff9640 0 client.25831 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 shutdown_connections 2024-09-21T06:14:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 1 -- 172.21.15.31:0/943483809 wait complete. 2024-09-21T06:14:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.604+0000 7ff81a43c740 20 client.25831 trim_cache size 0 max 16384 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.620+0000 7ff81a43c740 1 Processor -- start 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 -- start start 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ab1c0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ada40 con 0x55b92bd3e1b0 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff81a43c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ab830 con 0x55b92bd152f0 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ff8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37564/0 (socket says 172.21.15.31:37564) 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 -- 172.21.15.31:0/324618700 learned_addr learned my addr 172.21.15.31:0/324618700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff811268640 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff811a69640 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 -- 172.21.15.31:0/324618700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 msgr2=0x55b92c0f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0f1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 -- 172.21.15.31:0/324618700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd3e1b0 msgr2=0x55b92c0e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff810a67640 1 -- 172.21.15.31:0/324618700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b92c0583c0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.621+0000 7ff811a69640 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.622+0000 7ff810a67640 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ff8a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e90 tx=0x7ff7fc095000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb336a32cc384d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.622+0000 7ff7f6ffd640 1 -- 172.21.15.31:0/324618700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc014020 con 0x55b92bd1d4d0 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.622+0000 7ff7f6ffd640 1 -- 172.21.15.31:0/324618700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc019ec0 con 0x55b92bd1d4d0 2024-09-21T06:14:15.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.622+0000 7ff7f6ffd640 1 -- 172.21.15.31:0/324618700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0a9e60 con 0x55b92bd1d4d0 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 -- 172.21.15.31:0/324618700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 msgr2=0x55b92c0ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ff8a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009e90 tx=0x7ff7fc095000 comp rx=0 tx=0).stop 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 -- 172.21.15.31:0/324618700 shutdown_connections 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd3e1b0 0x55b92c0e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd1d4d0 0x55b92c0ff8a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 --2- 172.21.15.31:0/324618700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd152f0 0x55b92c0f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 -- 172.21.15.31:0/324618700 >> 172.21.15.31:0/324618700 conn(0x55b92bd32300 msgr2=0x55b92c0e4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 -- 172.21.15.31:0/324618700 shutdown_connections 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 -- 172.21.15.31:0/324618700 wait complete. 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 -- start start 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.623+0000 7ff81a43c740 1 Processor -- start 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 0x55b92c107660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd3e1b0 0x55b92c16daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1abf00 con 0x55b92bd152f0 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ad370 con 0x55b92bd1d4d0 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff80c1ac5d0 con 0x55b92bd3e1b0 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff81a43c740 10 client.0 _ll_register_callbacks cb 0x7ff804005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff811268640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 0x55b92c107660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff811268640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 0x55b92c107660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37576/0 (socket says 172.21.15.31:37576) 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff811268640 1 -- 172.21.15.31:0/1821022362 learned_addr learned my addr 172.21.15.31:0/1821022362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff811a69640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd3e1b0 0x55b92c16daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.624+0000 7ff810a67640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff811268640 1 -- 172.21.15.31:0/1821022362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd3e1b0 msgr2=0x55b92c16daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff811268640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd3e1b0 0x55b92c16daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff811268640 1 -- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 msgr2=0x55b92c107ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff811268640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c107ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff811268640 1 -- 172.21.15.31:0/1821022362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b92c1083f0 con 0x55b92bd152f0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff811268640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 0x55b92c107660 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff808005e30 tx=0x7ff80802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fe1ebe7d3428ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff7f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/1821022362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808003e40 con 0x55b92bd152f0 2024-09-21T06:14:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/1821022362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff808004040 con 0x55b92bd152f0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc019ec0 con 0x55b92bd152f0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/1821022362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff808004240 con 0x55b92bd152f0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff81a43c740 10 client.16065 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b92c16f050 con 0x55b92bd152f0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.625+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/1821022362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc019ec0 con 0x55b92bd152f0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/1821022362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff808043040 con 0x55b92bd152f0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff81a43c740 10 client.16065 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff81a43c740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff81a43c740 20 client.16065 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcd21c0 0x55b92c171830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff81a43c740 4 client.16065 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b92c0560f0 con 0x55b92bcd21c0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff810a67640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcd21c0 0x55b92c171830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff7f4ff9640 10 client.16065 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.626+0000 7ff810a67640 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcd21c0 0x55b92c171830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd7015f51db04c9 server_cookie=73641655125db22d in_seq=0 out_seq=0 2024-09-21T06:14:15.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:15 smithi031 ceph-mon[31237]: pgmap v399: 97 pgs: 97 active+clean; 2.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 48 MiB/s wr, 1.53k op/s 2024-09-21T06:14:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff7f4ff9640 1 -- 172.21.15.31:0/1821022362 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b92c0560f0 con 0x55b92bcd21c0 2024-09-21T06:14:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff7f4ff9640 10 client.16065 handle_command_reply: tid=0 2024-09-21T06:14:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 client.16065 shutdown 2024-09-21T06:14:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 msgr2=0x55b92c107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 0x55b92c107660 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff808005e30 tx=0x7ff80802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 shutdown_connections 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b92bcd21c0 0x55b92c171830 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b92bd3e1b0 0x55b92c16daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b92bd1d4d0 0x55b92c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 --2- 172.21.15.31:0/1821022362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b92bd152f0 0x55b92c107660 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff7f4ff9640 0 client.16065 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 >> 172.21.15.31:0/1821022362 conn(0x55b92bd32300 msgr2=0x55b92c0e54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 shutdown_connections 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 1 -- 172.21.15.31:0/1821022362 wait complete. 2024-09-21T06:14:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81a43c740 20 client.16065 trim_cache size 0 max 16384 2024-09-21T06:14:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:14:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:14:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:14:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7f80b6680 msgr2=0x7ff7f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7f80b6680 0x7ff7f80b8b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff808009a60 tx=0x7ff808005d80 comp rx=0 tx=0).stop 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108120 msgr2=0x7ff80c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108120 0x7ff80c19a830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff80c06a370 tx=0x7ff7fc00dbb0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 shutdown_connections 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff7f80b6680 0x7ff7f80b8b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff80c108a70 0x7ff80c19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff80c108120 0x7ff80c19a830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 --2- 172.21.15.31:0/3777145182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff80c1027f0 0x7ff80c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 >> 172.21.15.31:0/3777145182 conn(0x7ff80c0fe210 msgr2=0x7ff80c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 shutdown_connections 2024-09-21T06:14:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:15.648+0000 7ff81226a640 1 -- 172.21.15.31:0/3777145182 wait complete. 2024-09-21T06:14:17.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:17 smithi031 ceph-mon[31237]: pgmap v400: 97 pgs: 97 active+clean; 2.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 56 MiB/s wr, 1.75k op/s 2024-09-21T06:14:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:17 smithi110 ceph-mon[34550]: pgmap v400: 97 pgs: 97 active+clean; 2.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 56 MiB/s wr, 1.75k op/s 2024-09-21T06:14:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:17 smithi107 ceph-mon[34380]: pgmap v400: 97 pgs: 97 active+clean; 2.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 56 MiB/s wr, 1.75k op/s 2024-09-21T06:14:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:19 smithi031 ceph-mon[31237]: pgmap v401: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 57 MiB/s wr, 1.59k op/s 2024-09-21T06:14:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:19 smithi110 ceph-mon[34550]: pgmap v401: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 57 MiB/s wr, 1.59k op/s 2024-09-21T06:14:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:19 smithi107 ceph-mon[34380]: pgmap v401: 97 pgs: 97 active+clean; 2.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 57 MiB/s wr, 1.59k op/s 2024-09-21T06:14:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 160.0 s, 3613.4 tps, lat 27.617 ms stddev 33.396 2024-09-21T06:14:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:21 smithi031 ceph-mon[31237]: pgmap v402: 97 pgs: 97 active+clean; 2.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 59 MiB/s wr, 1.57k op/s 2024-09-21T06:14:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:21 smithi110 ceph-mon[34550]: pgmap v402: 97 pgs: 97 active+clean; 2.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 59 MiB/s wr, 1.57k op/s 2024-09-21T06:14:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:21 smithi107 ceph-mon[34380]: pgmap v402: 97 pgs: 97 active+clean; 2.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 59 MiB/s wr, 1.57k op/s 2024-09-21T06:14:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:23 smithi031 ceph-mon[31237]: pgmap v403: 97 pgs: 97 active+clean; 2.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 59 MiB/s wr, 1.59k op/s 2024-09-21T06:14:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:23 smithi110 ceph-mon[34550]: pgmap v403: 97 pgs: 97 active+clean; 2.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 59 MiB/s wr, 1.59k op/s 2024-09-21T06:14:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:23 smithi107 ceph-mon[34380]: pgmap v403: 97 pgs: 97 active+clean; 2.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 59 MiB/s wr, 1.59k op/s 2024-09-21T06:14:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 165.0 s, 3157.4 tps, lat 31.684 ms stddev 59.706 2024-09-21T06:14:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:25 smithi031 ceph-mon[31237]: pgmap v404: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 31 MiB/s wr, 863 op/s 2024-09-21T06:14:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:25 smithi110 ceph-mon[34550]: pgmap v404: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 31 MiB/s wr, 863 op/s 2024-09-21T06:14:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:25 smithi107 ceph-mon[34380]: pgmap v404: 97 pgs: 97 active+clean; 2.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 31 MiB/s wr, 863 op/s 2024-09-21T06:14:27.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:27 smithi031 ceph-mon[31237]: pgmap v405: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 46 MiB/s wr, 1.82k op/s 2024-09-21T06:14:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:27 smithi110 ceph-mon[34550]: pgmap v405: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 46 MiB/s wr, 1.82k op/s 2024-09-21T06:14:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:27 smithi107 ceph-mon[34380]: pgmap v405: 97 pgs: 97 active+clean; 2.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 46 MiB/s wr, 1.82k op/s 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.798+0000 7ff9cc630640 1 -- 172.21.15.31:0/3741116039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 msgr2=0x7ff9c4108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.798+0000 7ff9cc630640 1 --2- 172.21.15.31:0/3741116039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c4108720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009900 tx=0x7ff9b402a410 comp rx=0 tx=0).stop 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 -- 172.21.15.31:0/3741116039 shutdown_connections 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 --2- 172.21.15.31:0/3741116039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9c4108c60 0x7ff9c410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 --2- 172.21.15.31:0/3741116039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c4108720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 --2- 172.21.15.31:0/3741116039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c4102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 -- 172.21.15.31:0/3741116039 >> 172.21.15.31:0/3741116039 conn(0x7ff9c407a6e0 msgr2=0x7ff9c407aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 -- 172.21.15.31:0/3741116039 shutdown_connections 2024-09-21T06:14:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.799+0000 7ff9cc630640 1 -- 172.21.15.31:0/3741116039 wait complete. 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.800+0000 7ff9cc630640 1 Processor -- start 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.800+0000 7ff9cc630640 1 -- start start 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.800+0000 7ff9cc630640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9cc630640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9cc630640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9c4108c60 0x7ff9c41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9cc630640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4112920 con 0x7ff9c41029e0 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9cc630640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41127a0 con 0x7ff9c4108c60 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9cc630640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4112aa0 con 0x7ff9c4108310 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9cabcd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9c4108c60 0x7ff9c41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9c9bcb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37986/0 (socket says 172.21.15.31:37986) 2024-09-21T06:14:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/1447939193 learned_addr learned my addr 172.21.15.31:0/1447939193 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9c9bcb640 1 -- 172.21.15.31:0/1447939193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9c4108c60 msgr2=0x7ff9c41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9c4108c60 0x7ff9c41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9c9bcb640 1 -- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 msgr2=0x7ff9c419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9c9bcb640 1 -- 172.21.15.31:0/1447939193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41a2600 con 0x7ff9c4108310 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.801+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.802+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c419cc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9b402a930 tx=0x7ff9b4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.802+0000 7ff9c37fe640 1 -- 172.21.15.31:0/1447939193 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b403f070 con 0x7ff9c4108310 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.802+0000 7ff9c37fe640 1 -- 172.21.15.31:0/1447939193 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b4004490 con 0x7ff9c4108310 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.802+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41a2890 con 0x7ff9c4108310 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.802+0000 7ff9c37fe640 1 -- 172.21.15.31:0/1447939193 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4035070 con 0x7ff9c4108310 2024-09-21T06:14:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.802+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a9080 con 0x7ff9c4108310 2024-09-21T06:14:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.805+0000 7ff9d3678740 1 Processor -- start 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 -- start start 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd74700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f9a54d0 0x56253fd74c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56253fce06f0 con 0x56253f99d2f0 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56253fccfb20 con 0x56253f9c61b0 2024-09-21T06:14:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.806+0000 7ff9d3678740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56253fcd3930 con 0x56253f9a54d0 2024-09-21T06:14:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd74700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd74700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38000/0 (socket says 172.21.15.31:38000) 2024-09-21T06:14:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/2539515524 learned_addr learned my addr 172.21.15.31:0/2539515524 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9cabcd640 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/2539515524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f9a54d0 msgr2=0x56253fd74c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:14:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f9a54d0 0x56253fd74c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/2539515524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 msgr2=0x56253fd741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.807+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/2539515524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fc95530 con 0x56253f99d2f0 2024-09-21T06:14:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.808+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd74700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80116a0 tx=0x7ff9b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598e4f37b5adf0a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.808+0000 7ff9c37fe640 1 -- 172.21.15.31:0/1447939193 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff9b40037f0 con 0x7ff9c4108310 2024-09-21T06:14:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.808+0000 7ff9c0bfe640 1 -- 172.21.15.31:0/2539515524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b801e070 con 0x56253f99d2f0 2024-09-21T06:14:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.808+0000 7ff9c0bfe640 1 -- 172.21.15.31:0/2539515524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b800fba0 con 0x56253f99d2f0 2024-09-21T06:14:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.808+0000 7ff9c0bfe640 1 -- 172.21.15.31:0/2539515524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8019c70 con 0x56253f99d2f0 2024-09-21T06:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.809+0000 7ff9c37fe640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff9ac0b6700 0x7ff9ac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.809+0000 7ff9c37fe640 1 -- 172.21.15.31:0/1447939193 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff9b4107af0 con 0x7ff9c4108310 2024-09-21T06:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 -- 172.21.15.31:0/2539515524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 msgr2=0x56253fd74700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd74700 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80116a0 tx=0x7ff9b8009f90 comp rx=0 tx=0).stop 2024-09-21T06:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9cabcd640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff9ac0b6700 0x7ff9ac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 -- 172.21.15.31:0/2539515524 shutdown_connections 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f9a54d0 0x56253fd74c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd74700 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 --2- 172.21.15.31:0/2539515524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.810+0000 7ff9d3678740 1 -- 172.21.15.31:0/2539515524 >> 172.21.15.31:0/2539515524 conn(0x56253f9ba300 msgr2=0x56253fd720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.811+0000 7ff9d3678740 1 -- 172.21.15.31:0/2539515524 shutdown_connections 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.811+0000 7ff9d3678740 1 -- 172.21.15.31:0/2539515524 wait complete. 2024-09-21T06:14:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.811+0000 7ff9cabcd640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff9ac0b6700 0x7ff9ac0b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005fd0 tx=0x7ff9bc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.812+0000 7ff9d3678740 1 -- start start 2024-09-21T06:14:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.812+0000 7ff9d3678740 1 Processor -- start 2024-09-21T06:14:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 0x56253fd7f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fd7ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56253fc07ab0 con 0x56253f9a54d0 2024-09-21T06:14:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56253fc75730 con 0x56253f9c61b0 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56253fd82940 con 0x56253f99d2f0 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9d3678740 10 client.0 _ll_register_callbacks cb 0x56253fcc96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9cabcd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 0x56253fd7f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9c9bcb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fd7ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9cabcd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd8bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55348/0 (socket says 172.21.15.31:55348) 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 0x56253fd7f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60768/0 (socket says 172.21.15.31:60768) 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9cabcd640 1 -- 172.21.15.31:0/949341068 learned_addr learned my addr 172.21.15.31:0/949341068 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 msgr2=0x56253fd8bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd8bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/949341068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 msgr2=0x56253fd7ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.813+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fd7ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.814+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/949341068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253fa31790 con 0x56253f99d2f0 2024-09-21T06:14:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.814+0000 7ff9cabcd640 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.814+0000 7ff9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 0x56253fd7f9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80197f0 tx=0x7ff9b8019520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a98257acb63fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/949341068 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8028060 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b800fba0 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/949341068 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff9b8016070 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/949341068 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff9b8028060 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9d3678740 10 client.25846 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.815+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56253faaa620 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/949341068 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b800fba0 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/949341068 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff9b8005160 con 0x56253f99d2f0 2024-09-21T06:14:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9d3678740 10 client.25846 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9d3678740 10 client.25846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9d3678740 20 client.25846 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9d3678740 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f984d80 0x56253fd91990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9d3678740 4 client.25846 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.816+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253fce2640 con 0x56253f984d80 2024-09-21T06:14:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.817+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f984d80 0x56253fd91990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.817+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f984d80 0x56253fd91990 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986ec1b4bd25c3a server_cookie=2e96dfd6ac282a97 in_seq=0 out_seq=0 2024-09-21T06:14:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.817+0000 7ff9a9ffb640 10 client.25846 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.819+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/949341068 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56253fce2640 con 0x56253f984d80 2024-09-21T06:14:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.819+0000 7ff9a9ffb640 10 client.25846 handle_command_reply: tid=0 2024-09-21T06:14:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.819+0000 7ff9d3678740 1 client.25846 shutdown 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.819+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 msgr2=0x56253fd7f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.819+0000 7ff9d3678740 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 0x56253fd7f9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80197f0 tx=0x7ff9b8019520 comp rx=0 tx=0).stop 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 shutdown_connections 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f984d80 0x56253fd91990 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9c61b0 0x56253fd8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9a9ffb640 0 client.25846 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fd7ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 --2- 172.21.15.31:0/949341068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f99d2f0 0x56253fd7f9c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 >> 172.21.15.31:0/949341068 conn(0x56253f9ba300 msgr2=0x56253fd67ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 shutdown_connections 2024-09-21T06:14:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 1 -- 172.21.15.31:0/949341068 wait complete. 2024-09-21T06:14:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.820+0000 7ff9d3678740 20 client.25846 trim_cache size 0 max 16384 2024-09-21T06:14:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.855+0000 7ff9d3678740 1 Processor -- start 2024-09-21T06:14:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.857+0000 7ff9d3678740 1 -- start start 2024-09-21T06:14:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.857+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd81ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.858+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd79a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.858+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9a54d0 0x56253fd8bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.858+0000 7ff9d3678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ab460 con 0x56253f99d2f0 2024-09-21T06:14:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.858+0000 7ff9d3678740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41adce0 con 0x56253f9a54d0 2024-09-21T06:14:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.858+0000 7ff9d3678740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41abad0 con 0x56253f984d80 2024-09-21T06:14:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.859+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd81ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.859+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd81ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60778/0 (socket says 172.21.15.31:60778) 2024-09-21T06:14:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.859+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/579562229 learned_addr learned my addr 172.21.15.31:0/579562229 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.860+0000 7ff9cabcd640 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9a54d0 0x56253fd8bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.860+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd79a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.861+0000 7ff9c9bcb640 1 -- 172.21.15.31:0/579562229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 msgr2=0x56253fd81ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.861+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd81ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.861+0000 7ff9c9bcb640 1 -- 172.21.15.31:0/579562229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9a54d0 msgr2=0x56253fd8bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.861+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9a54d0 0x56253fd8bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.861+0000 7ff9c9bcb640 1 -- 172.21.15.31:0/579562229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fd68f20 con 0x56253f99d2f0 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.861+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd79a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4105280 tx=0x7ff9b4003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6332e6bab8c9cf68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.862+0000 7ff9abfff640 1 -- 172.21.15.31:0/579562229 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40b6070 con 0x56253f99d2f0 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.862+0000 7ff9abfff640 1 -- 172.21.15.31:0/579562229 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56253fce2640 con 0x56253f99d2f0 2024-09-21T06:14:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.862+0000 7ff9abfff640 1 -- 172.21.15.31:0/579562229 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4048040 con 0x56253f99d2f0 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.863+0000 7ff9d3678740 1 -- 172.21.15.31:0/579562229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 msgr2=0x56253fd79a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.863+0000 7ff9d3678740 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd79a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4105280 tx=0x7ff9b4003f50 comp rx=0 tx=0).stop 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.863+0000 7ff9d3678740 1 -- 172.21.15.31:0/579562229 shutdown_connections 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.863+0000 7ff9d3678740 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f9a54d0 0x56253fd8bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.863+0000 7ff9d3678740 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f99d2f0 0x56253fd79a00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.863+0000 7ff9d3678740 1 --2- 172.21.15.31:0/579562229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd81ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.864+0000 7ff9d3678740 1 -- 172.21.15.31:0/579562229 >> 172.21.15.31:0/579562229 conn(0x56253f9ba300 msgr2=0x56253fd792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.865+0000 7ff9d3678740 1 -- 172.21.15.31:0/579562229 shutdown_connections 2024-09-21T06:14:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.870+0000 7ff9d3678740 1 -- 172.21.15.31:0/579562229 wait complete. 2024-09-21T06:14:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.870+0000 7ff9d3678740 1 -- start start 2024-09-21T06:14:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.870+0000 7ff9d3678740 1 Processor -- start 2024-09-21T06:14:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd87e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f99d2f0 0x56253fd88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fdf4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ac1a0 con 0x56253f9a54d0 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ad610 con 0x56253f99d2f0 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ac870 con 0x56253f984d80 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9d3678740 10 client.0 _ll_register_callbacks cb 0x7ff9b801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd87e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9ca3cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd87e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60792/0 (socket says 172.21.15.31:60792) 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/567807532 learned_addr learned my addr 172.21.15.31:0/567807532 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.871+0000 7ff9cabcd640 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fdf4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f99d2f0 msgr2=0x56253fd88400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f99d2f0 0x56253fd88400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/567807532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 msgr2=0x56253fdf4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fdf4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9ca3cc640 1 -- 172.21.15.31:0/567807532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253fd766f0 con 0x56253f984d80 2024-09-21T06:14:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9ca3cc640 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd87e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80116a0 tx=0x7ff9b800f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=707c86ba3cf1db64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/567807532 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b801e040 con 0x56253f984d80 2024-09-21T06:14:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/567807532 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff9b8004240 con 0x56253f984d80 2024-09-21T06:14:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.872+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56253fce2640 con 0x56253f984d80 2024-09-21T06:14:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.873+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/567807532 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff9b8005c50 con 0x56253f984d80 2024-09-21T06:14:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.874+0000 7ff9d3678740 10 client.25852 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.874+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56253fd857c0 con 0x56253f984d80 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/567807532 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56253fce2640 con 0x56253f984d80 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/567807532 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff9b8003a40 con 0x56253f984d80 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9d3678740 10 client.25852 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9d3678740 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9d3678740 20 client.25852 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9d3678740 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f9c61b0 0x56253fe0f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9d3678740 4 client.25852 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:14:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.875+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56253fcde0f0 con 0x56253f9c61b0 2024-09-21T06:14:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.876+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f9c61b0 0x56253fe0f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.876+0000 7ff9a9ffb640 10 client.25852 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:28.877+0000 7ff9c9bcb640 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f9c61b0 0x56253fe0f270 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53c17f683a5ce07 server_cookie=d2890a247bd1047f in_seq=0 out_seq=0 2024-09-21T06:14:29.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:29 smithi031 ceph-mon[31237]: pgmap v406: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 42 MiB/s wr, 1.89k op/s 2024-09-21T06:14:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:29 smithi110 ceph-mon[34550]: pgmap v406: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 42 MiB/s wr, 1.89k op/s 2024-09-21T06:14:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:29 smithi107 ceph-mon[34380]: pgmap v406: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 42 MiB/s wr, 1.89k op/s 2024-09-21T06:14:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 170.0 s, 4264.8 tps, lat 23.411 ms stddev 25.539 2024-09-21T06:14:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:31 smithi031 ceph-mon[31237]: pgmap v407: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 40 MiB/s wr, 1.87k op/s 2024-09-21T06:14:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:31 smithi110 ceph-mon[34550]: pgmap v407: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 40 MiB/s wr, 1.87k op/s 2024-09-21T06:14:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:31 smithi107 ceph-mon[34380]: pgmap v407: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 40 MiB/s wr, 1.87k op/s 2024-09-21T06:14:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:33 smithi031 ceph-mon[31237]: pgmap v408: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 40 MiB/s wr, 2.06k op/s 2024-09-21T06:14:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:33 smithi110 ceph-mon[34550]: pgmap v408: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 40 MiB/s wr, 2.06k op/s 2024-09-21T06:14:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:33 smithi107 ceph-mon[34380]: pgmap v408: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 40 MiB/s wr, 2.06k op/s 2024-09-21T06:14:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 175.0 s, 4059.6 tps, lat 24.686 ms stddev 27.213 2024-09-21T06:14:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:35 smithi031 ceph-mon[31237]: pgmap v409: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 32 MiB/s wr, 1.87k op/s 2024-09-21T06:14:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:35 smithi110 ceph-mon[34550]: pgmap v409: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 32 MiB/s wr, 1.87k op/s 2024-09-21T06:14:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:35 smithi107 ceph-mon[34380]: pgmap v409: 97 pgs: 97 active+clean; 2.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 32 MiB/s wr, 1.87k op/s 2024-09-21T06:14:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:37 smithi031 ceph-mon[31237]: pgmap v410: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2024-09-21T06:14:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:37 smithi110 ceph-mon[34550]: pgmap v410: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2024-09-21T06:14:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:37 smithi107 ceph-mon[34380]: pgmap v410: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 32 MiB/s wr, 1.92k op/s 2024-09-21T06:14:39.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:39 smithi031 ceph-mon[31237]: pgmap v411: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 992 op/s 2024-09-21T06:14:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:39 smithi110 ceph-mon[34550]: pgmap v411: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 992 op/s 2024-09-21T06:14:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:39 smithi107 ceph-mon[34380]: pgmap v411: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 992 op/s 2024-09-21T06:14:40.207 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 180.0 s, 1357.6 tps, lat 27.089 ms stddev 31.549 2024-09-21T06:14:41.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:41 smithi031 ceph-mon[31237]: pgmap v412: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 720 op/s 2024-09-21T06:14:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:41 smithi110 ceph-mon[34550]: pgmap v412: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 720 op/s 2024-09-21T06:14:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:41 smithi107 ceph-mon[34380]: pgmap v412: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 11 MiB/s wr, 720 op/s 2024-09-21T06:14:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.099+0000 7ff9a9ffb640 1 -- 172.21.15.31:0/567807532 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56253fcde0f0 con 0x56253f9c61b0 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.099+0000 7ff9a9ffb640 10 client.25852 handle_command_reply: tid=0 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.099+0000 7ff9d3678740 1 client.25852 shutdown 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.099+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 msgr2=0x56253fd87e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.099+0000 7ff9d3678740 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd87e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80116a0 tx=0x7ff9b800f4f0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 shutdown_connections 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56253f9c61b0 0x56253fe0f270 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56253f9a54d0 0x56253fdf4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56253f99d2f0 0x56253fd88400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 --2- 172.21.15.31:0/567807532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56253f984d80 0x56253fd87e40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 >> 172.21.15.31:0/567807532 conn(0x56253f9ba300 msgr2=0x56253fd7ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9a9ffb640 0 client.25852 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 shutdown_connections 2024-09-21T06:14:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 1 -- 172.21.15.31:0/567807532 wait complete. 2024-09-21T06:14:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.100+0000 7ff9d3678740 20 client.25852 trim_cache size 0 max 16384 2024-09-21T06:14:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:14:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:14:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:14:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:14:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff9ac0b6700 msgr2=0x7ff9ac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff9ac0b6700 0x7ff9ac0b8c10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005fd0 tx=0x7ff9bc009500 comp rx=0 tx=0).stop 2024-09-21T06:14:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 msgr2=0x7ff9c419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c419cc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9b402a930 tx=0x7ff9b4031a20 comp rx=0 tx=0).stop 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 shutdown_connections 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9c4108c60 0x7ff9c41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9c4108310 0x7ff9c419cc70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff9ac0b6700 0x7ff9ac0b8c10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 --2- 172.21.15.31:0/1447939193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9c41029e0 0x7ff9c419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.101+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 >> 172.21.15.31:0/1447939193 conn(0x7ff9c407a6e0 msgr2=0x7ff9c41044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.102+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 shutdown_connections 2024-09-21T06:14:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:42.102+0000 7ff9cc630640 1 -- 172.21.15.31:0/1447939193 wait complete. 2024-09-21T06:14:42.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:14:42.862 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 -- 172.21.15.31:0/285889566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec112dc0 msgr2=0x7fbdec1151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 --2- 172.21.15.31:0/285889566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec112dc0 0x7fbdec1151e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009900 tx=0x7fbddc02a400 comp rx=0 tx=0).stop 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 -- 172.21.15.31:0/285889566 shutdown_connections 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 --2- 172.21.15.31:0/285889566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdec115720 0x7fbdec117b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 --2- 172.21.15.31:0/285889566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec112dc0 0x7fbdec1151e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 --2- 172.21.15.31:0/285889566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec10bff0 0x7fbdec10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.468+0000 7fbdf108b640 1 -- 172.21.15.31:0/285889566 >> 172.21.15.31:0/285889566 conn(0x7fbdec0712f0 msgr2=0x7fbdec073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.469+0000 7fbdf108b640 1 -- 172.21.15.31:0/285889566 shutdown_connections 2024-09-21T06:14:43.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.469+0000 7fbdf108b640 1 -- 172.21.15.31:0/285889566 wait complete. 2024-09-21T06:14:43.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.470+0000 7fbdf108b640 1 Processor -- start 2024-09-21T06:14:43.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.471+0000 7fbdf108b640 1 -- start start 2024-09-21T06:14:43.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.471+0000 7fbdf108b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec10bff0 0x7fbdec1a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdf108b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec112dc0 0x7fbdec1a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdf108b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdec115720 0x7fbdec1a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdead9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec10bff0 0x7fbdec1a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdf108b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec11b180 con 0x7fbdec115720 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdf108b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec11b000 con 0x7fbdec112dc0 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdf108b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec11b300 con 0x7fbdec10bff0 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdec115720 0x7fbdec1a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdead9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec10bff0 0x7fbdec1a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51736/0 (socket says 172.21.15.31:51736) 2024-09-21T06:14:43.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdea59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec112dc0 0x7fbdec1a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.472+0000 7fbdead9d640 1 -- 172.21.15.31:0/2051086655 learned_addr learned my addr 172.21.15.31:0/2051086655 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdea59c640 1 -- 172.21.15.31:0/2051086655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec10bff0 msgr2=0x7fbdec1a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdea59c640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec10bff0 0x7fbdec1a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdea59c640 1 -- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdec115720 msgr2=0x7fbdec1a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdea59c640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdec115720 0x7fbdec1a73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdea59c640 1 -- 172.21.15.31:0/2051086655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1a7b00 con 0x7fbdec112dc0 2024-09-21T06:14:43.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdea59c640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec112dc0 0x7fbdec1a2130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbddc02a920 tx=0x7fbddc0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdd3fff640 1 -- 172.21.15.31:0/2051086655 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc040070 con 0x7fbdec112dc0 2024-09-21T06:14:43.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec1a7d90 con 0x7fbdec112dc0 2024-09-21T06:14:43.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.474+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec1ae550 con 0x7fbdec112dc0 2024-09-21T06:14:43.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.473+0000 7fbdd3fff640 1 -- 172.21.15.31:0/2051086655 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbddc031480 con 0x7fbdec112dc0 2024-09-21T06:14:43.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.474+0000 7fbdd3fff640 1 -- 172.21.15.31:0/2051086655 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc035020 con 0x7fbdec112dc0 2024-09-21T06:14:43.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.477+0000 7fbdf75ff740 1 Processor -- start 2024-09-21T06:14:43.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.477+0000 7fbdf75ff740 1 -- start start 2024-09-21T06:14:43.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb40070 0x55dcbeee7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 0x55dcbeee77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dcbee5de50 con 0x55dcbeb1f390 2024-09-21T06:14:43.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dcbee4d280 con 0x55dcbeb171b0 2024-09-21T06:14:43.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dcbee51090 con 0x55dcbeb40070 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 0x55dcbeee77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 0x55dcbeee77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47634/0 (socket says 172.21.15.31:47634) 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 -- 172.21.15.31:0/2568639222 learned_addr learned my addr 172.21.15.31:0/2568639222 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb40070 0x55dcbeee7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 -- 172.21.15.31:0/2568639222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb40070 msgr2=0x55dcbeee7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb40070 0x55dcbeee7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 -- 172.21.15.31:0/2568639222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 msgr2=0x55dcbeee7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee7cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.478+0000 7fbdead9d640 1 -- 172.21.15.31:0/2568639222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbebab650 con 0x55dcbeb171b0 2024-09-21T06:14:43.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.479+0000 7fbdead9d640 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 0x55dcbeee77b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d30 tx=0x7fbde00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b5fc7f9ed07272f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.479+0000 7fbdd3fff640 1 -- 172.21.15.31:0/2051086655 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbddc043070 con 0x7fbdec112dc0 2024-09-21T06:14:43.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.479+0000 7fbdd17fa640 1 -- 172.21.15.31:0/2568639222 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0012650 con 0x55dcbeb171b0 2024-09-21T06:14:43.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.479+0000 7fbdd17fa640 1 -- 172.21.15.31:0/2568639222 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0009e20 con 0x55dcbeb171b0 2024-09-21T06:14:43.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.480+0000 7fbdf75ff740 1 -- 172.21.15.31:0/2568639222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 msgr2=0x55dcbeee77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.481+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 0x55dcbeee77b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d30 tx=0x7fbde00117b0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.479+0000 7fbdd17fa640 1 -- 172.21.15.31:0/2568639222 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000fc70 con 0x55dcbeb171b0 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.481+0000 7fbdf75ff740 1 -- 172.21.15.31:0/2568639222 shutdown_connections 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.481+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.481+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb171b0 0x55dcbeee77b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.481+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/2568639222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb40070 0x55dcbeee7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.481+0000 7fbdf75ff740 1 -- 172.21.15.31:0/2568639222 >> 172.21.15.31:0/2568639222 conn(0x55dcbeb34460 msgr2=0x55dcbeee5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.482+0000 7fbdd3fff640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdd40b6660 0x7fbdd40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.482+0000 7fbdd3fff640 1 -- 172.21.15.31:0/2051086655 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbddc1081c0 con 0x7fbdec112dc0 2024-09-21T06:14:43.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.484+0000 7fbdf75ff740 1 -- 172.21.15.31:0/2568639222 shutdown_connections 2024-09-21T06:14:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.486+0000 7fbdead9d640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdd40b6660 0x7fbdd40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.486+0000 7fbdf75ff740 1 -- 172.21.15.31:0/2568639222 wait complete. 2024-09-21T06:14:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.486+0000 7fbdf75ff740 1 -- start start 2024-09-21T06:14:43.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.487+0000 7fbdf75ff740 1 Processor -- start 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.487+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbeef2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 0x55dcbeef3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dcbecae700 con 0x55dcbeb40070 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dcbede8110 con 0x55dcbeb1f390 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dcbeef5aa0 con 0x55dcbeb171b0 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdf75ff740 10 client.0 _ll_register_callbacks cb 0x55dcbee01990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 0x55dcbeef3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 0x55dcbeef3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33496/0 (socket says 172.21.15.31:33496) 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.488+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3549516090 learned_addr learned my addr 172.21.15.31:0/3549516090 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.489+0000 7fbdea59c640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbeef2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.489+0000 7fbdead9d640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.489+0000 7fbdead9d640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdd40b6660 0x7fbdd40b8b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009ac0 tx=0x7fbde000f810 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3549516090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 msgr2=0x55dcbeef28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 msgr2=0x55dcbeef2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbeef2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3549516090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbee15c40 con 0x55dcbeb40070 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdea59c640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbeef2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 0x55dcbeef3350 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbde4012cf0 tx=0x7fbde400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21141152d4aade93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.490+0000 7fbdceff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:43.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.491+0000 7fbdceff5640 1 -- 172.21.15.31:0/3549516090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4012110 con 0x55dcbeb40070 2024-09-21T06:14:43.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.492+0000 7fbdceff5640 1 -- 172.21.15.31:0/3549516090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbde4003970 con 0x55dcbeb40070 2024-09-21T06:14:43.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.493+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde0009e20 con 0x55dcbeb40070 2024-09-21T06:14:43.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.495+0000 7fbdceff5640 1 -- 172.21.15.31:0/3549516090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbde40122b0 con 0x55dcbeb40070 2024-09-21T06:14:43.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.495+0000 7fbdf75ff740 10 client.16092 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:43.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.495+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dcbeef8700 con 0x55dcbeb40070 2024-09-21T06:14:43.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.496+0000 7fbdceff5640 1 -- 172.21.15.31:0/3549516090 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0009e20 con 0x55dcbeb40070 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdceff5640 1 -- 172.21.15.31:0/3549516090 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fbde4010430 con 0x55dcbeb40070 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdf75ff740 10 client.16092 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdf75ff740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdf75ff740 20 client.16092 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeafe9f0 0x55dcbeefc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdf75ff740 4 client.16092 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.497+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcbee5fda0 con 0x55dcbeafe9f0 2024-09-21T06:14:43.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.499+0000 7fbdead9d640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeafe9f0 0x55dcbeefc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.500+0000 7fbdead9d640 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeafe9f0 0x55dcbeefc050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2c229e067a2e5a server_cookie=fbbce2ffecc6725b in_seq=0 out_seq=0 2024-09-21T06:14:43.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.500+0000 7fbdceff5640 10 client.16092 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:43.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdceff5640 1 -- 172.21.15.31:0/3549516090 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcbee5fda0 con 0x55dcbeafe9f0 2024-09-21T06:14:43.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdceff5640 10 client.16092 handle_command_reply: tid=0 2024-09-21T06:14:43.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 client.16092 shutdown 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 msgr2=0x55dcbeef3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 0x55dcbeef3350 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbde4012cf0 tx=0x7fbde400f790 comp rx=0 tx=0).stop 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 shutdown_connections 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeafe9f0 0x55dcbeefc050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb40070 0x55dcbeef3350 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbeef2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3549516090 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.502+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 >> 172.21.15.31:0/3549516090 conn(0x55dcbeb34460 msgr2=0x55dcbeeea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.503+0000 7fbdceff5640 0 client.16092 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.503+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 shutdown_connections 2024-09-21T06:14:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.503+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3549516090 wait complete. 2024-09-21T06:14:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.503+0000 7fbdf75ff740 20 client.16092 trim_cache size 0 max 16384 2024-09-21T06:14:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 Processor -- start 2024-09-21T06:14:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 -- start start 2024-09-21T06:14:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb34460 0x55dcbef07210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800c740 con 0x55dcbeb1f390 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800ef10 con 0x55dcbeb34460 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.537+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800cd00 con 0x55dcbeb171b0 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33504/0 (socket says 172.21.15.31:33504) 2024-09-21T06:14:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3192931791 learned_addr learned my addr 172.21.15.31:0/3192931791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdea59c640 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3192931791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 msgr2=0x55dcbeef44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3192931791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb34460 msgr2=0x55dcbef07210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb34460 0x55dcbef07210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.538+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3192931791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbeef5460 con 0x55dcbeb1f390 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.539+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee4710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006010 tx=0x7fbde4012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ab4c7633a630be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.539+0000 7fbdd0ff9640 1 -- 172.21.15.31:0/3192931791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4010430 con 0x55dcbeb1f390 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.539+0000 7fbdd0ff9640 1 -- 172.21.15.31:0/3192931791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde40105d0 con 0x55dcbeb1f390 2024-09-21T06:14:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.539+0000 7fbdd0ff9640 1 -- 172.21.15.31:0/3192931791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4005270 con 0x55dcbeb1f390 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.540+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3192931791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 msgr2=0x55dcbeee4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.540+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee4710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbde4006010 tx=0x7fbde4012910 comp rx=0 tx=0).stop 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.541+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3192931791 shutdown_connections 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.541+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb1f390 0x55dcbeee4710 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.541+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbeef44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.541+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3192931791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb34460 0x55dcbef07210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.541+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3192931791 >> 172.21.15.31:0/3192931791 conn(0x55dcbeefc390 msgr2=0x55dcbeef3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.542+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3192931791 shutdown_connections 2024-09-21T06:14:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.542+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3192931791 wait complete. 2024-09-21T06:14:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 -- start start 2024-09-21T06:14:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 Processor -- start 2024-09-21T06:14:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbef0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbef0a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 0x55dcbef0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800d3d0 con 0x55dcbeb34460 2024-09-21T06:14:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800e840 con 0x55dcbeb1f390 2024-09-21T06:14:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd800daa0 con 0x55dcbeb171b0 2024-09-21T06:14:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.543+0000 7fbdf75ff740 10 client.0 _ll_register_callbacks cb 0x7fbde4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 0x55dcbef0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 0x55dcbef0a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33508/0 (socket says 172.21.15.31:33508) 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdea59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbef0a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3522897366 learned_addr learned my addr 172.21.15.31:0/3522897366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3522897366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 msgr2=0x55dcbef0cd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbef0cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 msgr2=0x55dcbef0a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbef0a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.544+0000 7fbdeb59e640 1 -- 172.21.15.31:0/3522897366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbef0ac90 con 0x55dcbeb34460 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.545+0000 7fbdeb59e640 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 0x55dcbef0a750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbde4005f40 tx=0x7fbde4012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cc826c75dd8aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.545+0000 7fbdceff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.545+0000 7fbdceff5640 1 -- 172.21.15.31:0/3522897366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4034070 con 0x55dcbeb34460 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.545+0000 7fbdceff5640 1 -- 172.21.15.31:0/3522897366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbde4005250 con 0x55dcbeb34460 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.545+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde40105d0 con 0x55dcbeb34460 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdceff5640 1 -- 172.21.15.31:0/3522897366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbde403f050 con 0x55dcbeb34460 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 10 client.16098 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dcbef6f6c0 con 0x55dcbeb34460 2024-09-21T06:14:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdceff5640 1 -- 172.21.15.31:0/3522897366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde4005900 con 0x55dcbeb34460 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdceff5640 1 -- 172.21.15.31:0/3522897366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fbde401ca80 con 0x55dcbeb34460 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 10 client.16098 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 20 client.16098 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeb40070 0x55dcbef71d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 4 client.16098 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:14:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.546+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbde40105d0 con 0x55dcbeb40070 2024-09-21T06:14:43.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.548+0000 7fbdead9d640 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeb40070 0x55dcbef71d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.549+0000 7fbdceff5640 10 client.16098 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.549+0000 7fbdead9d640 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeb40070 0x55dcbef71d30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1862d737a2a368 server_cookie=22c987eb2ecf31e9 in_seq=0 out_seq=0 2024-09-21T06:14:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.549+0000 7fbdceff5640 1 -- 172.21.15.31:0/3522897366 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x7fbde40105d0 con 0x55dcbeb40070 2024-09-21T06:14:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.549+0000 7fbdceff5640 10 client.16098 handle_command_reply: tid=0 2024-09-21T06:14:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.549+0000 7fbdf75ff740 1 client.16098 shutdown 2024-09-21T06:14:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.550+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 msgr2=0x55dcbef0a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.550+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 0x55dcbef0a750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbde4005f40 tx=0x7fbde4012a00 comp rx=0 tx=0).stop 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 shutdown_connections 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dcbeb40070 0x55dcbef71d30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dcbeb34460 0x55dcbef0a750 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdceff5640 0 client.16098 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dcbeb1f390 0x55dcbef0a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 --2- 172.21.15.31:0/3522897366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dcbeb171b0 0x55dcbef0cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 >> 172.21.15.31:0/3522897366 conn(0x55dcbeefc390 msgr2=0x55dcbef06fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 shutdown_connections 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 1 -- 172.21.15.31:0/3522897366 wait complete. 2024-09-21T06:14:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.551+0000 7fbdf75ff740 20 client.16098 trim_cache size 0 max 16384 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.554 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:14:43.555 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.556 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:14:43.557 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:43.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:43.560 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:43.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:43.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:43.560 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdd40b6660 msgr2=0x7fbdd40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdd40b6660 0x7fbdd40b8b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009ac0 tx=0x7fbde000f810 comp rx=0 tx=0).stop 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec112dc0 msgr2=0x7fbdec1a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec112dc0 0x7fbdec1a2130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbddc02a920 tx=0x7fbddc0045f0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 shutdown_connections 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdd40b6660 0x7fbdd40b8b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdec115720 0x7fbdec1a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdec112dc0 0x7fbdec1a2130 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 --2- 172.21.15.31:0/2051086655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdec10bff0 0x7fbdec1a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 >> 172.21.15.31:0/2051086655 conn(0x7fbdec0712f0 msgr2=0x7fbdec115b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 shutdown_connections 2024-09-21T06:14:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:43.552+0000 7fbdf108b640 1 -- 172.21.15.31:0/2051086655 wait complete. 2024-09-21T06:14:43.572 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:43 smithi031 ceph-mon[31237]: pgmap v413: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 857 op/s 2024-09-21T06:14:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:43 smithi110 ceph-mon[34550]: pgmap v413: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 857 op/s 2024-09-21T06:14:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:43 smithi107 ceph-mon[34380]: pgmap v413: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 13 MiB/s wr, 857 op/s 2024-09-21T06:14:44.400 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:14:44.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:44.824 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 185.0 s, 1829.0 tps, lat 88.527 ms stddev 531.800 2024-09-21T06:14:45.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.415+0000 7f4042dcc640 1 -- 172.21.15.31:0/3909084502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c1031c0 msgr2=0x7f403c1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.415+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3909084502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c1031c0 0x7f403c1035d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4030009900 tx=0x7f403002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 -- 172.21.15.31:0/3909084502 shutdown_connections 2024-09-21T06:14:45.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3909084502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105ae0 0x7f403c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3909084502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c105190 0x7f403c1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3909084502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c1031c0 0x7f403c1035d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 -- 172.21.15.31:0/3909084502 >> 172.21.15.31:0/3909084502 conn(0x7f403c077c60 msgr2=0x7f403c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:45.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 -- 172.21.15.31:0/3909084502 shutdown_connections 2024-09-21T06:14:45.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.416+0000 7f4042dcc640 1 -- 172.21.15.31:0/3909084502 wait complete. 2024-09-21T06:14:45.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.417+0000 7f4042dcc640 1 Processor -- start 2024-09-21T06:14:45.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.418+0000 7f4042dcc640 1 -- start start 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4042dcc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 0x7f403c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4042dcc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105190 0x7f403c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4042dcc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c105ae0 0x7f403c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4042dcc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1129d0 con 0x7f403c105ae0 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4042dcc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f403c112850 con 0x7f403c105190 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4042dcc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f403c112b50 con 0x7f403c1031c0 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 0x7f403c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:45.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 0x7f403c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51788/0 (socket says 172.21.15.31:51788) 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 -- 172.21.15.31:0/3195371920 learned_addr learned my addr 172.21.15.31:0/3195371920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f403bfff640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105190 0x7f403c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4041369640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c105ae0 0x7f403c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 -- 172.21.15.31:0/3195371920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105190 msgr2=0x7f403c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105190 0x7f403c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 -- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c105ae0 msgr2=0x7f403c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c105ae0 0x7f403c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.419+0000 7f4040b68640 1 -- 172.21.15.31:0/3195371920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c1a25d0 con 0x7f403c1031c0 2024-09-21T06:14:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.420+0000 7f4040b68640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 0x7f403c19c660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4030005ab0 tx=0x7f4030003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.420+0000 7f403bfff640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105190 0x7f403c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.420+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403003f070 con 0x7f403c1031c0 2024-09-21T06:14:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.420+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4030031d60 con 0x7f403c1031c0 2024-09-21T06:14:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.421+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c1a2860 con 0x7f403c1031c0 2024-09-21T06:14:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.421+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a9020 con 0x7f403c1031c0 2024-09-21T06:14:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.422+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030035030 con 0x7f403c1031c0 2024-09-21T06:14:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.422+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c1035d0 con 0x7f403c1031c0 2024-09-21T06:14:45.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.425+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4030042070 con 0x7f403c1031c0 2024-09-21T06:14:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.426+0000 7f4039ffb640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f402c0b6560 0x7f402c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.426+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4030108480 con 0x7f403c1031c0 2024-09-21T06:14:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.426+0000 7f403bfff640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f402c0b6560 0x7f402c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:45.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.427+0000 7f403bfff640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f402c0b6560 0x7f402c0b8a70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f403c1067e0 tx=0x7f4028005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.428+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40300d2df0 con 0x7f403c1031c0 2024-09-21T06:14:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:45 smithi031 ceph-mon[31237]: pgmap v414: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-09-21T06:14:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:45 smithi110 ceph-mon[34550]: pgmap v414: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-09-21T06:14:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:45 smithi107 ceph-mon[34380]: pgmap v414: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-09-21T06:14:45.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.826+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f403c06ac10 con 0x7f403c1031c0 2024-09-21T06:14:45.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.827+0000 7f4039ffb640 1 -- 172.21.15.31:0/3195371920 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f40300d7d30 con 0x7f403c1031c0 2024-09-21T06:14:45.829 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:14:45.829 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:14:45.830 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:14:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f402c0b6560 msgr2=0x7f402c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f402c0b6560 0x7f402c0b8a70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f403c1067e0 tx=0x7f4028005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 msgr2=0x7f403c19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 0x7f403c19c660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4030005ab0 tx=0x7f4030003c10 comp rx=0 tx=0).stop 2024-09-21T06:14:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 shutdown_connections 2024-09-21T06:14:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f402c0b6560 0x7f402c0b8a70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f403c105ae0 0x7f403c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f403c105190 0x7f403c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 --2- 172.21.15.31:0/3195371920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f403c1031c0 0x7f403c19c660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.831+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 >> 172.21.15.31:0/3195371920 conn(0x7f403c077c60 msgr2=0x7f403c10e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.832+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 shutdown_connections 2024-09-21T06:14:45.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:45.832+0000 7f4042dcc640 1 -- 172.21.15.31:0/3195371920 wait complete. 2024-09-21T06:14:46.590 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:46 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3195371920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:46.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:14:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:46 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3195371920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:46 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3195371920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:47.135 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:47.611 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:47 smithi031 ceph-mon[31237]: pgmap v415: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.1 MiB/s wr, 662 op/s 2024-09-21T06:14:47.611 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:14:47.611 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:14:47.611 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:14:47.611 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.752+0000 7f9313fff640 1 -- 172.21.15.31:0/3992094184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93140307c0 con 0x7f931c110250 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 -- 172.21.15.31:0/3992094184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 msgr2=0x7f931c116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 --2- 172.21.15.31:0/3992094184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 0x7f931c116a80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9314009ca0 tx=0x7f931402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 -- 172.21.15.31:0/3992094184 shutdown_connections 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 --2- 172.21.15.31:0/3992094184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 0x7f931c116a80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 --2- 172.21.15.31:0/3992094184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c10f900 0x7f931c10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 --2- 172.21.15.31:0/3992094184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c109fc0 0x7f931c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 -- 172.21.15.31:0/3992094184 >> 172.21.15.31:0/3992094184 conn(0x7f931c0795c0 msgr2=0x7f931c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:47.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 -- 172.21.15.31:0/3992094184 shutdown_connections 2024-09-21T06:14:47.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.753+0000 7f9322067640 1 -- 172.21.15.31:0/3992094184 wait complete. 2024-09-21T06:14:47.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.754+0000 7f9322067640 1 Processor -- start 2024-09-21T06:14:47.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.754+0000 7f9322067640 1 -- start start 2024-09-21T06:14:47.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9322067640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 0x7f931c0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9322067640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c10f900 0x7f931c074f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9322067640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 0x7f931c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9322067640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f931c06d2b0 con 0x7f931c110250 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9322067640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f931c06d130 con 0x7f931c10f900 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9322067640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f931c06d430 con 0x7f931c109fc0 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 0x7f931c0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 0x7f931c0749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51816/0 (socket says 172.21.15.31:51816) 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 -- 172.21.15.31:0/191975779 learned_addr learned my addr 172.21.15.31:0/191975779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9320864640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c10f900 0x7f931c074f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321866640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 0x7f931c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 -- 172.21.15.31:0/191975779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c10f900 msgr2=0x7f931c074f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c10f900 0x7f931c074f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 -- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 msgr2=0x7f931c06ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 0x7f931c06ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.755+0000 7f9321065640 1 -- 172.21.15.31:0/191975779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c06f580 con 0x7f931c109fc0 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f9320864640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c10f900 0x7f931c074f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f9321065640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 0x7f931c0749d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f931800ea60 tx=0x7f931800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f93127fc640 1 -- 172.21.15.31:0/191975779 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931800cde0 con 0x7f931c109fc0 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c06f870 con 0x7f931c109fc0 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c1b59a0 con 0x7f931c109fc0 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f93127fc640 1 -- 172.21.15.31:0/191975779 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93180035f0 con 0x7f931c109fc0 2024-09-21T06:14:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.756+0000 7f93127fc640 1 -- 172.21.15.31:0/191975779 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318010690 con 0x7f931c109fc0 2024-09-21T06:14:47.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.760+0000 7f93127fc640 1 -- 172.21.15.31:0/191975779 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9318005020 con 0x7f931c109fc0 2024-09-21T06:14:47.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.760+0000 7f932983d740 1 Processor -- start 2024-09-21T06:14:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 -- start start 2024-09-21T06:14:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae72860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaa21b0 0x55885ae72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 0x55885ae732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55885ade8ea0 con 0x55885aacb070 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55885add82d0 con 0x55885aaaa390 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f932983d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55885addc0e0 con 0x55885aaa21b0 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f9320864640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 0x55885ae732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f9320864640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 0x55885ae732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47704/0 (socket says 172.21.15.31:47704) 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f9320864640 1 -- 172.21.15.31:0/3614332471 learned_addr learned my addr 172.21.15.31:0/3614332471 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.761+0000 7f9321866640 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae72860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.762+0000 7f9321065640 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaa21b0 0x55885ae72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.762+0000 7f9320864640 1 -- 172.21.15.31:0/3614332471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaa21b0 msgr2=0x55885ae72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.762+0000 7f9320864640 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaa21b0 0x55885ae72da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.762+0000 7f9320864640 1 -- 172.21.15.31:0/3614332471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 msgr2=0x55885ae72860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.762+0000 7f9320864640 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae72860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.762+0000 7f9320864640 1 -- 172.21.15.31:0/3614332471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55885ab36650 con 0x55885aaaa390 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.763+0000 7f9320864640 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 0x55885ae732e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f930c00bd30 tx=0x7f930c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf68001f8c2c762 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.763+0000 7f93127fc640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93080b6700 0x7f93080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.763+0000 7f93077fe640 1 -- 172.21.15.31:0/3614332471 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c004230 con 0x55885aaaa390 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.763+0000 7f93077fe640 1 -- 172.21.15.31:0/3614332471 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c00be20 con 0x55885aaaa390 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.764+0000 7f9321866640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93080b6700 0x7f93080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.764+0000 7f93127fc640 1 -- 172.21.15.31:0/191975779 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f93180e77c0 con 0x7f931c109fc0 2024-09-21T06:14:47.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.764+0000 7f93077fe640 1 -- 172.21.15.31:0/3614332471 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c011730 con 0x55885aaaa390 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f9321866640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93080b6700 0x7f93080b8c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9314005e30 tx=0x7f9314031c20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 -- 172.21.15.31:0/3614332471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 msgr2=0x55885ae732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 0x55885ae732e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f930c00bd30 tx=0x7f930c00f730 comp rx=0 tx=0).stop 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 -- 172.21.15.31:0/3614332471 shutdown_connections 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaaa390 0x55885ae732e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaa21b0 0x55885ae72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 --2- 172.21.15.31:0/3614332471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae72860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 -- 172.21.15.31:0/3614332471 >> 172.21.15.31:0/3614332471 conn(0x55885aabf460 msgr2=0x55885ae70750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 -- 172.21.15.31:0/3614332471 shutdown_connections 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 -- 172.21.15.31:0/3614332471 wait complete. 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.765+0000 7f932983d740 1 -- start start 2024-09-21T06:14:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 Processor -- start 2024-09-21T06:14:47.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaa21b0 0x55885ae7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aaaa390 0x55885ae7e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aacb070 0x55885ae7e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55885ac39700 con 0x55885aaaa390 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55885ad73160 con 0x55885aaa21b0 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.766+0000 7f932983d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55885ae81090 con 0x55885aacb070 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f932983d740 10 client.0 _ll_register_callbacks cb 0x55885ad8c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9320864640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aaaa390 0x55885ae7e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaa21b0 0x55885ae7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9320864640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aaaa390 0x55885ae7e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33564/0 (socket says 172.21.15.31:33564) 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9320864640 1 -- 172.21.15.31:0/2009713469 learned_addr learned my addr 172.21.15.31:0/2009713469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321866640 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aacb070 0x55885ae7e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 -- 172.21.15.31:0/2009713469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aacb070 msgr2=0x55885ae7e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aacb070 0x55885ae7e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 -- 172.21.15.31:0/2009713469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aaaa390 msgr2=0x55885ae7e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aaaa390 0x55885ae7e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 -- 172.21.15.31:0/2009713469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55885ada0c90 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9321065640 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaa21b0 0x55885ae7df40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93180e4f50 tx=0x7f93180098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b28bca33a4e391b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.767+0000 7f9304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f9304ff9640 1 -- 172.21.15.31:0/2009713469 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93180a94f0 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f9304ff9640 1 -- 172.21.15.31:0/2009713469 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f93180a8fa0 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c00be20 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f9304ff9640 1 -- 172.21.15.31:0/2009713469 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f93180acdc0 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 10 client.25850 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55885ae8cda0 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f9304ff9640 1 -- 172.21.15.31:0/2009713469 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c00be20 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f9304ff9640 1 -- 172.21.15.31:0/2009713469 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f93180a91a0 con 0x55885aaa21b0 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 10 client.25850 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:47.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 20 client.25850 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aa899f0 0x55885ae90400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 4 client.25850 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.768+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55885adeadf0 con 0x55885aa899f0 2024-09-21T06:14:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.769+0000 7f9320864640 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aa899f0 0x55885ae90400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.769+0000 7f9304ff9640 10 client.25850 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.769+0000 7f9320864640 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aa899f0 0x55885ae90400 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c9b2614496e09d server_cookie=2823d815f1f064d5 in_seq=0 out_seq=0 2024-09-21T06:14:47.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.770+0000 7f9304ff9640 1 -- 172.21.15.31:0/2009713469 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55885adeadf0 con 0x55885aa899f0 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.770+0000 7f9304ff9640 10 client.25850 handle_command_reply: tid=0 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.770+0000 7f932983d740 1 client.25850 shutdown 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaa21b0 msgr2=0x55885ae7df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaa21b0 0x55885ae7df40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93180e4f50 tx=0x7f93180098b0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 shutdown_connections 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aa899f0 0x55885ae90400 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aacb070 0x55885ae7e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aaaa390 0x55885ae7e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 --2- 172.21.15.31:0/2009713469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aaa21b0 0x55885ae7df40 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f93180e4f50 tx=0x7f93180098b0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 >> 172.21.15.31:0/2009713469 conn(0x55885aabf460 msgr2=0x55885ae759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:47.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f9304ff9640 0 client.25850 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 shutdown_connections 2024-09-21T06:14:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 1 -- 172.21.15.31:0/2009713469 wait complete. 2024-09-21T06:14:47.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.771+0000 7f932983d740 20 client.25850 trim_cache size 0 max 16384 2024-09-21T06:14:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:47 smithi110 ceph-mon[34550]: pgmap v415: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.1 MiB/s wr, 662 op/s 2024-09-21T06:14:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:14:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:14:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:14:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:14:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:47 smithi107 ceph-mon[34380]: pgmap v415: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 8.1 MiB/s wr, 662 op/s 2024-09-21T06:14:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:14:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:14:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:14:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.790+0000 7f932983d740 1 Processor -- start 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 -- start start 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aacb070 0x55885ae6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aa5eda0 0x55885ae70260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f930000c6f0 con 0x55885aa5eda0 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f930000ef70 con 0x55885aacb070 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f932983d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f930000cd60 con 0x55885aaaa390 2024-09-21T06:14:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f9321866640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aa5eda0 0x55885ae70260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f9321866640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aa5eda0 0x55885ae70260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33578/0 (socket says 172.21.15.31:33578) 2024-09-21T06:14:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f9321866640 1 -- 172.21.15.31:0/1470958467 learned_addr learned my addr 172.21.15.31:0/1470958467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.791+0000 7f9320864640 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aacb070 0x55885ae6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9321065640 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9320864640 1 -- 172.21.15.31:0/1470958467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 msgr2=0x55885ae73fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9320864640 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae73fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9320864640 1 -- 172.21.15.31:0/1470958467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aa5eda0 msgr2=0x55885ae70260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9320864640 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aa5eda0 0x55885ae70260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9320864640 1 -- 172.21.15.31:0/1470958467 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55885ae74b50 con 0x55885aacb070 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.792+0000 7f9320864640 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aacb070 0x55885ae6fd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55885ae90b70 tx=0x7f930c00ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9572820b4ba7deaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.793+0000 7f9306ffd640 1 -- 172.21.15.31:0/1470958467 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c004040 con 0x55885aacb070 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.793+0000 7f9306ffd640 1 -- 172.21.15.31:0/1470958467 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f930c0054f0 con 0x55885aacb070 2024-09-21T06:14:47.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.793+0000 7f9306ffd640 1 -- 172.21.15.31:0/1470958467 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c003a30 con 0x55885aacb070 2024-09-21T06:14:47.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 -- 172.21.15.31:0/1470958467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aacb070 msgr2=0x55885ae6fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aacb070 0x55885ae6fd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55885ae90b70 tx=0x7f930c00ba60 comp rx=0 tx=0).stop 2024-09-21T06:14:47.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 -- 172.21.15.31:0/1470958467 shutdown_connections 2024-09-21T06:14:47.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aa5eda0 0x55885ae70260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aacb070 0x55885ae6fd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 --2- 172.21.15.31:0/1470958467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 -- 172.21.15.31:0/1470958467 >> 172.21.15.31:0/1470958467 conn(0x55885aabf460 msgr2=0x55885ae73890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 -- 172.21.15.31:0/1470958467 shutdown_connections 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.794+0000 7f932983d740 1 -- 172.21.15.31:0/1470958467 wait complete. 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.795+0000 7f932983d740 1 -- start start 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.795+0000 7f932983d740 1 Processor -- start 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 0x55885ae86810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 0x55885ae86810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 0x55885ae86810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47720/0 (socket says 172.21.15.31:47720) 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 -- 172.21.15.31:0/1845580129 learned_addr learned my addr 172.21.15.31:0/1845580129 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f930000d430 con 0x55885aacb070 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321866640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9320864640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f930000e8a0 con 0x55885aa5eda0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f930000db00 con 0x55885aaaa390 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 10 client.0 _ll_register_callbacks cb 0x7f931800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 -- 172.21.15.31:0/1845580129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 msgr2=0x55885ae81cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae81cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 -- 172.21.15.31:0/1845580129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 msgr2=0x55885ae82200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae82200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f9321065640 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55885ae82740 con 0x55885aa5eda0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.796+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c0054f0 con 0x55885aa5eda0 2024-09-21T06:14:47.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f9321065640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 0x55885ae86810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f931800c760 tx=0x7f93180b6480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdce624b79f8e2cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f9304ff9640 10 client.25856 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f9304ff9640 1 -- 172.21.15.31:0/1845580129 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93180acfa0 con 0x55885aa5eda0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f9304ff9640 1 -- 172.21.15.31:0/1845580129 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f9318005310 con 0x55885aa5eda0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f9304ff9640 1 -- 172.21.15.31:0/1845580129 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f93180ad420 con 0x55885aa5eda0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f932983d740 10 client.25856 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55885aefa920 con 0x55885aa5eda0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.797+0000 7f9304ff9640 1 -- 172.21.15.31:0/1845580129 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f9318005530 con 0x55885aa5eda0 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f932983d740 10 client.25856 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f932983d740 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f932983d740 20 client.25856 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:47.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f932983d740 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aefaf60 0x55885aefd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f932983d740 4 client.25856 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55885ade68a0 con 0x55885aefaf60 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.798+0000 7f9320864640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aefaf60 0x55885aefd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.799+0000 7f9320864640 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aefaf60 0x55885aefd380 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5496ef33b041828 server_cookie=12571b02e000ac55 in_seq=0 out_seq=0 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.799+0000 7f9304ff9640 10 client.25856 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.799+0000 7f9304ff9640 1 -- 172.21.15.31:0/1845580129 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55885ade68a0 con 0x55885aefaf60 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.799+0000 7f9304ff9640 10 client.25856 handle_command_reply: tid=0 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.799+0000 7f932983d740 1 client.25856 shutdown 2024-09-21T06:14:47.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 msgr2=0x55885ae86810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 0x55885ae86810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f931800c760 tx=0x7f93180b6480 comp rx=0 tx=0).stop 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 shutdown_connections 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55885aefaf60 0x55885aefd380 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55885aacb070 0x55885ae82200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55885aaaa390 0x55885ae81cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 --2- 172.21.15.31:0/1845580129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55885aa5eda0 0x55885ae86810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 >> 172.21.15.31:0/1845580129 conn(0x55885aabf460 msgr2=0x55885ae7f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f9304ff9640 0 client.25856 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 shutdown_connections 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 1 -- 172.21.15.31:0/1845580129 wait complete. 2024-09-21T06:14:47.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.802+0000 7f932983d740 20 client.25856 trim_cache size 0 max 16384 2024-09-21T06:14:47.806 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:14:47.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93080b6700 msgr2=0x7f93080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93080b6700 0x7f93080b8c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9314005e30 tx=0x7f9314031c20 comp rx=0 tx=0).stop 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 msgr2=0x7f931c0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 0x7f931c0749d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f931800ea60 tx=0x7f931800ef20 comp rx=0 tx=0).stop 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 shutdown_connections 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f93080b6700 0x7f93080b8c10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f931c110250 0x7f931c06ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f931c10f900 0x7f931c074f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 --2- 172.21.15.31:0/191975779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f931c109fc0 0x7f931c0749d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 >> 172.21.15.31:0/191975779 conn(0x7f931c0795c0 msgr2=0x7f931c079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 shutdown_connections 2024-09-21T06:14:47.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:47.803+0000 7f9322067640 1 -- 172.21.15.31:0/191975779 wait complete. 2024-09-21T06:14:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:49 smithi031 ceph-mon[31237]: pgmap v416: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.2 MiB/s wr, 566 op/s 2024-09-21T06:14:49.742 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:14:49.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:14:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:49 smithi110 ceph-mon[34550]: pgmap v416: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.2 MiB/s wr, 566 op/s 2024-09-21T06:14:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:49 smithi107 ceph-mon[34380]: pgmap v416: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.2 MiB/s wr, 566 op/s 2024-09-21T06:14:50.200 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 190.0 s, 3610.7 tps, lat 28.121 ms stddev 32.173 2024-09-21T06:14:50.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 -- 172.21.15.31:0/818507625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c681083a0 msgr2=0x7f6c681087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/818507625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c681083a0 0x7f6c681087b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009860 tx=0x7f6c5802a340 comp rx=0 tx=0).stop 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 -- 172.21.15.31:0/818507625 shutdown_connections 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/818507625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c68108cf0 0x7f6c6810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/818507625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c681083a0 0x7f6c681087b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/818507625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c68102a60 0x7f6c68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 -- 172.21.15.31:0/818507625 >> 172.21.15.31:0/818507625 conn(0x7f6c68079730 msgr2=0x7f6c68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 -- 172.21.15.31:0/818507625 shutdown_connections 2024-09-21T06:14:50.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.812+0000 7f6c6d42f640 1 -- 172.21.15.31:0/818507625 wait complete. 2024-09-21T06:14:50.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.813+0000 7f6c6d42f640 1 Processor -- start 2024-09-21T06:14:50.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.813+0000 7f6c6d42f640 1 -- start start 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.814+0000 7f6c6d42f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c68102a60 0x7f6c681135e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.814+0000 7f6c6d42f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 0x7f6c68113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.814+0000 7f6c6d42f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c68108cf0 0x7f6c68118e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.814+0000 7f6c6d42f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68111850 con 0x7f6c68102a60 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.814+0000 7f6c6d42f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681116d0 con 0x7f6c68108cf0 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.814+0000 7f6c6d42f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681119d0 con 0x7f6c681083a0 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 0x7f6c68113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 0x7f6c68113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60780/0 (socket says 172.21.15.31:60780) 2024-09-21T06:14:50.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 -- 172.21.15.31:0/276471187 learned_addr learned my addr 172.21.15.31:0/276471187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:50.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c68102a60 0x7f6c681135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c677fe640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c68108cf0 0x7f6c68118e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 -- 172.21.15.31:0/276471187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c68108cf0 msgr2=0x7f6c68118e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c68108cf0 0x7f6c68118e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 -- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c68102a60 msgr2=0x7f6c681135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c68102a60 0x7f6c681135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c667fc640 1 -- 172.21.15.31:0/276471187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c681194f0 con 0x7f6c681083a0 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.815+0000 7f6c677fe640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c68108cf0 0x7f6c68118e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.816+0000 7f6c667fc640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 0x7f6c68113b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c58035040 tx=0x7f6c58032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.816+0000 7f6c4ffff640 1 -- 172.21.15.31:0/276471187 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58037040 con 0x7f6c681083a0 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.816+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c68119780 con 0x7f6c681083a0 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.816+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c68119c50 con 0x7f6c681083a0 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.816+0000 7f6c4ffff640 1 -- 172.21.15.31:0/276471187 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c58041070 con 0x7f6c681083a0 2024-09-21T06:14:50.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.816+0000 7f6c4ffff640 1 -- 172.21.15.31:0/276471187 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58003850 con 0x7f6c681083a0 2024-09-21T06:14:50.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.821+0000 7f6c4ffff640 1 -- 172.21.15.31:0/276471187 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6c580039f0 con 0x7f6c681083a0 2024-09-21T06:14:50.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.821+0000 7f6c739a3740 1 Processor -- start 2024-09-21T06:14:50.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.822+0000 7f6c739a3740 1 -- start start 2024-09-21T06:14:50.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.822+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed055430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.822+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed055970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.822+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed055eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.822+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed055970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.822+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed055970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36282/0 (socket says 172.21.15.31:36282) 2024-09-21T06:14:50.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed055430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed055430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60794/0 (socket says 172.21.15.31:60794) 2024-09-21T06:14:50.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 -- 172.21.15.31:0/490214628 learned_addr learned my addr 172.21.15.31:0/490214628 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:50.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563aecfcbe60 con 0x563aecc8d670 2024-09-21T06:14:50.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563aecfbb290 con 0x563aecc85490 2024-09-21T06:14:50.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563aecfbf0a0 con 0x563aeccae350 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 -- 172.21.15.31:0/490214628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 msgr2=0x563aed055430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed055430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 -- 172.21.15.31:0/490214628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 msgr2=0x563aed055eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed055eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 -- 172.21.15.31:0/490214628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aecd19930 con 0x563aecc85490 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.823+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed055970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e550 tx=0x7f6c600095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46baeb3c98694423 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.824+0000 7f6c4d7fa640 1 -- 172.21.15.31:0/490214628 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6001d070 con 0x563aecc85490 2024-09-21T06:14:50.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.824+0000 7f6c4d7fa640 1 -- 172.21.15.31:0/490214628 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c60003e40 con 0x563aecc85490 2024-09-21T06:14:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.824+0000 7f6c4d7fa640 1 -- 172.21.15.31:0/490214628 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000db70 con 0x563aecc85490 2024-09-21T06:14:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.825+0000 7f6c4ffff640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6c500b6700 0x7f6c500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.825+0000 7f6c4ffff640 1 -- 172.21.15.31:0/276471187 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6c58107290 con 0x7f6c681083a0 2024-09-21T06:14:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.825+0000 7f6c677fe640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6c500b6700 0x7f6c500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.825+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 msgr2=0x563aed055970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.825+0000 7f6c739a3740 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed055970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000e550 tx=0x7f6c600095c0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.827+0000 7f6c677fe640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6c500b6700 0x7f6c500b8c10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009c40 tx=0x7f6c5c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.827+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 shutdown_connections 2024-09-21T06:14:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.827+0000 7f6c739a3740 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed055eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.827+0000 7f6c739a3740 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed055970 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.827+0000 7f6c739a3740 1 --2- 172.21.15.31:0/490214628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed055430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.827+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 >> 172.21.15.31:0/490214628 conn(0x563aecca2740 msgr2=0x563aed053320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.830+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 shutdown_connections 2024-09-21T06:14:50.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.830+0000 7f6c739a3740 1 -- 172.21.15.31:0/490214628 wait complete. 2024-09-21T06:14:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.830+0000 7f6c739a3740 1 -- start start 2024-09-21T06:14:50.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.831+0000 7f6c739a3740 1 Processor -- start 2024-09-21T06:14:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.831+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed0610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.831+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed0616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed0610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed0610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36292/0 (socket says 172.21.15.31:36292) 2024-09-21T06:14:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 -- 172.21.15.31:0/2960531998 learned_addr learned my addr 172.21.15.31:0/2960531998 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed066f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563aece1c890 con 0x563aecc8d670 2024-09-21T06:14:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563aecf55d40 con 0x563aecc85490 2024-09-21T06:14:50.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563aed063c10 con 0x563aeccae350 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c667fc640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed0616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c739a3740 10 client.0 _ll_register_callbacks cb 0x563aecfb9c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c677fe640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 -- 172.21.15.31:0/2960531998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 msgr2=0x563aed066f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed066f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 -- 172.21.15.31:0/2960531998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 msgr2=0x563aed0616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed0616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c66ffd640 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aed067570 con 0x563aecc85490 2024-09-21T06:14:50.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.832+0000 7f6c677fe640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed066f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.834+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed0610f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f80 tx=0x7f6c6000dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=574eaeef00443e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.834+0000 7f6c4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:14:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.834+0000 7f6c4aff5640 1 -- 172.21.15.31:0/2960531998 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6001d040 con 0x563aecc85490 2024-09-21T06:14:50.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.835+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563aecfcddb0 con 0x563aecc85490 2024-09-21T06:14:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.835+0000 7f6c4aff5640 1 -- 172.21.15.31:0/2960531998 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6c6000dbd0 con 0x563aecc85490 2024-09-21T06:14:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.835+0000 7f6c4aff5640 1 -- 172.21.15.31:0/2960531998 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6c6000ddd0 con 0x563aecc85490 2024-09-21T06:14:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.835+0000 7f6c739a3740 10 client.25865 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.835+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563aed06ff00 con 0x563aecc85490 2024-09-21T06:14:50.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.836+0000 7f6c4aff5640 1 -- 172.21.15.31:0/2960531998 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563aecfcddb0 con 0x563aecc85490 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c4aff5640 1 -- 172.21.15.31:0/2960531998 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6c60023040 con 0x563aecc85490 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c739a3740 10 client.25865 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c739a3740 10 client.25865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c739a3740 20 client.25865 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc6ccd0 0x563aed073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c739a3740 4 client.25865 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:50.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.837+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aecfc9860 con 0x563aecc6ccd0 2024-09-21T06:14:50.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.838+0000 7f6c667fc640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc6ccd0 0x563aed073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.839+0000 7f6c4aff5640 10 client.25865 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:50.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.839+0000 7f6c667fc640 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc6ccd0 0x563aed073260 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1855391ca5d117 server_cookie=7f5865830b816d62 in_seq=0 out_seq=0 2024-09-21T06:14:50.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.840+0000 7f6c4aff5640 1 -- 172.21.15.31:0/2960531998 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563aecfc9860 con 0x563aecc6ccd0 2024-09-21T06:14:50.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.840+0000 7f6c4aff5640 10 client.25865 handle_command_reply: tid=0 2024-09-21T06:14:50.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.840+0000 7f6c739a3740 1 client.25865 shutdown 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 msgr2=0x563aed0610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed0610f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f80 tx=0x7f6c6000dae0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 shutdown_connections 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc6ccd0 0x563aed073260 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aeccae350 0x563aed066f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed0616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c4aff5640 0 client.25865 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:50.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 --2- 172.21.15.31:0/2960531998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed0610f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 >> 172.21.15.31:0/2960531998 conn(0x563aecca2740 msgr2=0x563aed0585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 shutdown_connections 2024-09-21T06:14:50.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 1 -- 172.21.15.31:0/2960531998 wait complete. 2024-09-21T06:14:50.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.841+0000 7f6c739a3740 20 client.25865 trim_cache size 0 max 16384 2024-09-21T06:14:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.876+0000 7f6c739a3740 1 Processor -- start 2024-09-21T06:14:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.877+0000 7f6c739a3740 1 -- start start 2024-09-21T06:14:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.877+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc85490 0x563aed058c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed066e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aeccae350 0x563aed05a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c739a3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68071ce0 con 0x563aecc8d670 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c739a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680744b0 con 0x563aeccae350 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c739a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680722a0 con 0x563aecc85490 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aeccae350 0x563aed05a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aeccae350 0x563aed05a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36296/0 (socket says 172.21.15.31:36296) 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed066e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c677fe640 1 -- 172.21.15.31:0/4044604274 learned_addr learned my addr 172.21.15.31:0/4044604274 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc85490 0x563aed058c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c66ffd640 1 -- 172.21.15.31:0/4044604274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aeccae350 msgr2=0x563aed05a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aeccae350 0x563aed05a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c66ffd640 1 -- 172.21.15.31:0/4044604274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 msgr2=0x563aed066e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed066e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.878+0000 7f6c66ffd640 1 -- 172.21.15.31:0/4044604274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aed061f90 con 0x563aecc85490 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.879+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc85490 0x563aed058c40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f80 tx=0x7f6c60003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f35c975cba4623f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:50.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.879+0000 7f6c4cff9640 1 -- 172.21.15.31:0/4044604274 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6001d040 con 0x563aecc85490 2024-09-21T06:14:50.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.879+0000 7f6c4cff9640 1 -- 172.21.15.31:0/4044604274 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6000d630 con 0x563aecc85490 2024-09-21T06:14:50.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.879+0000 7f6c4cff9640 1 -- 172.21.15.31:0/4044604274 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60004c00 con 0x563aecc85490 2024-09-21T06:14:50.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 -- 172.21.15.31:0/4044604274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc85490 msgr2=0x563aed058c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc85490 0x563aed058c40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f80 tx=0x7f6c60003dd0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 -- 172.21.15.31:0/4044604274 shutdown_connections 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aeccae350 0x563aed05a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aecc8d670 0x563aed066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 --2- 172.21.15.31:0/4044604274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc85490 0x563aed058c40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 -- 172.21.15.31:0/4044604274 >> 172.21.15.31:0/4044604274 conn(0x563aecca2740 msgr2=0x563aed058530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 -- 172.21.15.31:0/4044604274 shutdown_connections 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.881+0000 7f6c739a3740 1 -- 172.21.15.31:0/4044604274 wait complete. 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.882+0000 7f6c739a3740 1 -- start start 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.882+0000 7f6c739a3740 1 Processor -- start 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 0x563aed069310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeccae350 0x563aed069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68072970 con 0x563aeccae350 2024-09-21T06:14:50.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68073de0 con 0x563aecc85490 2024-09-21T06:14:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68073040 con 0x563aecc8d670 2024-09-21T06:14:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c739a3740 10 client.0 _ll_register_callbacks cb 0x7f6c6001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 0x563aed069310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.883+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 0x563aed069310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60802/0 (socket says 172.21.15.31:60802) 2024-09-21T06:14:50.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 -- 172.21.15.31:0/206089308 learned_addr learned my addr 172.21.15.31:0/206089308 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c677fe640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeccae350 0x563aed069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 -- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 msgr2=0x563aed068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed068dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 -- 172.21.15.31:0/206089308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeccae350 msgr2=0x563aed069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:50.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeccae350 0x563aed069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 -- 172.21.15.31:0/206089308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aed0dc9d0 con 0x563aecc8d670 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c667fc640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 0x563aed069310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c5802a930 tx=0x7f6c58005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f1ca19bc50c73d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.884+0000 7f6c4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.885+0000 7f6c4aff5640 1 -- 172.21.15.31:0/206089308 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58030e70 con 0x563aecc8d670 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.885+0000 7f6c4aff5640 1 -- 172.21.15.31:0/206089308 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6c580cc3b0 con 0x563aecc8d670 2024-09-21T06:14:50.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.885+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6000d630 con 0x563aecc8d670 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.885+0000 7f6c4aff5640 1 -- 172.21.15.31:0/206089308 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6c58030450 con 0x563aecc8d670 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.885+0000 7f6c739a3740 10 client.25906 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.885+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563aed0ddc30 con 0x563aecc8d670 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c4aff5640 1 -- 172.21.15.31:0/206089308 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6000d630 con 0x563aecc8d670 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c4aff5640 1 -- 172.21.15.31:0/206089308 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6c580e5020 con 0x563aecc8d670 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c739a3740 10 client.25906 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c739a3740 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:50.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c739a3740 20 client.25906 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c739a3740 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc42080 0x563aed0e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c739a3740 4 client.25906 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:14:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.886+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563aecfc6c90 con 0x563aecc42080 2024-09-21T06:14:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.887+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc42080 0x563aed0e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.889+0000 7f6c4aff5640 10 client.25906 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:50.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.889+0000 7f6c66ffd640 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc42080 0x563aed0e03a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba35e1946604fe9f server_cookie=e5f4c45d394017e5 in_seq=0 out_seq=0 2024-09-21T06:14:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c4aff5640 1 -- 172.21.15.31:0/206089308 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563aecfc6c90 con 0x563aecc42080 2024-09-21T06:14:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c4aff5640 10 client.25906 handle_command_reply: tid=0 2024-09-21T06:14:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c739a3740 1 client.25906 shutdown 2024-09-21T06:14:50.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 msgr2=0x563aed069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c739a3740 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 0x563aed069310 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c5802a930 tx=0x7f6c58005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 shutdown_connections 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.891+0000 7f6c739a3740 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563aecc42080 0x563aed0e03a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c739a3740 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563aeccae350 0x563aed069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c739a3740 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563aecc8d670 0x563aed069310 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c739a3740 1 --2- 172.21.15.31:0/206089308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563aecc85490 0x563aed068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 >> 172.21.15.31:0/206089308 conn(0x563aecca2740 msgr2=0x563aed056460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c4aff5640 0 client.25906 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 shutdown_connections 2024-09-21T06:14:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.892+0000 7f6c739a3740 1 -- 172.21.15.31:0/206089308 wait complete. 2024-09-21T06:14:50.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.894+0000 7f6c739a3740 20 client.25906 trim_cache size 0 max 16384 2024-09-21T06:14:50.899 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:14:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "e621855d-93b4-415c-97d3-d2c5305a6bfc", 2024-09-21T06:14:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:14:50.899 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.896+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6c500b6700 msgr2=0x7f6c500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6c500b6700 0x7f6c500b8c10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009c40 tx=0x7f6c5c005f90 comp rx=0 tx=0).stop 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 msgr2=0x7f6c68113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 0x7f6c68113b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c58035040 tx=0x7f6c58032d50 comp rx=0 tx=0).stop 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 shutdown_connections 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c68108cf0 0x7f6c68118e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c681083a0 0x7f6c68113b60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6c500b6700 0x7f6c500b8c10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 --2- 172.21.15.31:0/276471187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c68102a60 0x7f6c681135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 >> 172.21.15.31:0/276471187 conn(0x7f6c68079730 msgr2=0x7f6c681044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 shutdown_connections 2024-09-21T06:14:50.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:50.897+0000 7f6c6d42f640 1 -- 172.21.15.31:0/276471187 wait complete. 2024-09-21T06:14:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:51 smithi031 ceph-mon[31237]: pgmap v417: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 530 op/s 2024-09-21T06:14:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:51 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:14:51.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:51 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:51.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:51 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:14:51.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:14:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:51 smithi110 ceph-mon[34550]: pgmap v417: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 530 op/s 2024-09-21T06:14:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:51 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:14:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:51 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:51 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:14:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:51 smithi107 ceph-mon[34380]: pgmap v417: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 530 op/s 2024-09-21T06:14:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:51 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:14:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:51 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:51 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:14:52.211 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:52.632 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:52 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:52.632 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:52 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:14:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:52 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:52 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:14:52.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.777+0000 7fbf6223b640 1 -- 172.21.15.31:0/1204574602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c1083a0 msgr2=0x7fbf5c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.777+0000 7fbf6223b640 1 --2- 172.21.15.31:0/1204574602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c1087b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009900 tx=0x7fbf5002a410 comp rx=0 tx=0).stop 2024-09-21T06:14:52.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.778+0000 7fbf6223b640 1 -- 172.21.15.31:0/1204574602 shutdown_connections 2024-09-21T06:14:52.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.778+0000 7fbf6223b640 1 --2- 172.21.15.31:0/1204574602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.778+0000 7fbf6223b640 1 --2- 172.21.15.31:0/1204574602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c1087b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.778+0000 7fbf6223b640 1 --2- 172.21.15.31:0/1204574602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c102a60 0x7fbf5c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.778+0000 7fbf6223b640 1 -- 172.21.15.31:0/1204574602 >> 172.21.15.31:0/1204574602 conn(0x7fbf5c079730 msgr2=0x7fbf5c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:52.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.778+0000 7fbf6223b640 1 -- 172.21.15.31:0/1204574602 shutdown_connections 2024-09-21T06:14:52.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.779+0000 7fbf6223b640 1 -- 172.21.15.31:0/1204574602 wait complete. 2024-09-21T06:14:52.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.780+0000 7fbf6223b640 1 Processor -- start 2024-09-21T06:14:52.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.780+0000 7fbf6223b640 1 -- start start 2024-09-21T06:14:52.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.780+0000 7fbf6223b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 0x7fbf5c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.780+0000 7fbf6223b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf6223b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 0x7fbf5c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf6223b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1118a0 con 0x7fbf5c1083a0 2024-09-21T06:14:52.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf6223b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c111720 con 0x7fbf5c102a60 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf6223b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c111a20 con 0x7fbf5c108cf0 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 0x7fbf5c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36322/0 (socket says 172.21.15.31:36322) 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 -- 172.21.15.31:0/2116018615 learned_addr learned my addr 172.21.15.31:0/2116018615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5affd640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 -- 172.21.15.31:0/2116018615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c108cf0 msgr2=0x7fbf5c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 -- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf5c1083a0 msgr2=0x7fbf5c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 -- 172.21.15.31:0/2116018615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c19e220 con 0x7fbf5c102a60 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:52.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.781+0000 7fbf5b7fe640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 0x7fbf5c1982b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d950 tx=0x7fbf4c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:52.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.782+0000 7fbf58ff9640 1 -- 172.21.15.31:0/2116018615 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c017030 con 0x7fbf5c102a60 2024-09-21T06:14:52.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.782+0000 7fbf58ff9640 1 -- 172.21.15.31:0/2116018615 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf4c004020 con 0x7fbf5c102a60 2024-09-21T06:14:52.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.782+0000 7fbf58ff9640 1 -- 172.21.15.31:0/2116018615 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0047a0 con 0x7fbf5c102a60 2024-09-21T06:14:52.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.784+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c19e510 con 0x7fbf5c102a60 2024-09-21T06:14:52.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.784+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1a4c70 con 0x7fbf5c102a60 2024-09-21T06:14:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:52 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:14:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:52 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:14:52.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.788+0000 7fbf687af740 1 Processor -- start 2024-09-21T06:14:52.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.788+0000 7fbf687af740 1 -- start start 2024-09-21T06:14:52.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.788+0000 7fbf58ff9640 1 -- 172.21.15.31:0/2116018615 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbf4c009a30 con 0x7fbf5c102a60 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.788+0000 7fbf687af740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf687af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf687af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb306390 0x558bfb6d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf687af740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558bfb644e50 con 0x558bfb327070 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf687af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558bfb634280 con 0x558bfb2fe1b0 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf687af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558bfb638090 con 0x558bfb306390 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6d6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38158/0 (socket says 172.21.15.31:38158) 2024-09-21T06:14:52.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 -- 172.21.15.31:0/2587032652 learned_addr learned my addr 172.21.15.31:0/2587032652 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5affd640 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb306390 0x558bfb6d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 -- 172.21.15.31:0/2587032652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb306390 msgr2=0x558bfb6d78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb306390 0x558bfb6d78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 -- 172.21.15.31:0/2587032652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 msgr2=0x558bfb6d73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.789+0000 7fbf5bfff640 1 -- 172.21.15.31:0/2587032652 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bfb392650 con 0x558bfb327070 2024-09-21T06:14:52.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6d6e60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009f10 tx=0x7fbf5400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5d70c0edca0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:52.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf41ffb640 1 -- 172.21.15.31:0/2587032652 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54003b70 con 0x558bfb327070 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf58ff9640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b67d0 0x7fbf440b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf41ffb640 1 -- 172.21.15.31:0/2587032652 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf54003d10 con 0x558bfb327070 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf41ffb640 1 -- 172.21.15.31:0/2587032652 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf540118b0 con 0x558bfb327070 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf58ff9640 1 -- 172.21.15.31:0/2116018615 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbf4c0e66e0 con 0x7fbf5c102a60 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.790+0000 7fbf5b7fe640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b67d0 0x7fbf440b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.791+0000 7fbf687af740 1 -- 172.21.15.31:0/2587032652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 msgr2=0x558bfb6d6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.791+0000 7fbf687af740 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6d6e60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009f10 tx=0x7fbf5400f730 comp rx=0 tx=0).stop 2024-09-21T06:14:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 -- 172.21.15.31:0/2587032652 shutdown_connections 2024-09-21T06:14:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb306390 0x558bfb6d78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 --2- 172.21.15.31:0/2587032652 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6d6e60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 -- 172.21.15.31:0/2587032652 >> 172.21.15.31:0/2587032652 conn(0x558bfb31b460 msgr2=0x558bfb6d4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 -- 172.21.15.31:0/2587032652 shutdown_connections 2024-09-21T06:14:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf687af740 1 -- 172.21.15.31:0/2587032652 wait complete. 2024-09-21T06:14:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.793+0000 7fbf5b7fe640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b67d0 0x7fbf440b8ce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0e3f20 tx=0x7fbf4c0ec550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.794+0000 7fbf687af740 1 -- start start 2024-09-21T06:14:52.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.794+0000 7fbf687af740 1 Processor -- start 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf687af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf687af740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf687af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf687af740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558bfb495700 con 0x558bfb306390 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf687af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558bfb5cf110 con 0x558bfb2fe1b0 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf687af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558bfb6e5690 con 0x558bfb327070 2024-09-21T06:14:52.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf5affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38162/0 (socket says 172.21.15.31:38162) 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6e24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36338/0 (socket says 172.21.15.31:36338) 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.795+0000 7fbf5affd640 1 -- 172.21.15.31:0/69408019 learned_addr learned my addr 172.21.15.31:0/69408019 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf687af740 10 client.0 _ll_register_callbacks cb 0x558bfb5e8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5bfff640 1 -- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 msgr2=0x558bfb6e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5bfff640 1 -- 172.21.15.31:0/69408019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 msgr2=0x558bfb6e29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5bfff640 1 -- 172.21.15.31:0/69408019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bfb5fcc40 con 0x558bfb327070 2024-09-21T06:14:52.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf54003d10 con 0x558bfb327070 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.796+0000 7fbf5affd640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.797+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6e2f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf540114f0 tx=0x7fbf54000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2192b4bec6e6604d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.797+0000 7fbf3affd640 10 client.25915 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.797+0000 7fbf3affd640 1 -- 172.21.15.31:0/69408019 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54025070 con 0x558bfb327070 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf3affd640 1 -- 172.21.15.31:0/69408019 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbf540042d0 con 0x558bfb327070 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf3affd640 1 -- 172.21.15.31:0/69408019 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbf540044d0 con 0x558bfb327070 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 10 client.25915 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x558bfb6f12c0 con 0x558bfb327070 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf3affd640 1 -- 172.21.15.31:0/69408019 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fbf54011550 con 0x558bfb327070 2024-09-21T06:14:52.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 10 client.25915 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 20 client.25915 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2e59f0 0x558bfb6f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 4 client.25915 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.798+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bfb646da0 con 0x558bfb2e59f0 2024-09-21T06:14:52.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.799+0000 7fbf5affd640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2e59f0 0x558bfb6f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.801+0000 7fbf5affd640 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2e59f0 0x558bfb6f4620 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f921d0ee51b755f server_cookie=a097b33c0c943543 in_seq=0 out_seq=0 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.802+0000 7fbf3affd640 10 client.25915 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.802+0000 7fbf3affd640 1 -- 172.21.15.31:0/69408019 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558bfb646da0 con 0x558bfb2e59f0 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.802+0000 7fbf3affd640 10 client.25915 handle_command_reply: tid=0 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.802+0000 7fbf687af740 1 client.25915 shutdown 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 msgr2=0x558bfb6e2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6e2f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf540114f0 tx=0x7fbf54000f40 comp rx=0 tx=0).stop 2024-09-21T06:14:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 shutdown_connections 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2e59f0 0x558bfb6f4620 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6e2f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 --2- 172.21.15.31:0/69408019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf3affd640 0 client.25915 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 >> 172.21.15.31:0/69408019 conn(0x558bfb31b460 msgr2=0x558bfb6da040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 shutdown_connections 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 1 -- 172.21.15.31:0/69408019 wait complete. 2024-09-21T06:14:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.803+0000 7fbf687af740 20 client.25915 trim_cache size 0 max 16384 2024-09-21T06:14:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 Processor -- start 2024-09-21T06:14:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 -- start start 2024-09-21T06:14:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6ed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ab4c0 con 0x558bfb306390 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1add40 con 0x558bfb2fe1b0 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf687af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1abb30 con 0x558bfb327070 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36346/0 (socket says 172.21.15.31:36346) 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf5bfff640 1 -- 172.21.15.31:0/103515898 learned_addr learned my addr 172.21.15.31:0/103515898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.835+0000 7fbf5affd640 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 -- 172.21.15.31:0/103515898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 msgr2=0x558bfb6ed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6ed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 -- 172.21.15.31:0/103515898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 msgr2=0x558bfb6e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 -- 172.21.15.31:0/103515898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bfb6e5520 con 0x558bfb2fe1b0 2024-09-21T06:14:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.836+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d8970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbf54005fd0 tx=0x7fbf5401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54520ba65baead2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.838+0000 7fbf417fa640 1 -- 172.21.15.31:0/103515898 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54041020 con 0x558bfb2fe1b0 2024-09-21T06:14:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.838+0000 7fbf417fa640 1 -- 172.21.15.31:0/103515898 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf54003d10 con 0x558bfb2fe1b0 2024-09-21T06:14:52.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.838+0000 7fbf417fa640 1 -- 172.21.15.31:0/103515898 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54011520 con 0x558bfb2fe1b0 2024-09-21T06:14:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 -- 172.21.15.31:0/103515898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 msgr2=0x558bfb6d8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d8970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbf54005fd0 tx=0x7fbf5401c000 comp rx=0 tx=0).stop 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 -- 172.21.15.31:0/103515898 shutdown_connections 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb327070 0x558bfb6ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb306390 0x558bfb6e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 --2- 172.21.15.31:0/103515898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb2fe1b0 0x558bfb6d8970 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 -- 172.21.15.31:0/103515898 >> 172.21.15.31:0/103515898 conn(0x558bfb31b460 msgr2=0x558bfb6dbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.840+0000 7fbf687af740 1 -- 172.21.15.31:0/103515898 shutdown_connections 2024-09-21T06:14:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.841+0000 7fbf687af740 1 -- 172.21.15.31:0/103515898 wait complete. 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.841+0000 7fbf687af740 1 -- start start 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.841+0000 7fbf687af740 1 Processor -- start 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 0x558bfb6f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb306390 0x558bfb6f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ac200 con 0x558bfb327070 2024-09-21T06:14:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ad670 con 0x558bfb306390 2024-09-21T06:14:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ac8d0 con 0x558bfb2fe1b0 2024-09-21T06:14:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 0x558bfb6f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf687af740 10 client.0 _ll_register_callbacks cb 0x7fbf4c0ec420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 0x558bfb6f7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60838/0 (socket says 172.21.15.31:60838) 2024-09-21T06:14:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf5bfff640 1 -- 172.21.15.31:0/1990644893 learned_addr learned my addr 172.21.15.31:0/1990644893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.842+0000 7fbf5affd640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb306390 0x558bfb6f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf5bfff640 1 -- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb306390 msgr2=0x558bfb6f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb306390 0x558bfb6f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf5bfff640 1 -- 172.21.15.31:0/1990644893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 msgr2=0x558bfb6f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf5bfff640 1 -- 172.21.15.31:0/1990644893 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bfb6f5d50 con 0x558bfb2fe1b0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf5bfff640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 0x558bfb6f7cb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbf540049d0 tx=0x7fbf540048b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db5c376799280a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf3affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf3affd640 1 -- 172.21.15.31:0/1990644893 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54041070 con 0x558bfb2fe1b0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf54003d10 con 0x558bfb2fe1b0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf3affd640 1 -- 172.21.15.31:0/1990644893 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fbf54011ba0 con 0x558bfb2fe1b0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf3affd640 1 -- 172.21.15.31:0/1990644893 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbf54025040 con 0x558bfb2fe1b0 2024-09-21T06:14:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf687af740 10 client.25921 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.843+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558bfb75eb60 con 0x558bfb2fe1b0 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf3affd640 1 -- 172.21.15.31:0/1990644893 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf54003d10 con 0x558bfb2fe1b0 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf3affd640 1 -- 172.21.15.31:0/1990644893 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fbf54046040 con 0x558bfb2fe1b0 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf687af740 10 client.25921 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf687af740 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf687af740 20 client.25921 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf687af740 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2bada0 0x558bfb761190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf687af740 4 client.25921 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:14:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bfb642850 con 0x558bfb2bada0 2024-09-21T06:14:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.844+0000 7fbf5affd640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2bada0 0x558bfb761190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.845+0000 7fbf5affd640 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2bada0 0x558bfb761190 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd0704247b4e825 server_cookie=f473b1725d42af7b in_seq=0 out_seq=0 2024-09-21T06:14:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.845+0000 7fbf3affd640 10 client.25921 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.846+0000 7fbf3affd640 1 -- 172.21.15.31:0/1990644893 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x558bfb642850 con 0x558bfb2bada0 2024-09-21T06:14:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.846+0000 7fbf3affd640 10 client.25921 handle_command_reply: tid=0 2024-09-21T06:14:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.847+0000 7fbf687af740 1 client.25921 shutdown 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 msgr2=0x558bfb6f7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 0x558bfb6f7cb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbf540049d0 tx=0x7fbf540048b0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 shutdown_connections 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558bfb2bada0 0x558bfb761190 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558bfb327070 0x558bfb6f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558bfb306390 0x558bfb6f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 --2- 172.21.15.31:0/1990644893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558bfb2fe1b0 0x558bfb6f7cb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf3affd640 0 client.25921 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 >> 172.21.15.31:0/1990644893 conn(0x558bfb31b460 msgr2=0x558bfb6dc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 shutdown_connections 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 1 -- 172.21.15.31:0/1990644893 wait complete. 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.848+0000 7fbf687af740 20 client.25921 trim_cache size 0 max 16384 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:14:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:14:52.860 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b67d0 msgr2=0x7fbf440b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b67d0 0x7fbf440b8ce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0e3f20 tx=0x7fbf4c0ec550 comp rx=0 tx=0).stop 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 msgr2=0x7fbf5c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 0x7fbf5c1982b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d950 tx=0x7fbf4c00de10 comp rx=0 tx=0).stop 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 shutdown_connections 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf5c108cf0 0x7fbf5c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b67d0 0x7fbf440b8ce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf5c1083a0 0x7fbf5c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 --2- 172.21.15.31:0/2116018615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf5c102a60 0x7fbf5c1982b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.849+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 >> 172.21.15.31:0/2116018615 conn(0x7fbf5c079730 msgr2=0x7fbf5c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.850+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 shutdown_connections 2024-09-21T06:14:52.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:52.850+0000 7fbf6223b640 1 -- 172.21.15.31:0/2116018615 wait complete. 2024-09-21T06:14:53.631 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:53 smithi031 ceph-mon[31237]: pgmap v418: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-21T06:14:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:53 smithi110 ceph-mon[34550]: pgmap v418: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-21T06:14:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:53 smithi107 ceph-mon[34380]: pgmap v418: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 760 op/s 2024-09-21T06:14:54.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:14:54.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:14:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:55.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 -- 172.21.15.31:0/2897682491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd01083d0 msgr2=0x7f7fd01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 --2- 172.21.15.31:0/2897682491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd01083d0 0x7f7fd01087e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009900 tx=0x7f7fb802a420 comp rx=0 tx=0).stop 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 -- 172.21.15.31:0/2897682491 shutdown_connections 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 --2- 172.21.15.31:0/2897682491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0108d20 0x7f7fd010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 --2- 172.21.15.31:0/2897682491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd01083d0 0x7f7fd01087e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 --2- 172.21.15.31:0/2897682491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 -- 172.21.15.31:0/2897682491 >> 172.21.15.31:0/2897682491 conn(0x7f7fd00797b0 msgr2=0x7f7fd0079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 -- 172.21.15.31:0/2897682491 shutdown_connections 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.059+0000 7f7fd589d640 1 -- 172.21.15.31:0/2897682491 wait complete. 2024-09-21T06:14:55.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.060+0000 7f7fd589d640 1 Processor -- start 2024-09-21T06:14:55.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.061+0000 7f7fd589d640 1 -- start start 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fd589d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fd589d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01083d0 0x7f7fd01999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fd589d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd0108d20 0x7f7fd019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fd589d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112a70 con 0x7f7fd01083d0 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fd589d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd01128f0 con 0x7f7fd0108d20 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fd589d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112bf0 con 0x7f7fd0102ae0 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fceffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd0108d20 0x7f7fd019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fc7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01083d0 0x7f7fd01999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fceffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60850/0 (socket says 172.21.15.31:60850) 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd0108d20 0x7f7fd019ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36368/0 (socket says 172.21.15.31:36368) 2024-09-21T06:14:55.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.062+0000 7f7fceffd640 1 -- 172.21.15.31:0/1497503782 learned_addr learned my addr 172.21.15.31:0/1497503782 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:55.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fceffd640 1 -- 172.21.15.31:0/1497503782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd0108d20 msgr2=0x7f7fd019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fceffd640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd0108d20 0x7f7fd019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fceffd640 1 -- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01083d0 msgr2=0x7f7fd01999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fceffd640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01083d0 0x7f7fd01999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fceffd640 1 -- 172.21.15.31:0/1497503782 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd019f3b0 con 0x7f7fd0102ae0 2024-09-21T06:14:55.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fceffd640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0199440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d950 tx=0x7f7fc000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017070 con 0x7f7fd0102ae0 2024-09-21T06:14:55.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc0009c30 con 0x7f7fd0102ae0 2024-09-21T06:14:55.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.063+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd019f6a0 con 0x7f7fd0102ae0 2024-09-21T06:14:55.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.064+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000f620 con 0x7f7fd0102ae0 2024-09-21T06:14:55.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.064+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd006b820 con 0x7f7fd0102ae0 2024-09-21T06:14:55.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.065+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd0103d10 con 0x7f7fd0102ae0 2024-09-21T06:14:55.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.067+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7fc0005030 con 0x7f7fd0102ae0 2024-09-21T06:14:55.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.068+0000 7f7fd489b640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fbc0b6880 0x7f7fbc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.069+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7fc00e71d0 con 0x7f7fd0102ae0 2024-09-21T06:14:55.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.069+0000 7f7fc7fff640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fbc0b6880 0x7f7fbc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.070+0000 7f7fc7fff640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fbc0b6880 0x7f7fbc0b8d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7fb802aa10 tx=0x7f7fb8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.071+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7fc00b1be0 con 0x7f7fd0102ae0 2024-09-21T06:14:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 195.0 s, 4042.2 tps, lat 24.670 ms stddev 29.915 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 -- 172.21.15.31:0/3092793158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c108c30 msgr2=0x7f141c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 --2- 172.21.15.31:0/3092793158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c108c30 0x7f141c06abe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1414009be0 tx=0x7f141402a6d0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 -- 172.21.15.31:0/3092793158 shutdown_connections 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 --2- 172.21.15.31:0/3092793158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c108c30 0x7f141c06abe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 --2- 172.21.15.31:0/3092793158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c1082e0 0x7f141c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 --2- 172.21.15.31:0/3092793158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1029b0 0x7f141c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 -- 172.21.15.31:0/3092793158 >> 172.21.15.31:0/3092793158 conn(0x7f141c07a740 msgr2=0x7f141c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 -- 172.21.15.31:0/3092793158 shutdown_connections 2024-09-21T06:14:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.254+0000 7f14241e4640 1 -- 172.21.15.31:0/3092793158 wait complete. 2024-09-21T06:14:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.255+0000 7f14241e4640 1 Processor -- start 2024-09-21T06:14:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.255+0000 7f14241e4640 1 -- start start 2024-09-21T06:14:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f14241e4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c1029b0 0x7f141c10b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f1421f80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c1029b0 0x7f141c10b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f1421f80640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c1029b0 0x7f141c10b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60860/0 (socket says 172.21.15.31:60860) 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f14241e4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 0x7f141c10b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f14241e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c108c30 0x7f141c10bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f14241e4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1139f0 con 0x7f141c108c30 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f14241e4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f141c113870 con 0x7f141c1082e0 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f14241e4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f141c113b70 con 0x7f141c1029b0 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f142177f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 0x7f141c10b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f142177f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 0x7f141c10b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36388/0 (socket says 172.21.15.31:36388) 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.256+0000 7f142177f640 1 -- 172.21.15.31:0/2886104318 learned_addr learned my addr 172.21.15.31:0/2886104318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f1422781640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c108c30 0x7f141c10bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f142177f640 1 -- 172.21.15.31:0/2886104318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c1029b0 msgr2=0x7f141c10b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f142177f640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c1029b0 0x7f141c10b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f142177f640 1 -- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c108c30 msgr2=0x7f141c10bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f142177f640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c108c30 0x7f141c10bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f142177f640 1 -- 172.21.15.31:0/2886104318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c074020 con 0x7f141c1082e0 2024-09-21T06:14:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.257+0000 7f1422781640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c108c30 0x7f141c10bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:14:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.258+0000 7f142177f640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 0x7f141c10b980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f140c00e9f0 tx=0x7f140c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.259+0000 7f1412ffd640 1 -- 172.21.15.31:0/2886104318 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c00ccc0 con 0x7f141c1082e0 2024-09-21T06:14:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.259+0000 7f1412ffd640 1 -- 172.21.15.31:0/2886104318 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f140c0108e0 con 0x7f141c1082e0 2024-09-21T06:14:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.259+0000 7f1412ffd640 1 -- 172.21.15.31:0/2886104318 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c018880 con 0x7f141c1082e0 2024-09-21T06:14:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.259+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c074310 con 0x7f141c1082e0 2024-09-21T06:14:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.259+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c074810 con 0x7f141c1082e0 2024-09-21T06:14:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.262+0000 7f142b234740 1 Processor -- start 2024-09-21T06:14:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 -- start start 2024-09-21T06:14:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 0x559464f709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559464ee56f0 con 0x559464ba22f0 2024-09-21T06:14:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559464ed4b20 con 0x559464bcb1b0 2024-09-21T06:14:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f142b234740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559464ed8930 con 0x559464baa4d0 2024-09-21T06:14:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f1422781640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 0x559464f709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f1422781640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 0x559464f709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36398/0 (socket says 172.21.15.31:36398) 2024-09-21T06:14:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.263+0000 7f1422781640 1 -- 172.21.15.31:0/393650887 learned_addr learned my addr 172.21.15.31:0/393650887 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.264+0000 7f1422781640 1 -- 172.21.15.31:0/393650887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 msgr2=0x559464f71430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.264+0000 7f1422781640 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f71430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.264+0000 7f1422781640 1 -- 172.21.15.31:0/393650887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 msgr2=0x559464f70ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.264+0000 7f1422781640 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f70ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.264+0000 7f1422781640 1 -- 172.21.15.31:0/393650887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559464e9a530 con 0x559464bcb1b0 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.264+0000 7f1422781640 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 0x559464f709b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1414009300 tx=0x7f1414009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96bfbaa6aff1cf73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.265+0000 7f1412ffd640 1 -- 172.21.15.31:0/2886104318 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f140c010430 con 0x7f141c1082e0 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.265+0000 7f1407fff640 1 -- 172.21.15.31:0/393650887 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414040070 con 0x559464bcb1b0 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.265+0000 7f1407fff640 1 -- 172.21.15.31:0/393650887 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1414003e20 con 0x559464bcb1b0 2024-09-21T06:14:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.265+0000 7f1407fff640 1 -- 172.21.15.31:0/393650887 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414036040 con 0x559464bcb1b0 2024-09-21T06:14:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.266+0000 7f1412ffd640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14080b6700 0x7f14080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.266+0000 7f142b234740 1 -- 172.21.15.31:0/393650887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 msgr2=0x559464f709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.266+0000 7f142b234740 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 0x559464f709b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1414009300 tx=0x7f1414009330 comp rx=0 tx=0).stop 2024-09-21T06:14:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.266+0000 7f1412ffd640 1 -- 172.21.15.31:0/2886104318 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f140c0e7960 con 0x7f141c1082e0 2024-09-21T06:14:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f1421f80640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14080b6700 0x7f14080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f142b234740 1 -- 172.21.15.31:0/393650887 shutdown_connections 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f142b234740 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f71430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f142b234740 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f70ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f142b234740 1 --2- 172.21.15.31:0/393650887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464bcb1b0 0x559464f709b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f142b234740 1 -- 172.21.15.31:0/393650887 >> 172.21.15.31:0/393650887 conn(0x559464bbf300 msgr2=0x559464f6e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.267+0000 7f142b234740 1 -- 172.21.15.31:0/393650887 shutdown_connections 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.268+0000 7f1421f80640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14080b6700 0x7f14080b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559464f6f630 tx=0x7f141800a400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.268+0000 7f142b234740 1 -- 172.21.15.31:0/393650887 wait complete. 2024-09-21T06:14:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.268+0000 7f142b234740 1 -- start start 2024-09-21T06:14:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.268+0000 7f142b234740 1 Processor -- start 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142b234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 0x559464f7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142b234740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464baa4d0 0x559464f7c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142b234740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464bcb1b0 0x559464f88490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142b234740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559464e0cab0 con 0x559464baa4d0 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142b234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559464e7a730 con 0x559464ba22f0 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f1421f80640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 0x559464f7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f1421f80640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 0x559464f7c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36410/0 (socket says 172.21.15.31:36410) 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f1421f80640 1 -- 172.21.15.31:0/2231391773 learned_addr learned my addr 172.21.15.31:0/2231391773 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559464f7f170 con 0x559464bcb1b0 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f1422781640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464bcb1b0 0x559464f88490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.269+0000 7f142177f640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464baa4d0 0x559464f7c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f142b234740 10 client.0 _ll_register_callbacks cb 0x559464ece6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f1422781640 1 -- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 msgr2=0x559464f7c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f1422781640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 0x559464f7c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f1422781640 1 -- 172.21.15.31:0/2231391773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464baa4d0 msgr2=0x559464f7c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f1422781640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464baa4d0 0x559464f7c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f1422781640 1 -- 172.21.15.31:0/2231391773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559464c36790 con 0x559464bcb1b0 2024-09-21T06:14:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f1422781640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464bcb1b0 0x559464f88490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f141400b010 tx=0x7f1414004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c009a926780ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f14057fa640 10 client.25933 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.270+0000 7f14057fa640 1 -- 172.21.15.31:0/2231391773 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414040050 con 0x559464bcb1b0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.271+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1414003e20 con 0x559464bcb1b0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.271+0000 7f1421f80640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 0x559464f7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.271+0000 7f14057fa640 1 -- 172.21.15.31:0/2231391773 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1414003970 con 0x559464bcb1b0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.271+0000 7f14057fa640 1 -- 172.21.15.31:0/2231391773 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1414036040 con 0x559464bcb1b0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.271+0000 7f142b234740 10 client.25933 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.271+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x559464caf620 con 0x559464bcb1b0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f14057fa640 1 -- 172.21.15.31:0/2231391773 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1414004000 con 0x559464bcb1b0 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f142b234740 10 client.25933 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f142b234740 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f142b234740 20 client.25933 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f142b234740 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464b89d80 0x559464f8df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f142b234740 4 client.25933 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559464ee7640 con 0x559464b89d80 2024-09-21T06:14:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.272+0000 7f1421f80640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464b89d80 0x559464f8df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.273+0000 7f14057fa640 10 client.25933 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.273+0000 7f1421f80640 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464b89d80 0x559464f8df40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a19b1cd316bf784 server_cookie=46cb9b52a7576c90 in_seq=0 out_seq=0 2024-09-21T06:14:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.274+0000 7f14057fa640 1 -- 172.21.15.31:0/2231391773 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559464ee7640 con 0x559464b89d80 2024-09-21T06:14:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.274+0000 7f14057fa640 10 client.25933 handle_command_reply: tid=0 2024-09-21T06:14:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.274+0000 7f142b234740 1 client.25933 shutdown 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.274+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464bcb1b0 msgr2=0x559464f88490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.274+0000 7f142b234740 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464bcb1b0 0x559464f88490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f141400b010 tx=0x7f1414004a80 comp rx=0 tx=0).stop 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 shutdown_connections 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464b89d80 0x559464f8df40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464bcb1b0 0x559464f88490 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f14057fa640 0 client.25933 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464baa4d0 0x559464f7c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 --2- 172.21.15.31:0/2231391773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464ba22f0 0x559464f7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 >> 172.21.15.31:0/2231391773 conn(0x559464bbf300 msgr2=0x559464f719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 shutdown_connections 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.275+0000 7f142b234740 1 -- 172.21.15.31:0/2231391773 wait complete. 2024-09-21T06:14:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.276+0000 7f142b234740 20 client.25933 trim_cache size 0 max 16384 2024-09-21T06:14:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.305+0000 7f142b234740 1 Processor -- start 2024-09-21T06:14:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.306+0000 7f142b234740 1 -- start start 2024-09-21T06:14:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.306+0000 7f142b234740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464b89d80 0x559464f6d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.306+0000 7f142b234740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 0x559464f7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142b234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464baa4d0 0x559464f88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142177f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 0x559464f7dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142177f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 0x559464f7dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60886/0 (socket says 172.21.15.31:60886) 2024-09-21T06:14:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142177f640 1 -- 172.21.15.31:0/2725914610 learned_addr learned my addr 172.21.15.31:0/2725914610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1bc4e0 con 0x559464b89d80 2024-09-21T06:14:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1bed60 con 0x559464baa4d0 2024-09-21T06:14:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.307+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1bcb50 con 0x559464ba22f0 2024-09-21T06:14:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.310+0000 7f1421f80640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464b89d80 0x559464f6d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1421f80640 1 -- 172.21.15.31:0/2725914610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 msgr2=0x559464f7dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1421f80640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 0x559464f7dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1422781640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464baa4d0 0x559464f88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1421f80640 1 -- 172.21.15.31:0/2725914610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464baa4d0 msgr2=0x559464f88400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1421f80640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464baa4d0 0x559464f88400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1421f80640 1 -- 172.21.15.31:0/2725914610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559464f69d50 con 0x559464b89d80 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f142177f640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 0x559464f7dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:14:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.311+0000 7f1422781640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464baa4d0 0x559464f88400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.315+0000 7f1421f80640 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464b89d80 0x559464f6d860 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x559464ed4b80 tx=0x7f1418012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb50c7cf7b9cc467 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.316+0000 7f14077fe640 1 -- 172.21.15.31:0/2725914610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418016070 con 0x559464b89d80 2024-09-21T06:14:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.316+0000 7f14077fe640 1 -- 172.21.15.31:0/2725914610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141802b070 con 0x559464b89d80 2024-09-21T06:14:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.316+0000 7f14077fe640 1 -- 172.21.15.31:0/2725914610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418026650 con 0x559464b89d80 2024-09-21T06:14:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.317+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464b89d80 msgr2=0x559464f6d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.317+0000 7f142b234740 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464b89d80 0x559464f6d860 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x559464ed4b80 tx=0x7f1418012000 comp rx=0 tx=0).stop 2024-09-21T06:14:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.323+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 shutdown_connections 2024-09-21T06:14:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.324+0000 7f142b234740 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464baa4d0 0x559464f88400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.324+0000 7f142b234740 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464ba22f0 0x559464f7dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.324+0000 7f142b234740 1 --2- 172.21.15.31:0/2725914610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464b89d80 0x559464f6d860 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.324+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 >> 172.21.15.31:0/2725914610 conn(0x559464bbf300 msgr2=0x559464f7d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.326+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 shutdown_connections 2024-09-21T06:14:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.327+0000 7f142b234740 1 -- 172.21.15.31:0/2725914610 wait complete. 2024-09-21T06:14:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.327+0000 7f142b234740 1 -- start start 2024-09-21T06:14:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.328+0000 7f142b234740 1 Processor -- start 2024-09-21T06:14:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142b234740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464b89d80 0x559464f91930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142b234740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f8edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142b234740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f8f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142b234740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1bd220 con 0x559464ba22f0 2024-09-21T06:14:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142b234740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1be690 con 0x559464b89d80 2024-09-21T06:14:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f1422781640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f8f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f1422781640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f8f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60902/0 (socket says 172.21.15.31:60902) 2024-09-21T06:14:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f1422781640 1 -- 172.21.15.31:0/3407463490 learned_addr learned my addr 172.21.15.31:0/3407463490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f141c1bd8f0 con 0x559464baa4d0 2024-09-21T06:14:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.329+0000 7f142177f640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f8edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f142b234740 10 client.0 _ll_register_callbacks cb 0x7f140c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1421f80640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464b89d80 0x559464f91930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1422781640 1 -- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464b89d80 msgr2=0x559464f91930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1422781640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464b89d80 0x559464f91930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1422781640 1 -- 172.21.15.31:0/3407463490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 msgr2=0x559464f8edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1422781640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f8edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1422781640 1 -- 172.21.15.31:0/3407463490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559464ff76c0 con 0x559464baa4d0 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f1422781640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f8f500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1414009550 tx=0x7f1414009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21fc5e3bf76aa67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.330+0000 7f14057fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f14057fa640 1 -- 172.21.15.31:0/3407463490 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414060060 con 0x559464baa4d0 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141802b070 con 0x559464baa4d0 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f14057fa640 1 -- 172.21.15.31:0/3407463490 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1414055020 con 0x559464baa4d0 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f14057fa640 1 -- 172.21.15.31:0/3407463490 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1414040050 con 0x559464baa4d0 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f142177f640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f8edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f142b234740 10 client.25939 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.331+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559464ff8720 con 0x559464baa4d0 2024-09-21T06:14:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f14057fa640 1 -- 172.21.15.31:0/3407463490 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1414003e20 con 0x559464baa4d0 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f14057fa640 1 -- 172.21.15.31:0/3407463490 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1414065750 con 0x559464baa4d0 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f142b234740 10 client.25939 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f142b234740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f142b234740 20 client.25939 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f142b234740 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464bcb1b0 0x559464ffae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f142b234740 4 client.25939 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.332+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f141802b070 con 0x559464bcb1b0 2024-09-21T06:14:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.333+0000 7f1421f80640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464bcb1b0 0x559464ffae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.333+0000 7f1421f80640 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464bcb1b0 0x559464ffae90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e4b645f364d618 server_cookie=fdd93bca0a67be8f in_seq=0 out_seq=0 2024-09-21T06:14:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.333+0000 7f14057fa640 10 client.25939 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.364+0000 7f14057fa640 1 -- 172.21.15.31:0/3407463490 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f141802b070 con 0x559464bcb1b0 2024-09-21T06:14:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.364+0000 7f14057fa640 10 client.25939 handle_command_reply: tid=0 2024-09-21T06:14:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.364+0000 7f142b234740 1 client.25939 shutdown 2024-09-21T06:14:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 msgr2=0x559464f8f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f8f500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1414009550 tx=0x7f1414009330 comp rx=0 tx=0).stop 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 shutdown_connections 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559464bcb1b0 0x559464ffae90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559464baa4d0 0x559464f8f500 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559464ba22f0 0x559464f8edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 --2- 172.21.15.31:0/3407463490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559464b89d80 0x559464f91930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 >> 172.21.15.31:0/3407463490 conn(0x559464bbf300 msgr2=0x559464f6d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.365+0000 7f14057fa640 0 client.25939 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.366+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 shutdown_connections 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.366+0000 7f142b234740 1 -- 172.21.15.31:0/3407463490 wait complete. 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.366+0000 7f142b234740 20 client.25939 trim_cache size 0 max 16384 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:14:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:14:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14080b6700 msgr2=0x7f14080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14080b6700 0x7f14080b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x559464f6f630 tx=0x7f141800a400 comp rx=0 tx=0).stop 2024-09-21T06:14:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 msgr2=0x7f141c10b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 0x7f141c10b980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f140c00e9f0 tx=0x7f140c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 shutdown_connections 2024-09-21T06:14:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14080b6700 0x7f14080b8c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f141c108c30 0x7f141c10bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f141c1082e0 0x7f141c10b980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 --2- 172.21.15.31:0/2886104318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f141c1029b0 0x7f141c10b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.367+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 >> 172.21.15.31:0/2886104318 conn(0x7f141c07a740 msgr2=0x7f141c069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.368+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 shutdown_connections 2024-09-21T06:14:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:14:55.368+0000 7f14241e4640 1 -- 172.21.15.31:0/2886104318 wait complete. 2024-09-21T06:14:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.481+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fd0102ef0 con 0x7f7fd0102ae0 2024-09-21T06:14:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.482+0000 7f7fd489b640 1 -- 172.21.15.31:0/1497503782 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f7fc00b6b20 con 0x7f7fd0102ae0 2024-09-21T06:14:55.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:14:55.485 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:14:55.485 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:14:55.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.485+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fbc0b6880 msgr2=0x7f7fbc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.485+0000 7f7fd589d640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fbc0b6880 0x7f7fbc0b8d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7fb802aa10 tx=0x7f7fb8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 msgr2=0x7f7fd0199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:55.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0199440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d950 tx=0x7f7fc000de10 comp rx=0 tx=0).stop 2024-09-21T06:14:55.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 shutdown_connections 2024-09-21T06:14:55.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd0108d20 0x7f7fd019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fbc0b6880 0x7f7fbc0b8d90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd01083d0 0x7f7fd01999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 --2- 172.21.15.31:0/1497503782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0102ae0 0x7f7fd0199440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:55.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 >> 172.21.15.31:0/1497503782 conn(0x7f7fd00797b0 msgr2=0x7f7fd0106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:55.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.486+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 shutdown_connections 2024-09-21T06:14:55.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:55.487+0000 7f7fd589d640 1 -- 172.21.15.31:0/1497503782 wait complete. 2024-09-21T06:14:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:55 smithi031 ceph-mon[31237]: pgmap v419: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.7 MiB/s wr, 568 op/s 2024-09-21T06:14:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:55 smithi110 ceph-mon[34550]: pgmap v419: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.7 MiB/s wr, 568 op/s 2024-09-21T06:14:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:55 smithi107 ceph-mon[34380]: pgmap v419: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.7 MiB/s wr, 568 op/s 2024-09-21T06:14:56.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:14:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:56 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1497503782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:56.758 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:56 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1497503782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:56 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1497503782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:14:57.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.360+0000 7f06f62b3640 1 -- 172.21.15.31:0/1575264198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 msgr2=0x7f06f010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.360+0000 7f06f62b3640 1 --2- 172.21.15.31:0/1575264198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f010bae0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009860 tx=0x7f06d802a340 comp rx=0 tx=0).stop 2024-09-21T06:14:57.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 -- 172.21.15.31:0/1575264198 shutdown_connections 2024-09-21T06:14:57.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 --2- 172.21.15.31:0/1575264198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06f010c020 0x7f06f0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 --2- 172.21.15.31:0/1575264198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f010bae0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 --2- 172.21.15.31:0/1575264198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06f0105d90 0x7f06f01061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 -- 172.21.15.31:0/1575264198 >> 172.21.15.31:0/1575264198 conn(0x7f06f01017e0 msgr2=0x7f06f0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:57.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 -- 172.21.15.31:0/1575264198 shutdown_connections 2024-09-21T06:14:57.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 -- 172.21.15.31:0/1575264198 wait complete. 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.361+0000 7f06f62b3640 1 Processor -- start 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 -- start start 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06f0105d90 0x7f06f019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06f010c020 0x7f06f01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0115d30 con 0x7f06f010c020 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0115bb0 con 0x7f06f010b6d0 2024-09-21T06:14:57.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.362+0000 7f06f62b3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0115eb0 con 0x7f06f0105d90 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.363+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.363+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36460/0 (socket says 172.21.15.31:36460) 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.363+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2874318140 learned_addr learned my addr 172.21.15.31:0/2874318140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.363+0000 7f06f4850640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06f010c020 0x7f06f01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.364+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2874318140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06f0105d90 msgr2=0x7f06f019c760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.364+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06f0105d90 0x7f06f019c760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.364+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06f010c020 msgr2=0x7f06f01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.364+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06f010c020 0x7f06f01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.364+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2874318140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f01a26d0 con 0x7f06f010b6d0 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.364+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f019cd00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06d802a8b0 tx=0x7f06d802ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.365+0000 7f06ed7fa640 1 -- 172.21.15.31:0/2874318140 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8037040 con 0x7f06f010b6d0 2024-09-21T06:14:57.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.365+0000 7f06ed7fa640 1 -- 172.21.15.31:0/2874318140 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d8004050 con 0x7f06f010b6d0 2024-09-21T06:14:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.365+0000 7f06ed7fa640 1 -- 172.21.15.31:0/2874318140 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8030850 con 0x7f06f010b6d0 2024-09-21T06:14:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.365+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f01a2960 con 0x7f06f010b6d0 2024-09-21T06:14:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.365+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f0077800 con 0x7f06f010b6d0 2024-09-21T06:14:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 Processor -- start 2024-09-21T06:14:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 -- start start 2024-09-21T06:14:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d953070 0x55c53dcfa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dcfabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dcfb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c53dc70ea0 con 0x55c53d932390 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06ed7fa640 1 -- 172.21.15.31:0/2874318140 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f06d8041a70 con 0x7f06f010b6d0 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c53dc602d0 con 0x55c53d92a1b0 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.369+0000 7f06fc827740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c53dc640e0 con 0x55c53d953070 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dcfb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dcfb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38292/0 (socket says 172.21.15.31:38292) 2024-09-21T06:14:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2019771508 learned_addr learned my addr 172.21.15.31:0/2019771508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2019771508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d953070 msgr2=0x55c53dcfa6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:14:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d953070 0x55c53dcfa6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2019771508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 msgr2=0x55c53dcfabe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dcfabe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.370+0000 7f06ef7fe640 1 -- 172.21.15.31:0/2019771508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53d9be650 con 0x55c53d932390 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.371+0000 7f06ed7fa640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f06dc0b6800 0x7f06dc0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.371+0000 7f06f4850640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f06dc0b6800 0x7f06dc0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.371+0000 7f06ed7fa640 1 -- 172.21.15.31:0/2874318140 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f06d810f7e0 con 0x7f06f010b6d0 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.371+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dcfb120 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f06d810cee0 tx=0x7f06d8115b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef008a09dccecc2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.371+0000 7f06e67fc640 1 -- 172.21.15.31:0/2019771508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80d1d40 con 0x55c53d932390 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.372+0000 7f06e67fc640 1 -- 172.21.15.31:0/2019771508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d80d5200 con 0x55c53d932390 2024-09-21T06:14:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.372+0000 7f06e67fc640 1 -- 172.21.15.31:0/2019771508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80d4e60 con 0x55c53d932390 2024-09-21T06:14:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.373+0000 7f06fc827740 1 -- 172.21.15.31:0/2019771508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 msgr2=0x55c53dcfb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.373+0000 7f06fc827740 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dcfb120 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f06d810cee0 tx=0x7f06d8115b40 comp rx=0 tx=0).stop 2024-09-21T06:14:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.373+0000 7f06f4850640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f06dc0b6800 0x7f06dc0b8d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f06e00096c0 tx=0x7f06e0009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.373+0000 7f06fc827740 1 -- 172.21.15.31:0/2019771508 shutdown_connections 2024-09-21T06:14:57.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.374+0000 7f06fc827740 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dcfb120 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.374+0000 7f06fc827740 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dcfabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.374+0000 7f06fc827740 1 --2- 172.21.15.31:0/2019771508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d953070 0x55c53dcfa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.374+0000 7f06fc827740 1 -- 172.21.15.31:0/2019771508 >> 172.21.15.31:0/2019771508 conn(0x55c53d947460 msgr2=0x55c53dcf8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.374+0000 7f06fc827740 1 -- 172.21.15.31:0/2019771508 shutdown_connections 2024-09-21T06:14:57.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.374+0000 7f06fc827740 1 -- 172.21.15.31:0/2019771508 wait complete. 2024-09-21T06:14:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.375+0000 7f06fc827740 1 -- start start 2024-09-21T06:14:57.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.375+0000 7f06fc827740 1 Processor -- start 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d92a1b0 0x55c53dd05d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dd06280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd11ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c53dac1700 con 0x55c53d92a1b0 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c53dbfb160 con 0x55c53d953070 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c53dd08c40 con 0x55c53d932390 2024-09-21T06:14:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06fc827740 10 client.0 _ll_register_callbacks cb 0x55c53dc149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06effff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d92a1b0 0x55c53dd05d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd11ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06effff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d92a1b0 0x55c53dd05d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38306/0 (socket says 172.21.15.31:38306) 2024-09-21T06:14:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd11ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36484/0 (socket says 172.21.15.31:36484) 2024-09-21T06:14:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06effff640 1 -- 172.21.15.31:0/2758789614 learned_addr learned my addr 172.21.15.31:0/2758789614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:57.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dd06280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 -- 172.21.15.31:0/2758789614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 msgr2=0x55c53dd06280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dd06280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 -- 172.21.15.31:0/2758789614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d92a1b0 msgr2=0x55c53dd05d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d92a1b0 0x55c53dd05d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.376+0000 7f06f4850640 1 -- 172.21.15.31:0/2758789614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53dc28c90 con 0x55c53d953070 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.377+0000 7f06f4850640 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd11ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89ea8b68e35f969 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:57.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.377+0000 7f06cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:14:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.378+0000 7f06cf7fe640 1 -- 172.21.15.31:0/2758789614 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e001f8f0 con 0x55c53d953070 2024-09-21T06:14:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.378+0000 7f06cf7fe640 1 -- 172.21.15.31:0/2758789614 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f06e001fa90 con 0x55c53d953070 2024-09-21T06:14:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.378+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d80d5200 con 0x55c53d953070 2024-09-21T06:14:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.378+0000 7f06cf7fe640 1 -- 172.21.15.31:0/2758789614 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f06e001fc90 con 0x55c53d953070 2024-09-21T06:14:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.378+0000 7f06fc827740 10 client.25913 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:57.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.378+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c53dd14900 con 0x55c53d953070 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06cf7fe640 1 -- 172.21.15.31:0/2758789614 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e001fe30 con 0x55c53d953070 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06cf7fe640 1 -- 172.21.15.31:0/2758789614 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f06e000f6c0 con 0x55c53d953070 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06fc827740 10 client.25913 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06fc827740 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06fc827740 20 client.25913 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06fc827740 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d9119f0 0x55c53dd17d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06fc827740 4 client.25913 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:14:57.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.379+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f06d80d5200 con 0x55c53d9119f0 2024-09-21T06:14:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.380+0000 7f06effff640 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d9119f0 0x55c53dd17d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.381+0000 7f06cf7fe640 10 client.25913 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:57.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.381+0000 7f06effff640 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d9119f0 0x55c53dd17d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5efcd3e6d585ddce server_cookie=62d2a3885a938aec in_seq=0 out_seq=0 2024-09-21T06:14:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.382+0000 7f06cf7fe640 1 -- 172.21.15.31:0/2758789614 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f06d80d5200 con 0x55c53d9119f0 2024-09-21T06:14:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.382+0000 7f06cf7fe640 10 client.25913 handle_command_reply: tid=0 2024-09-21T06:14:57.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.383+0000 7f06fc827740 1 client.25913 shutdown 2024-09-21T06:14:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.383+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 msgr2=0x55c53dd11ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.383+0000 7f06fc827740 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd11ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e0004730 comp rx=0 tx=0).stop 2024-09-21T06:14:57.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 shutdown_connections 2024-09-21T06:14:57.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d9119f0 0x55c53dd17d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd11ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dd06280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 --2- 172.21.15.31:0/2758789614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d92a1b0 0x55c53dd05d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 >> 172.21.15.31:0/2758789614 conn(0x55c53d947460 msgr2=0x55c53dcfd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:57.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06cf7fe640 0 client.25913 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:57.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 shutdown_connections 2024-09-21T06:14:57.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 1 -- 172.21.15.31:0/2758789614 wait complete. 2024-09-21T06:14:57.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.384+0000 7f06fc827740 20 client.25913 trim_cache size 0 max 16384 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.407+0000 7f06fc827740 1 Processor -- start 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.407+0000 7f06fc827740 1 -- start start 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06fc827740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dd06200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06fc827740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dcfd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06fc827740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d953070 0x55c53dcfdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06fc827740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ab650 con 0x55c53d953070 2024-09-21T06:14:57.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06fc827740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aded0 con 0x55c53d92a1b0 2024-09-21T06:14:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06fc827740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01abcc0 con 0x55c53d932390 2024-09-21T06:14:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dd06200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dd06200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:36490/0 (socket says 172.21.15.31:36490) 2024-09-21T06:14:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.408+0000 7f06effff640 1 -- 172.21.15.31:0/1858741790 learned_addr learned my addr 172.21.15.31:0/1858741790 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:57.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dcfd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06f4850640 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d953070 0x55c53dcfdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06effff640 1 -- 172.21.15.31:0/1858741790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 msgr2=0x55c53dcfd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06effff640 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dcfd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06effff640 1 -- 172.21.15.31:0/1858741790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d953070 msgr2=0x55c53dcfdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06effff640 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d953070 0x55c53dcfdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06effff640 1 -- 172.21.15.31:0/1858741790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53dcfd500 con 0x55c53d92a1b0 2024-09-21T06:14:57.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.409+0000 7f06effff640 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dd06200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009fd0 tx=0x7f06e8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2738ea20f6433c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:57.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.410+0000 7f06e5ffb640 1 -- 172.21.15.31:0/1858741790 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8005860 con 0x55c53d92a1b0 2024-09-21T06:14:57.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.410+0000 7f06e5ffb640 1 -- 172.21.15.31:0/1858741790 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e8003e20 con 0x55c53d92a1b0 2024-09-21T06:14:57.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.410+0000 7f06e5ffb640 1 -- 172.21.15.31:0/1858741790 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e801a450 con 0x55c53d92a1b0 2024-09-21T06:14:57.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 -- 172.21.15.31:0/1858741790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 msgr2=0x55c53dd06200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dd06200 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009fd0 tx=0x7f06e8009500 comp rx=0 tx=0).stop 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 -- 172.21.15.31:0/1858741790 shutdown_connections 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d953070 0x55c53dcfdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d932390 0x55c53dcfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 --2- 172.21.15.31:0/1858741790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d92a1b0 0x55c53dd06200 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 -- 172.21.15.31:0/1858741790 >> 172.21.15.31:0/1858741790 conn(0x55c53d947460 msgr2=0x55c53dd05af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 -- 172.21.15.31:0/1858741790 shutdown_connections 2024-09-21T06:14:57.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 -- 172.21.15.31:0/1858741790 wait complete. 2024-09-21T06:14:57.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.412+0000 7f06fc827740 1 -- start start 2024-09-21T06:14:57.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 Processor -- start 2024-09-21T06:14:57.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d92a1b0 0x55c53dd09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dd0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ac390 con 0x55c53d932390 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ad800 con 0x55c53d953070 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aca60 con 0x55c53d92a1b0 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.413+0000 7f06fc827740 10 client.0 _ll_register_callbacks cb 0x7f06e0009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dd0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dd0a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38330/0 (socket says 172.21.15.31:38330) 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 -- 172.21.15.31:0/1286583545 learned_addr learned my addr 172.21.15.31:0/1286583545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06effff640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d92a1b0 0x55c53dd09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06f4850640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 -- 172.21.15.31:0/1286583545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d92a1b0 msgr2=0x55c53dd09d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:57.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d92a1b0 0x55c53dd09d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 -- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 msgr2=0x55c53dd0a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:14:57.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd0a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.414+0000 7f06ef7fe640 1 -- 172.21.15.31:0/1286583545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53dd926d0 con 0x55c53d932390 2024-09-21T06:14:57.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.417+0000 7f06ef7fe640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dd0a2a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f06d80d2960 tx=0x7f06d8030e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac8930b813d26ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:14:57.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.417+0000 7f06cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:14:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.417+0000 7f06cf7fe640 1 -- 172.21.15.31:0/1286583545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80d5040 con 0x55c53d932390 2024-09-21T06:14:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.417+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e8003e20 con 0x55c53d932390 2024-09-21T06:14:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.417+0000 7f06cf7fe640 1 -- 172.21.15.31:0/1286583545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f06d8004200 con 0x55c53d932390 2024-09-21T06:14:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06cf7fe640 1 -- 172.21.15.31:0/1286583545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f06d80d5040 con 0x55c53d932390 2024-09-21T06:14:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 10 client.16191 fetch_fsmap learned FSMap version 19 2024-09-21T06:14:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c53dd93920 con 0x55c53d932390 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06cf7fe640 1 -- 172.21.15.31:0/1286583545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e8003e20 con 0x55c53d932390 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06cf7fe640 1 -- 172.21.15.31:0/1286583545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f06d80d1960 con 0x55c53d932390 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 10 client.16191 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 20 client.16191 populate_metadata read hostname 'smithi031' 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d8e6da0 0x55c53dd960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 4 client.16191 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.418+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c53dc72df0 con 0x55c53d8e6da0 2024-09-21T06:14:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.419+0000 7f06effff640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d8e6da0 0x55c53dd960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06cf7fe640 10 client.16191 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06effff640 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d8e6da0 0x55c53dd960b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87e46b9f2b570cb server_cookie=ed1af79fd6863999 in_seq=0 out_seq=0 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06cf7fe640 1 -- 172.21.15.31:0/1286583545 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c53dc72df0 con 0x55c53d8e6da0 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06cf7fe640 10 client.16191 handle_command_reply: tid=0 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 client.16191 shutdown 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 msgr2=0x55c53dd0a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dd0a2a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f06d80d2960 tx=0x7f06d8030e90 comp rx=0 tx=0).stop 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 shutdown_connections 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c53d8e6da0 0x55c53dd960b0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c53d953070 0x55c53dd0a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c53d932390 0x55c53dd0a2a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 --2- 172.21.15.31:0/1286583545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c53d92a1b0 0x55c53dd09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 >> 172.21.15.31:0/1286583545 conn(0x55c53d947460 msgr2=0x55c53dcfd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.420+0000 7f06cf7fe640 0 client.16191 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:14:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 shutdown_connections 2024-09-21T06:14:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06fc827740 1 -- 172.21.15.31:0/1286583545 wait complete. 2024-09-21T06:14:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06fc827740 20 client.16191 trim_cache size 0 max 16384 2024-09-21T06:14:57.424 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f06dc0b6800 msgr2=0x7f06dc0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06f62b3640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f06dc0b6800 0x7f06dc0b8d10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f06e00096c0 tx=0x7f06e0009260 comp rx=0 tx=0).stop 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 msgr2=0x7f06f019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.421+0000 7f06f62b3640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f019cd00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f06d802a8b0 tx=0x7f06d802ada0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 shutdown_connections 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f06dc0b6800 0x7f06dc0b8d10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06f010c020 0x7f06f01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f010b6d0 0x7f06f019cd00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 --2- 172.21.15.31:0/2874318140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f06f0105d90 0x7f06f019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:14:57.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 >> 172.21.15.31:0/2874318140 conn(0x7f06f01017e0 msgr2=0x7f06f0103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:14:57.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 shutdown_connections 2024-09-21T06:14:57.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:14:57.422+0000 7f06f62b3640 1 -- 172.21.15.31:0/2874318140 wait complete. 2024-09-21T06:14:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:57 smithi031 ceph-mon[31237]: pgmap v420: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-09-21T06:14:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:57 smithi110 ceph-mon[34550]: pgmap v420: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-09-21T06:14:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:57 smithi107 ceph-mon[34380]: pgmap v420: 97 pgs: 97 active+clean; 2.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 11 MiB/s wr, 886 op/s 2024-09-21T06:14:59.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:14:59.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:14:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:14:59 smithi031 ceph-mon[31237]: pgmap v421: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 10 MiB/s wr, 828 op/s 2024-09-21T06:14:59.649 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:14:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:14:59 smithi110 ceph-mon[34550]: pgmap v421: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 10 MiB/s wr, 828 op/s 2024-09-21T06:14:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:14:59 smithi107 ceph-mon[34380]: pgmap v421: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 10 MiB/s wr, 828 op/s 2024-09-21T06:15:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 200.0 s, 3619.8 tps, lat 27.627 ms stddev 30.967 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/3123934679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4075cf0 msgr2=0x7fb2a4076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/3123934679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4075cf0 0x7fb2a4076100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb298009860 tx=0x7fb29802a340 comp rx=0 tx=0).stop 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/3123934679 shutdown_connections 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/3123934679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4114430 0x7fb2a4116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/3123934679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4075cf0 0x7fb2a4076100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/3123934679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 0x7fb2a40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.369+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/3123934679 >> 172.21.15.31:0/3123934679 conn(0x7fb2a407b910 msgr2=0x7fb2a407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.370+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/3123934679 shutdown_connections 2024-09-21T06:15:00.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.370+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/3123934679 wait complete. 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.371+0000 7fb2ab1fa640 1 Processor -- start 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 -- start start 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4075cf0 0x7fb2a410fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 0x7fb2a4108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4114430 0x7fb2a4109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a411a060 con 0x7fb2a4114430 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a4119ee0 con 0x7fb2a4075cf0 2024-09-21T06:15:00.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.372+0000 7fb2ab1fa640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a411a1e0 con 0x7fb2a4077330 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 0x7fb2a4108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 0x7fb2a4108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50170/0 (socket says 172.21.15.31:50170) 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a3fff640 1 -- 172.21.15.31:0/1234306678 learned_addr learned my addr 172.21.15.31:0/1234306678 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a9797640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4114430 0x7fb2a4109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a8f96640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4075cf0 0x7fb2a410fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a8f96640 1 -- 172.21.15.31:0/1234306678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 msgr2=0x7fb2a4108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a8f96640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 0x7fb2a4108f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a8f96640 1 -- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4114430 msgr2=0x7fb2a4109550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a8f96640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4114430 0x7fb2a4109550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.373+0000 7fb2a8f96640 1 -- 172.21.15.31:0/1234306678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a4109d70 con 0x7fb2a4075cf0 2024-09-21T06:15:00.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.374+0000 7fb2a8f96640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4075cf0 0x7fb2a410fc80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb29400e790 tx=0x7fb29400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.374+0000 7fb2a1ffb640 1 -- 172.21.15.31:0/1234306678 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29400eea0 con 0x7fb2a4075cf0 2024-09-21T06:15:00.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.374+0000 7fb2a1ffb640 1 -- 172.21.15.31:0/1234306678 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb29400ccc0 con 0x7fb2a4075cf0 2024-09-21T06:15:00.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.375+0000 7fb2a1ffb640 1 -- 172.21.15.31:0/1234306678 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2940187a0 con 0x7fb2a4075cf0 2024-09-21T06:15:00.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.375+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a410ea10 con 0x7fb2a4075cf0 2024-09-21T06:15:00.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.376+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a410ee10 con 0x7fb2a4075cf0 2024-09-21T06:15:00.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.381+0000 7fb2a1ffb640 1 -- 172.21.15.31:0/1234306678 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb294018940 con 0x7fb2a4075cf0 2024-09-21T06:15:00.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.381+0000 7fb2b176e740 1 Processor -- start 2024-09-21T06:15:00.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.381+0000 7fb2b176e740 1 -- start start 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 0x5573c4ef5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2a1ffb640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb28c0b66b0 0x7fb28c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 0x5573c4ef5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4ef5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2b176e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5573c4e6be60 con 0x5573c4b25490 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2b176e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5573c4e5b290 con 0x5573c4b2d670 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2b176e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5573c4e5f0a0 con 0x5573c4b4e350 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2a1ffb640 1 -- 172.21.15.31:0/1234306678 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb2940e7980 con 0x7fb2a4075cf0 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 0x5573c4ef5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 0x5573c4ef5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54340/0 (socket says 172.21.15.31:54340) 2024-09-21T06:15:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.382+0000 7fb2a3fff640 1 -- 172.21.15.31:0/3720658078 learned_addr learned my addr 172.21.15.31:0/3720658078 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:00.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.383+0000 7fb2a3fff640 1 -- 172.21.15.31:0/3720658078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 msgr2=0x5573c4ef5430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:15:00.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.383+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 0x5573c4ef5430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.383+0000 7fb2a3fff640 1 -- 172.21.15.31:0/3720658078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 msgr2=0x5573c4ef5eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:00.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.383+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4ef5eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.383+0000 7fb2a3fff640 1 -- 172.21.15.31:0/3720658078 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c4bb9930 con 0x5573c4b25490 2024-09-21T06:15:00.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.383+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 0x5573c4ef5970 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb29802a860 tx=0x7fb298005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b9d7d9c92eb81f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.384+0000 7fb2a8f96640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb28c0b66b0 0x7fb28c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.384+0000 7fb28affd640 1 -- 172.21.15.31:0/3720658078 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298041020 con 0x5573c4b25490 2024-09-21T06:15:00.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.384+0000 7fb28affd640 1 -- 172.21.15.31:0/3720658078 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298030450 con 0x5573c4b25490 2024-09-21T06:15:00.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.384+0000 7fb28affd640 1 -- 172.21.15.31:0/3720658078 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298037040 con 0x5573c4b25490 2024-09-21T06:15:00.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.386+0000 7fb2b176e740 1 -- 172.21.15.31:0/3720658078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 msgr2=0x5573c4ef5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.386+0000 7fb2b176e740 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 0x5573c4ef5970 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb29802a860 tx=0x7fb298005e00 comp rx=0 tx=0).stop 2024-09-21T06:15:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.387+0000 7fb2b176e740 1 -- 172.21.15.31:0/3720658078 shutdown_connections 2024-09-21T06:15:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.387+0000 7fb2a8f96640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb28c0b66b0 0x7fb28c0b8bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb294009510 tx=0x7fb294009480 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.387+0000 7fb2b176e740 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4ef5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.387+0000 7fb2b176e740 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b25490 0x5573c4ef5970 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.387+0000 7fb2b176e740 1 --2- 172.21.15.31:0/3720658078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 0x5573c4ef5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.387+0000 7fb2b176e740 1 -- 172.21.15.31:0/3720658078 >> 172.21.15.31:0/3720658078 conn(0x5573c4b42740 msgr2=0x5573c4ef3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.388+0000 7fb2b176e740 1 -- 172.21.15.31:0/3720658078 shutdown_connections 2024-09-21T06:15:00.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.388+0000 7fb2b176e740 1 -- 172.21.15.31:0/3720658078 wait complete. 2024-09-21T06:15:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.389+0000 7fb2b176e740 1 -- start start 2024-09-21T06:15:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.389+0000 7fb2b176e740 1 Processor -- start 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4f00fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f01510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b4e350 0x5573c4f0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5573c4cbc890 con 0x5573c4b2d670 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5573c4df5d40 con 0x5573c4b4e350 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5573c4f03bb0 con 0x5573c4b25490 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2b176e740 10 client.0 _ll_register_callbacks cb 0x5573c4e59c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2a9797640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4f00fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2a9797640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4f00fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50192/0 (socket says 172.21.15.31:50192) 2024-09-21T06:15:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2a9797640 1 -- 172.21.15.31:0/1535575181 learned_addr learned my addr 172.21.15.31:0/1535575181 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f01510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.390+0000 7fb2a9797640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b4e350 0x5573c4f0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a9797640 1 -- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b4e350 msgr2=0x5573c4f0cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a9797640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b4e350 0x5573c4f0cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a9797640 1 -- 172.21.15.31:0/1535575181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 msgr2=0x5573c4f01510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a9797640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f01510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a9797640 1 -- 172.21.15.31:0/1535575181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c4f0d460 con 0x5573c4b25490 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f01510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb2a9797640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4f00fd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb29c013f60 tx=0x7fb29c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a16163b37c4f73d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.391+0000 7fb283fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.392+0000 7fb283fff640 1 -- 172.21.15.31:0/1535575181 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c013130 con 0x5573c4b25490 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.392+0000 7fb283fff640 1 -- 172.21.15.31:0/1535575181 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb29c003b80 con 0x5573c4b25490 2024-09-21T06:15:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.392+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb298030450 con 0x5573c4b25490 2024-09-21T06:15:00.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.392+0000 7fb283fff640 1 -- 172.21.15.31:0/1535575181 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb29c0132d0 con 0x5573c4b25490 2024-09-21T06:15:00.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.392+0000 7fb2b176e740 10 client.25960 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:00.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.392+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5573c4f0fd90 con 0x5573c4b25490 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb283fff640 1 -- 172.21.15.31:0/1535575181 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb29c005020 con 0x5573c4b25490 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb283fff640 1 -- 172.21.15.31:0/1535575181 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb29c010430 con 0x5573c4b25490 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2b176e740 10 client.25960 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2b176e740 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2b176e740 20 client.25960 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4b0ccd0 0x5573c4f13290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2b176e740 4 client.25960 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb298030450 con 0x5573c4b0ccd0 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.393+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4b0ccd0 0x5573c4f13290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.394+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4b0ccd0 0x5573c4f13290 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ece20342210f066 server_cookie=64b613c048bea50c in_seq=0 out_seq=0 2024-09-21T06:15:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.394+0000 7fb283fff640 10 client.25960 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:00.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.394+0000 7fb283fff640 1 -- 172.21.15.31:0/1535575181 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb298030450 con 0x5573c4b0ccd0 2024-09-21T06:15:00.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.394+0000 7fb283fff640 10 client.25960 handle_command_reply: tid=0 2024-09-21T06:15:00.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.394+0000 7fb2b176e740 1 client.25960 shutdown 2024-09-21T06:15:00.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 msgr2=0x5573c4f00fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4f00fd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb29c013f60 tx=0x7fb29c009500 comp rx=0 tx=0).stop 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 shutdown_connections 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4b0ccd0 0x5573c4f13290 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b4e350 0x5573c4f0cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f01510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1535575181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4f00fd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb283fff640 0 client.25960 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 >> 172.21.15.31:0/1535575181 conn(0x5573c4b42740 msgr2=0x5573c4ef43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 shutdown_connections 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.395+0000 7fb2b176e740 1 -- 172.21.15.31:0/1535575181 wait complete. 2024-09-21T06:15:00.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.396+0000 7fb2b176e740 20 client.25960 trim_cache size 0 max 16384 2024-09-21T06:15:00.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.430+0000 7fb2b176e740 1 Processor -- start 2024-09-21T06:15:00.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 -- start start 2024-09-21T06:15:00.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4eff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4effbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b4e350 0x5573c4efbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41bc210 con 0x5573c4b4e350 2024-09-21T06:15:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41bea90 con 0x5573c4b2d670 2024-09-21T06:15:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.431+0000 7fb2b176e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41bc880 con 0x5573c4b25490 2024-09-21T06:15:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.432+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4effbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.432+0000 7fb2a9797640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b4e350 0x5573c4efbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.432+0000 7fb2a9797640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b4e350 0x5573c4efbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54354/0 (socket says 172.21.15.31:54354) 2024-09-21T06:15:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.432+0000 7fb2a9797640 1 -- 172.21.15.31:0/1667236239 learned_addr learned my addr 172.21.15.31:0/1667236239 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.435+0000 7fb2a9797640 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4eff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.437+0000 7fb2a3fff640 1 -- 172.21.15.31:0/1667236239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 msgr2=0x5573c4eff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.437+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4eff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.437+0000 7fb2a3fff640 1 -- 172.21.15.31:0/1667236239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b4e350 msgr2=0x5573c4efbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.437+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b4e350 0x5573c4efbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.437+0000 7fb2a3fff640 1 -- 172.21.15.31:0/1667236239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c4eed520 con 0x5573c4b2d670 2024-09-21T06:15:00.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.438+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4effbb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb298009830 tx=0x7fb298005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8679a78d2094f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.440+0000 7fb28a7fc640 1 -- 172.21.15.31:0/1667236239 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298053020 con 0x5573c4b2d670 2024-09-21T06:15:00.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.440+0000 7fb28a7fc640 1 -- 172.21.15.31:0/1667236239 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298030450 con 0x5573c4b2d670 2024-09-21T06:15:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.442+0000 7fb2b176e740 1 -- 172.21.15.31:0/1667236239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 msgr2=0x5573c4effbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.442+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4effbb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb298009830 tx=0x7fb298005e70 comp rx=0 tx=0).stop 2024-09-21T06:15:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.445+0000 7fb2b176e740 1 -- 172.21.15.31:0/1667236239 shutdown_connections 2024-09-21T06:15:00.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.445+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b4e350 0x5573c4efbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.445+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b2d670 0x5573c4effbb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.445+0000 7fb2b176e740 1 --2- 172.21.15.31:0/1667236239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b25490 0x5573c4eff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.445+0000 7fb2b176e740 1 -- 172.21.15.31:0/1667236239 >> 172.21.15.31:0/1667236239 conn(0x5573c4b42740 msgr2=0x5573c4ef1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.448+0000 7fb2b176e740 1 -- 172.21.15.31:0/1667236239 shutdown_connections 2024-09-21T06:15:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.449+0000 7fb2b176e740 1 -- 172.21.15.31:0/1667236239 wait complete. 2024-09-21T06:15:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.450+0000 7fb2b176e740 1 -- start start 2024-09-21T06:15:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.450+0000 7fb2b176e740 1 Processor -- start 2024-09-21T06:15:00.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b25490 0x5573c4f09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 0x5573c4f03e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41bcf50 con 0x5573c4b2d670 2024-09-21T06:15:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41be3c0 con 0x5573c4b25490 2024-09-21T06:15:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a41bd620 con 0x5573c4b4e350 2024-09-21T06:15:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.453+0000 7fb2b176e740 10 client.0 _ll_register_callbacks cb 0x7fb29c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.454+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.454+0000 7fb2a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54356/0 (socket says 172.21.15.31:54356) 2024-09-21T06:15:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.454+0000 7fb2a3fff640 1 -- 172.21.15.31:0/2696365808 learned_addr learned my addr 172.21.15.31:0/2696365808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb2a3fff640 1 -- 172.21.15.31:0/2696365808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 msgr2=0x5573c4f03e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 0x5573c4f03e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb2a3fff640 1 -- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b25490 msgr2=0x5573c4f09090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b25490 0x5573c4f09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb2a3fff640 1 -- 172.21.15.31:0/2696365808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c4f03af0 con 0x5573c4b2d670 2024-09-21T06:15:00.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb2a3fff640 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f095d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb298005720 tx=0x7fb298005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b185b66abdbf80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:00.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.455+0000 7fb283fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.456+0000 7fb283fff640 1 -- 172.21.15.31:0/2696365808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29805d020 con 0x5573c4b2d670 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.456+0000 7fb283fff640 1 -- 172.21.15.31:0/2696365808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb2980046c0 con 0x5573c4b2d670 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.456+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb298030450 con 0x5573c4b2d670 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.456+0000 7fb283fff640 1 -- 172.21.15.31:0/2696365808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb298037040 con 0x5573c4b2d670 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.456+0000 7fb2b176e740 10 client.16209 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.456+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5573c4f8f2e0 con 0x5573c4b2d670 2024-09-21T06:15:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb283fff640 1 -- 172.21.15.31:0/2696365808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb298030450 con 0x5573c4b2d670 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb283fff640 1 -- 172.21.15.31:0/2696365808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb29804f9b0 con 0x5573c4b2d670 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb2b176e740 10 client.16209 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb2b176e740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb2b176e740 20 client.16209 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb2b176e740 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4ae2080 0x5573c4f91640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb2b176e740 4 client.16209 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:15:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.457+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573c4e6ddb0 con 0x5573c4ae2080 2024-09-21T06:15:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.464+0000 7fb2a9797640 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4ae2080 0x5573c4f91640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:00.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.474+0000 7fb283fff640 10 client.16209 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:00.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.474+0000 7fb2a9797640 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4ae2080 0x5573c4f91640 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62af0fb29ba48e91 server_cookie=fc000ffce039bcf in_seq=0 out_seq=0 2024-09-21T06:15:00.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.477+0000 7fb283fff640 1 -- 172.21.15.31:0/2696365808 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573c4e6ddb0 con 0x5573c4ae2080 2024-09-21T06:15:00.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.477+0000 7fb283fff640 10 client.16209 handle_command_reply: tid=0 2024-09-21T06:15:00.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.477+0000 7fb2b176e740 1 client.16209 shutdown 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.477+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 msgr2=0x5573c4f095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.477+0000 7fb2b176e740 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f095d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb298005720 tx=0x7fb298005750 comp rx=0 tx=0).stop 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 shutdown_connections 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5573c4ae2080 0x5573c4f91640 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573c4b4e350 0x5573c4f03e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5573c4b2d670 0x5573c4f095d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 --2- 172.21.15.31:0/2696365808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573c4b25490 0x5573c4f09090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 >> 172.21.15.31:0/2696365808 conn(0x5573c4b42740 msgr2=0x5573c4eff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb283fff640 0 client.16209 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.480+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 shutdown_connections 2024-09-21T06:15:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.481+0000 7fb2b176e740 1 -- 172.21.15.31:0/2696365808 wait complete. 2024-09-21T06:15:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.482+0000 7fb2b176e740 20 client.16209 trim_cache size 0 max 16384 2024-09-21T06:15:00.484 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:15:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "8716d24c-4ac0-4c19-bb0a-5288b818dbd0", 2024-09-21T06:15:00.484 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:15:00.484 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb28c0b66b0 msgr2=0x7fb28c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb28c0b66b0 0x7fb28c0b8bc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb294009510 tx=0x7fb294009480 comp rx=0 tx=0).stop 2024-09-21T06:15:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4075cf0 msgr2=0x7fb2a410fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4075cf0 0x7fb2a410fc80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb29400e790 tx=0x7fb29400ec50 comp rx=0 tx=0).stop 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 shutdown_connections 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb28c0b66b0 0x7fb28c0b8bc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4114430 0x7fb2a4109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.483+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2a4077330 0x7fb2a4108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.484+0000 7fb2ab1fa640 1 --2- 172.21.15.31:0/1234306678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2a4075cf0 0x7fb2a410fc80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.484+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 >> 172.21.15.31:0/1234306678 conn(0x7fb2a407b910 msgr2=0x7fb2a41131e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:00.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.484+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 shutdown_connections 2024-09-21T06:15:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:00.484+0000 7fb2ab1fa640 1 -- 172.21.15.31:0/1234306678 wait complete. 2024-09-21T06:15:01.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:15:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:01 smithi110 ceph-mon[34550]: pgmap v422: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.1 MiB/s wr, 789 op/s 2024-09-21T06:15:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:01 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:15:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:01 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:01 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:15:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:01 smithi107 ceph-mon[34380]: pgmap v422: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.1 MiB/s wr, 789 op/s 2024-09-21T06:15:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:01 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:15:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:01 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:01 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:15:01.839 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:01 smithi031 ceph-mon[31237]: pgmap v422: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 9.1 MiB/s wr, 789 op/s 2024-09-21T06:15:01.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:01 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:15:01.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:01 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:01.840 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:01 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:15:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 -- 172.21.15.31:0/1339696746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85810b6f0 msgr2=0x7fa85810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 --2- 172.21.15.31:0/1339696746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85810b6f0 0x7fa85810bb00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa840009900 tx=0x7fa84002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 -- 172.21.15.31:0/1339696746 shutdown_connections 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 --2- 172.21.15.31:0/1339696746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 0x7fa858112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 --2- 172.21.15.31:0/1339696746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85810b6f0 0x7fa85810bb00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 --2- 172.21.15.31:0/1339696746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa858105db0 0x7fa8581061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 -- 172.21.15.31:0/1339696746 >> 172.21.15.31:0/1339696746 conn(0x7fa858101800 msgr2=0x7fa858103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 -- 172.21.15.31:0/1339696746 shutdown_connections 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.243+0000 7fa85d212640 1 -- 172.21.15.31:0/1339696746 wait complete. 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.244+0000 7fa85d212640 1 Processor -- start 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.244+0000 7fa85d212640 1 -- start start 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.244+0000 7fa85d212640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 0x7fa85819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.244+0000 7fa85d212640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa85810b6f0 0x7fa85819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.244+0000 7fa85d212640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 0x7fa8581a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa85d212640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa858115d00 con 0x7fa858105db0 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa85d212640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa858115b80 con 0x7fa85810c040 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa85d212640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa858115e80 con 0x7fa85810b6f0 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 0x7fa85819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 0x7fa85819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54382/0 (socket says 172.21.15.31:54382) 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 -- 172.21.15.31:0/1346345511 learned_addr learned my addr 172.21.15.31:0/1346345511 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa84ffff640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa85810b6f0 0x7fa85819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa85759e640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 0x7fa8581a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 -- 172.21.15.31:0/1346345511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa85810b6f0 msgr2=0x7fa85819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa85810b6f0 0x7fa85819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 -- 172.21.15.31:0/1346345511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 msgr2=0x7fa8581a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 0x7fa8581a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 -- 172.21.15.31:0/1346345511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a2640 con 0x7fa858105db0 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa85759e640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 0x7fa8581a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa856d9d640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 0x7fa85819c6d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa84800ea60 tx=0x7fa84800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.245+0000 7fa854d99640 1 -- 172.21.15.31:0/1346345511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84800cde0 con 0x7fa858105db0 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.246+0000 7fa854d99640 1 -- 172.21.15.31:0/1346345511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8480035f0 con 0x7fa858105db0 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.246+0000 7fa854d99640 1 -- 172.21.15.31:0/1346345511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848010690 con 0x7fa858105db0 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.246+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a2930 con 0x7fa858105db0 2024-09-21T06:15:02.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.246+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581a9090 con 0x7fa858105db0 2024-09-21T06:15:02.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.248+0000 7fa863786740 1 Processor -- start 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 -- start start 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa854d99640 1 -- 172.21.15.31:0/1346345511 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa848010830 con 0x7fa858105db0 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a362650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a362b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a3630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac2a2d8e50 con 0x55ac29f921b0 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ac2a2c8280 con 0x55ac29f9a390 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa863786740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ac2a2cc090 con 0x55ac29fbb070 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a362b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa85759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a362650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa85759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a362650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50234/0 (socket says 172.21.15.31:50234) 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa85759e640 1 -- 172.21.15.31:0/3663721597 learned_addr learned my addr 172.21.15.31:0/3663721597 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.249+0000 7fa84ffff640 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a3630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa85759e640 1 -- 172.21.15.31:0/3663721597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 msgr2=0x55ac2a3630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa85759e640 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a3630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa85759e640 1 -- 172.21.15.31:0/3663721597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 msgr2=0x55ac2a362b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa85759e640 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a362b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa85759e640 1 -- 172.21.15.31:0/3663721597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2a026650 con 0x55ac29fbb070 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa84ffff640 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a3630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa85759e640 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a362650 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8500117e0 tx=0x7fa850011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d47677f209a9974 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa854d99640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8440b6700 0x7fa8440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa84d7fa640 1 -- 172.21.15.31:0/3663721597 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850011310 con 0x55ac29fbb070 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa856d9d640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8440b6700 0x7fa8440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa84d7fa640 1 -- 172.21.15.31:0/3663721597 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa850003e20 con 0x55ac29fbb070 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa84d7fa640 1 -- 172.21.15.31:0/3663721597 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500044f0 con 0x55ac29fbb070 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.250+0000 7fa854d99640 1 -- 172.21.15.31:0/1346345511 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa8480e7930 con 0x7fa858105db0 2024-09-21T06:15:02.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 -- 172.21.15.31:0/3663721597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 msgr2=0x55ac2a362650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa856d9d640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8440b6700 0x7fa8440b8c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8480e5170 tx=0x7fa8480eaf90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a362650 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8500117e0 tx=0x7fa850011ca0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 -- 172.21.15.31:0/3663721597 shutdown_connections 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a3630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a362b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 --2- 172.21.15.31:0/3663721597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a362650 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 -- 172.21.15.31:0/3663721597 >> 172.21.15.31:0/3663721597 conn(0x55ac29faf460 msgr2=0x55ac2a360540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 -- 172.21.15.31:0/3663721597 shutdown_connections 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.251+0000 7fa863786740 1 -- 172.21.15.31:0/3663721597 wait complete. 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 -- start start 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 Processor -- start 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a36dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f9a390 0x55ac2a36e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29fbb070 0x55ac2a37a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ac2a129700 con 0x55ac29f921b0 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ac2a263110 con 0x55ac29fbb070 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ac2a370e30 con 0x55ac29f9a390 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa863786740 10 client.0 _ll_register_callbacks cb 0x55ac2a27c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa85759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a36dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f9a390 0x55ac2a36e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa85759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a36dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54386/0 (socket says 172.21.15.31:54386) 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.252+0000 7fa85759e640 1 -- 172.21.15.31:0/3579474747 learned_addr learned my addr 172.21.15.31:0/3579474747 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa85759e640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29fbb070 0x55ac2a37a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa84ffff640 1 -- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29fbb070 msgr2=0x55ac2a37a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa84ffff640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29fbb070 0x55ac2a37a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa84ffff640 1 -- 172.21.15.31:0/3579474747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 msgr2=0x55ac2a36dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa84ffff640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a36dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa84ffff640 1 -- 172.21.15.31:0/3579474747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2a290c40 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa85759e640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a36dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa84ffff640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f9a390 0x55ac2a36e280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a930 tx=0x7fa840032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f00085dba7308a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa83e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa83e7fc640 1 -- 172.21.15.31:0/3579474747 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840035040 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa83e7fc640 1 -- 172.21.15.31:0/3579474747 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa8400316a0 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.253+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa850003e20 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa83e7fc640 1 -- 172.21.15.31:0/3579474747 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa8400318a0 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 10 client.25975 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ac2a37cb70 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa83e7fc640 1 -- 172.21.15.31:0/3579474747 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa850003e20 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa83e7fc640 1 -- 172.21.15.31:0/3579474747 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa840004750 con 0x55ac29f9a390 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 10 client.25975 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 20 client.25975 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:02.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f799f0 0x55ac2a37ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 4 client.25975 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac2a2dada0 con 0x55ac29f799f0 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.254+0000 7fa85759e640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f799f0 0x55ac2a37ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.255+0000 7fa85759e640 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f799f0 0x55ac2a37ff90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f2964c1d0ab83e server_cookie=dbb47c575dd029a5 in_seq=0 out_seq=0 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.255+0000 7fa83e7fc640 10 client.25975 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa83e7fc640 1 -- 172.21.15.31:0/3579474747 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac2a2dada0 con 0x55ac29f799f0 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa83e7fc640 10 client.25975 handle_command_reply: tid=0 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 client.25975 shutdown 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f9a390 msgr2=0x55ac2a36e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f9a390 0x55ac2a36e280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a930 tx=0x7fa840032d50 comp rx=0 tx=0).stop 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 shutdown_connections 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f799f0 0x55ac2a37ff90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29fbb070 0x55ac2a37a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f9a390 0x55ac2a36e280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 --2- 172.21.15.31:0/3579474747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f921b0 0x55ac2a36dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 >> 172.21.15.31:0/3579474747 conn(0x55ac29faf460 msgr2=0x55ac2a3657e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:02.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa83e7fc640 0 client.25975 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:02.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 shutdown_connections 2024-09-21T06:15:02.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 1 -- 172.21.15.31:0/3579474747 wait complete. 2024-09-21T06:15:02.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.256+0000 7fa863786740 20 client.25975 trim_cache size 0 max 16384 2024-09-21T06:15:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.277+0000 7fa863786740 1 Processor -- start 2024-09-21T06:15:02.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.277+0000 7fa863786740 1 -- start start 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa863786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f921b0 0x55ac2a36e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa863786740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 0x55ac2a37a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa863786740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a36af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa863786740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581af940 con 0x55ac29f9a390 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa863786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b21c0 con 0x55ac29f921b0 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa863786740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581affb0 con 0x55ac29fbb070 2024-09-21T06:15:02.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 0x55ac2a37a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa85759e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f921b0 0x55ac2a36e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 0x55ac2a37a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54396/0 (socket says 172.21.15.31:54396) 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa84ffff640 1 -- 172.21.15.31:0/1042332184 learned_addr learned my addr 172.21.15.31:0/1042332184 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.278+0000 7fa85759e640 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a36af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84ffff640 1 -- 172.21.15.31:0/1042332184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 msgr2=0x55ac2a36af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84ffff640 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a36af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84ffff640 1 -- 172.21.15.31:0/1042332184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f921b0 msgr2=0x55ac2a36e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84ffff640 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f921b0 0x55ac2a36e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84ffff640 1 -- 172.21.15.31:0/1042332184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac2a363460 con 0x55ac29f9a390 2024-09-21T06:15:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84ffff640 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 0x55ac2a37a020 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa840006010 tx=0x7fa84002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f400d314f58458 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84cff9640 1 -- 172.21.15.31:0/1042332184 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840035070 con 0x55ac29f9a390 2024-09-21T06:15:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84cff9640 1 -- 172.21.15.31:0/1042332184 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa840005db0 con 0x55ac29f9a390 2024-09-21T06:15:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.279+0000 7fa84cff9640 1 -- 172.21.15.31:0/1042332184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840004d40 con 0x55ac29f9a390 2024-09-21T06:15:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 -- 172.21.15.31:0/1042332184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 msgr2=0x55ac2a37a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 0x55ac2a37a020 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa840006010 tx=0x7fa84002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 -- 172.21.15.31:0/1042332184 shutdown_connections 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29fbb070 0x55ac2a36af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29f9a390 0x55ac2a37a020 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 --2- 172.21.15.31:0/1042332184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f921b0 0x55ac2a36e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 -- 172.21.15.31:0/1042332184 >> 172.21.15.31:0/1042332184 conn(0x55ac29faf460 msgr2=0x55ac2a36daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 -- 172.21.15.31:0/1042332184 shutdown_connections 2024-09-21T06:15:02.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.280+0000 7fa863786740 1 -- 172.21.15.31:0/1042332184 wait complete. 2024-09-21T06:15:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 -- start start 2024-09-21T06:15:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 Processor -- start 2024-09-21T06:15:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f921b0 0x55ac2a376a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a371f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29fbb070 0x55ac2a372680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b0680 con 0x55ac29fbb070 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b1af0 con 0x55ac29f9a390 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8581b0d50 con 0x55ac29f921b0 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa863786740 10 client.0 _ll_register_callbacks cb 0x7fa848005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a371f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa85759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29fbb070 0x55ac2a372680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa84ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a371f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38998/0 (socket says 172.21.15.31:38998) 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.281+0000 7fa84ffff640 1 -- 172.21.15.31:0/2559673221 learned_addr learned my addr 172.21.15.31:0/2559673221 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa85759e640 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f921b0 0x55ac2a376a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa84ffff640 1 -- 172.21.15.31:0/2559673221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f921b0 msgr2=0x55ac2a376a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa84ffff640 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f921b0 0x55ac2a376a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa84ffff640 1 -- 172.21.15.31:0/2559673221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29fbb070 msgr2=0x55ac2a372680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa84ffff640 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29fbb070 0x55ac2a372680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa84ffff640 1 -- 172.21.15.31:0/2559673221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac2a3e9900 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa84ffff640 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a371f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a930 tx=0x7fa840009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa527950900085d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa83e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa83e7fc640 1 -- 172.21.15.31:0/2559673221 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840035040 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa83e7fc640 1 -- 172.21.15.31:0/2559673221 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa840005150 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.282+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa840005db0 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa83e7fc640 1 -- 172.21.15.31:0/2559673221 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa840065050 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 10 client.25949 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ac2a3eaa60 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa83e7fc640 1 -- 172.21.15.31:0/2559673221 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa840005db0 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa83e7fc640 1 -- 172.21.15.31:0/2559673221 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa84003c440 con 0x55ac29f9a390 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 10 client.25949 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 20 client.25949 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f4eda0 0x55ac2a3ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:02.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 4 client.25949 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:15:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac2a2d6850 con 0x55ac29f4eda0 2024-09-21T06:15:02.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.283+0000 7fa85759e640 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f4eda0 0x55ac2a3ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.284+0000 7fa83e7fc640 10 client.25949 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.284+0000 7fa85759e640 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f4eda0 0x55ac2a3ed1a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5d427755fc22a8 server_cookie=7badb2642280c5a9 in_seq=0 out_seq=0 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.284+0000 7fa83e7fc640 1 -- 172.21.15.31:0/2559673221 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x55ac2a2d6850 con 0x55ac29f4eda0 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.284+0000 7fa83e7fc640 10 client.25949 handle_command_reply: tid=0 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.284+0000 7fa863786740 1 client.25949 shutdown 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 msgr2=0x55ac2a371f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a371f70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa84002a930 tx=0x7fa840009780 comp rx=0 tx=0).stop 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 shutdown_connections 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ac29f4eda0 0x55ac2a3ed1a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ac29fbb070 0x55ac2a372680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ac29f9a390 0x55ac2a371f70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa83e7fc640 0 client.25949 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 --2- 172.21.15.31:0/2559673221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac29f921b0 0x55ac2a376a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 >> 172.21.15.31:0/2559673221 conn(0x55ac29faf460 msgr2=0x55ac2a363680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 shutdown_connections 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 1 -- 172.21.15.31:0/2559673221 wait complete. 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.285+0000 7fa863786740 20 client.25949 trim_cache size 0 max 16384 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:15:02.290 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:15:02.291 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:02.296 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8440b6700 msgr2=0x7fa8440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8440b6700 0x7fa8440b8c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8480e5170 tx=0x7fa8480eaf90 comp rx=0 tx=0).stop 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 msgr2=0x7fa85819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 0x7fa85819c6d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa84800ea60 tx=0x7fa84800ef20 comp rx=0 tx=0).stop 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 shutdown_connections 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85810c040 0x7fa8581a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa85810b6f0 0x7fa85819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8440b6700 0x7fa8440b8c10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 --2- 172.21.15.31:0/1346345511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa858105db0 0x7fa85819c6d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:02.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 >> 172.21.15.31:0/1346345511 conn(0x7fa858101800 msgr2=0x7fa858103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:02.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 shutdown_connections 2024-09-21T06:15:02.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:02.286+0000 7fa85d212640 1 -- 172.21.15.31:0/1346345511 wait complete. 2024-09-21T06:15:02.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:02 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:02.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:02 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:15:02.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:02 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:02.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:02 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:15:02.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:02 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:02.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:02 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:15:03.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:15:03.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:15:03.370 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:03.676 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:03 smithi031 ceph-mon[31237]: pgmap v423: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T06:15:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:03 smithi110 ceph-mon[34550]: pgmap v423: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T06:15:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:03 smithi107 ceph-mon[34380]: pgmap v423: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T06:15:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 -- 172.21.15.31:0/148718792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 msgr2=0x7f18cc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 --2- 172.21.15.31:0/148718792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc102ef0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009900 tx=0x7f18c802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:03.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 -- 172.21.15.31:0/148718792 shutdown_connections 2024-09-21T06:15:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 --2- 172.21.15.31:0/148718792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18cc108d70 0x7f18cc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 --2- 172.21.15.31:0/148718792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18cc108420 0x7f18cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 --2- 172.21.15.31:0/148718792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc102ef0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 -- 172.21.15.31:0/148718792 >> 172.21.15.31:0/148718792 conn(0x7f18cc079770 msgr2=0x7f18cc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 -- 172.21.15.31:0/148718792 shutdown_connections 2024-09-21T06:15:03.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.909+0000 7f18d3d29640 1 -- 172.21.15.31:0/148718792 wait complete. 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 Processor -- start 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 -- start start 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18cc108420 0x7f18cc19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18cc108d70 0x7f18cc19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc112a30 con 0x7f18cc102ae0 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1128b0 con 0x7f18cc108420 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d3d29640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc112bb0 con 0x7f18cc108d70 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.910+0000 7f18d1ac5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54430/0 (socket says 172.21.15.31:54430) 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 -- 172.21.15.31:0/2513006575 learned_addr learned my addr 172.21.15.31:0/2513006575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:03.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d12c4640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18cc108420 0x7f18cc19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d22c6640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18cc108d70 0x7f18cc19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 -- 172.21.15.31:0/2513006575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18cc108d70 msgr2=0x7f18cc19be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18cc108d70 0x7f18cc19be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 -- 172.21.15.31:0/2513006575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18cc108420 msgr2=0x7f18cc19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18cc108420 0x7f18cc19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 -- 172.21.15.31:0/2513006575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc1a6690 con 0x7f18cc102ae0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d1ac5640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc19b380 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005940 tx=0x7f18c80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c803f030 con 0x7f18cc102ae0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c80044c0 con 0x7f18cc102ae0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc1a6920 con 0x7f18cc102ae0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8035070 con 0x7f18cc102ae0 2024-09-21T06:15:03.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.911+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1a6db0 con 0x7f18cc102ae0 2024-09-21T06:15:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.913+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc103d60 con 0x7f18cc102ae0 2024-09-21T06:15:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.914+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f18c8042030 con 0x7f18cc102ae0 2024-09-21T06:15:03.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.915+0000 7f18c2ffd640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f18b80b6680 0x7f18b80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.915+0000 7f18d12c4640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f18b80b6680 0x7f18b80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.915+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f18c8107ca0 con 0x7f18cc102ae0 2024-09-21T06:15:03.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.915+0000 7f18d12c4640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f18b80b6680 0x7f18b80b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f18cc105440 tx=0x7f18bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:03.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:03.918+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18c80d2610 con 0x7f18cc102ae0 2024-09-21T06:15:04.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.176+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18cc06bfd0 con 0x7f18cc102ae0 2024-09-21T06:15:04.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.177+0000 7f18c2ffd640 1 -- 172.21.15.31:0/2513006575 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f18c80d7550 con 0x7f18cc102ae0 2024-09-21T06:15:04.179 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:15:04.179 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:15:04.179 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:15:04.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.179+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f18b80b6680 msgr2=0x7f18b80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:04.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.179+0000 7f18d3d29640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f18b80b6680 0x7f18b80b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f18cc105440 tx=0x7f18bc005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.179+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 msgr2=0x7f18cc19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.179+0000 7f18d3d29640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc19b380 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005940 tx=0x7f18c80040c0 comp rx=0 tx=0).stop 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 shutdown_connections 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18cc108d70 0x7f18cc19be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18cc108420 0x7f18cc19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f18b80b6680 0x7f18b80b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 --2- 172.21.15.31:0/2513006575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18cc102ae0 0x7f18cc19b380 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 >> 172.21.15.31:0/2513006575 conn(0x7f18cc079770 msgr2=0x7f18cc106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 shutdown_connections 2024-09-21T06:15:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:04.180+0000 7f18d3d29640 1 -- 172.21.15.31:0/2513006575 wait complete. 2024-09-21T06:15:04.490 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:04 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2513006575' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:04.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:04 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2513006575' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:04.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:04 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2513006575' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:05.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:15:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 205.0 s, 556.2 tps, lat 171.329 ms stddev 200.037 2024-09-21T06:15:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:05.707 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:05 smithi031 ceph-mon[31237]: pgmap v424: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.2 MiB/s wr, 765 op/s 2024-09-21T06:15:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:05 smithi110 ceph-mon[34550]: pgmap v424: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.2 MiB/s wr, 765 op/s 2024-09-21T06:15:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:05 smithi107 ceph-mon[34380]: pgmap v424: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.2 MiB/s wr, 765 op/s 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.948+0000 7feb8acbe640 1 -- 172.21.15.31:0/1413682649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb841094b0 msgr2=0x7feb841098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.948+0000 7feb8acbe640 1 --2- 172.21.15.31:0/1413682649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb841094b0 0x7feb841098c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feb70009900 tx=0x7feb7002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 -- 172.21.15.31:0/1413682649 shutdown_connections 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 --2- 172.21.15.31:0/1413682649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 --2- 172.21.15.31:0/1413682649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb841094b0 0x7feb841098c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 --2- 172.21.15.31:0/1413682649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb84101f70 0x7feb84102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 -- 172.21.15.31:0/1413682649 >> 172.21.15.31:0/1413682649 conn(0x7feb84077c60 msgr2=0x7feb840780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 -- 172.21.15.31:0/1413682649 shutdown_connections 2024-09-21T06:15:05.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 -- 172.21.15.31:0/1413682649 wait complete. 2024-09-21T06:15:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.949+0000 7feb8acbe640 1 Processor -- start 2024-09-21T06:15:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 -- start start 2024-09-21T06:15:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 0x7feb84102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb841094b0 0x7feb84103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb84113ac0 con 0x7feb841094b0 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb84113940 con 0x7feb84101f70 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8acbe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb84113c40 con 0x7feb84109e00 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 0x7feb84102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50282/0 (socket says 172.21.15.31:50282) 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 0x7feb84102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39042/0 (socket says 172.21.15.31:39042) 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 -- 172.21.15.31:0/4129868427 learned_addr learned my addr 172.21.15.31:0/4129868427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 -- 172.21.15.31:0/4129868427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 msgr2=0x7feb84102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 0x7feb84102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 -- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb841094b0 msgr2=0x7feb84103450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.950+0000 7feb8925b640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb841094b0 0x7feb84103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb8925b640 1 -- 172.21.15.31:0/4129868427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb841a8710 con 0x7feb84109e00 2024-09-21T06:15:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb88a5a640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 0x7feb84102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb8925b640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84103990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00eb10 tx=0x7feb7c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb81ffb640 1 -- 172.21.15.31:0/4129868427 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c004b20 con 0x7feb84109e00 2024-09-21T06:15:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb81ffb640 1 -- 172.21.15.31:0/4129868427 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c004cc0 con 0x7feb84109e00 2024-09-21T06:15:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb81ffb640 1 -- 172.21.15.31:0/4129868427 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c010670 con 0x7feb84109e00 2024-09-21T06:15:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb841a8a00 con 0x7feb84109e00 2024-09-21T06:15:05.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.951+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a8f90 con 0x7feb84109e00 2024-09-21T06:15:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.954+0000 7feb91232740 1 Processor -- start 2024-09-21T06:15:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.954+0000 7feb81ffb640 1 -- 172.21.15.31:0/4129868427 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7feb7c005020 con 0x7feb84109e00 2024-09-21T06:15:05.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 -- start start 2024-09-21T06:15:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc50be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b22dbc6ea0 con 0x55b22d8a9070 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b22dbb62d0 con 0x55b22d8801b0 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb91232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b22dbba0e0 con 0x55b22d888390 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54460/0 (socket says 172.21.15.31:54460) 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb83fff640 1 -- 172.21.15.31:0/3137731715 learned_addr learned my addr 172.21.15.31:0/3137731715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb88a5a640 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc50be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.955+0000 7feb83fff640 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb83fff640 1 -- 172.21.15.31:0/3137731715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 msgr2=0x55b22dc50be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb83fff640 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc50be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb83fff640 1 -- 172.21.15.31:0/3137731715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 msgr2=0x55b22dc506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb81ffb640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb740b6700 0x7feb740b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb83fff640 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb83fff640 1 -- 172.21.15.31:0/3137731715 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22d914650 con 0x55b22d888390 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb83fff640 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc51120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb700096c0 tx=0x7feb70005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c92ccb5614d249 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb8925b640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb740b6700 0x7feb740b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb81ffb640 1 -- 172.21.15.31:0/4129868427 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feb7c0e7c90 con 0x7feb84109e00 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb6effd640 1 -- 172.21.15.31:0/3137731715 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70035040 con 0x55b22d888390 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb6effd640 1 -- 172.21.15.31:0/3137731715 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb70044030 con 0x55b22d888390 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.956+0000 7feb6effd640 1 -- 172.21.15.31:0/3137731715 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb700035f0 con 0x55b22d888390 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 -- 172.21.15.31:0/3137731715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 msgr2=0x55b22dc51120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb8925b640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb740b6700 0x7feb740b8c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0e54d0 tx=0x7feb7c00cee0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc51120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb700096c0 tx=0x7feb70005fd0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 -- 172.21.15.31:0/3137731715 shutdown_connections 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc51120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc50be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 --2- 172.21.15.31:0/3137731715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 -- 172.21.15.31:0/3137731715 >> 172.21.15.31:0/3137731715 conn(0x55b22d89d460 msgr2=0x55b22dc4e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 -- 172.21.15.31:0/3137731715 shutdown_connections 2024-09-21T06:15:05.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 -- 172.21.15.31:0/3137731715 wait complete. 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.958+0000 7feb91232740 1 -- start start 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 Processor -- start 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc5c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc68110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b22da17700 con 0x55b22d8a9070 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b22db51160 con 0x55b22d8801b0 2024-09-21T06:15:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b22dc5ee20 con 0x55b22d888390 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb91232740 10 client.0 _ll_register_callbacks cb 0x55b22db6a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.959+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc5bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39060/0 (socket says 172.21.15.31:39060) 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb83fff640 1 -- 172.21.15.31:0/1923446703 learned_addr learned my addr 172.21.15.31:0/1923446703 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc5c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb83fff640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc68110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 -- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 msgr2=0x55b22dc5bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc5bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 -- 172.21.15.31:0/1923446703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 msgr2=0x55b22dc68110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc68110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 -- 172.21.15.31:0/1923446703 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22db7ec90 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb83fff640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb88a5a640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc5c2e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feb84104960 tx=0x7feb7800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e35dcaf8b81a495 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb63fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb63fff640 1 -- 172.21.15.31:0/1923446703 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78012110 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb63fff640 1 -- 172.21.15.31:0/1923446703 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feb78004030 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.960+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb70044030 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb63fff640 1 -- 172.21.15.31:0/1923446703 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7feb780122b0 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 10 client.25999 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b22dc6ab70 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb63fff640 1 -- 172.21.15.31:0/1923446703 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb70044030 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb63fff640 1 -- 172.21.15.31:0/1923446703 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7feb78025440 con 0x55b22d888390 2024-09-21T06:15:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 10 client.25999 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 20 client.25999 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8679f0 0x55b22dc6e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 4 client.25999 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.961+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22dbc8df0 con 0x55b22d8679f0 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.962+0000 7feb83fff640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8679f0 0x55b22dc6e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.962+0000 7feb63fff640 10 client.25999 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.962+0000 7feb83fff640 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8679f0 0x55b22dc6e030 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb5e3effe397e9e server_cookie=cf2e2c6e8418c872 in_seq=0 out_seq=0 2024-09-21T06:15:05.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb63fff640 1 -- 172.21.15.31:0/1923446703 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b22dbc8df0 con 0x55b22d8679f0 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb63fff640 10 client.25999 handle_command_reply: tid=0 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 client.25999 shutdown 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 msgr2=0x55b22dc5c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc5c2e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feb84104960 tx=0x7feb7800f790 comp rx=0 tx=0).stop 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 shutdown_connections 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8679f0 0x55b22dc6e030 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8a9070 0x55b22dc68110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb63fff640 0 client.25999 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dc5c2e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 --2- 172.21.15.31:0/1923446703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8801b0 0x55b22dc5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 >> 172.21.15.31:0/1923446703 conn(0x55b22d89d460 msgr2=0x55b22dc53880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 shutdown_connections 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.963+0000 7feb91232740 1 -- 172.21.15.31:0/1923446703 wait complete. 2024-09-21T06:15:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.964+0000 7feb91232740 20 client.25999 trim_cache size 0 max 16384 2024-09-21T06:15:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 Processor -- start 2024-09-21T06:15:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 -- start start 2024-09-21T06:15:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc5dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 0x55b22dc51de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d888390 0x55b22dc68010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb841af860 con 0x55b22d888390 2024-09-21T06:15:05.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b20e0 con 0x55b22d8679f0 2024-09-21T06:15:05.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.983+0000 7feb91232740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb841afed0 con 0x55b22d8801b0 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 0x55b22dc51de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc5dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc5dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39066/0 (socket says 172.21.15.31:39066) 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 0x55b22dc51de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50314/0 (socket says 172.21.15.31:50314) 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 -- 172.21.15.31:0/2505715861 learned_addr learned my addr 172.21.15.31:0/2505715861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d888390 0x55b22dc68010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 -- 172.21.15.31:0/2505715861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 msgr2=0x55b22dc51de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 0x55b22dc51de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 -- 172.21.15.31:0/2505715861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d888390 msgr2=0x55b22dc68010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d888390 0x55b22dc68010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 -- 172.21.15.31:0/2505715861 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22dc52990 con 0x55b22d8679f0 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb88a5a640 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 0x55b22dc51de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.984+0000 7feb83fff640 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc5dc80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb7002a910 tx=0x7feb70005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537092f8b9c1432a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.985+0000 7feb6e7fc640 1 -- 172.21.15.31:0/2505715861 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70032640 con 0x55b22d8679f0 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.985+0000 7feb6e7fc640 1 -- 172.21.15.31:0/2505715861 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb70044070 con 0x55b22d8679f0 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.985+0000 7feb6e7fc640 1 -- 172.21.15.31:0/2505715861 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70003c10 con 0x55b22d8679f0 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.985+0000 7feb91232740 1 -- 172.21.15.31:0/2505715861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 msgr2=0x55b22dc5dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.985+0000 7feb91232740 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc5dc80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feb7002a910 tx=0x7feb70005d50 comp rx=0 tx=0).stop 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 -- 172.21.15.31:0/2505715861 shutdown_connections 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d888390 0x55b22dc68010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d8801b0 0x55b22dc51de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 --2- 172.21.15.31:0/2505715861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc5dc80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 -- 172.21.15.31:0/2505715861 >> 172.21.15.31:0/2505715861 conn(0x55b22d89d460 msgr2=0x55b22dc516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 -- 172.21.15.31:0/2505715861 shutdown_connections 2024-09-21T06:15:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 -- 172.21.15.31:0/2505715861 wait complete. 2024-09-21T06:15:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 -- start start 2024-09-21T06:15:05.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.986+0000 7feb91232740 1 Processor -- start 2024-09-21T06:15:05.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8801b0 0x55b22dc60910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dcd7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8801b0 0x55b22dc60910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39070/0 (socket says 172.21.15.31:39070) 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb88a5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8801b0 0x55b22dc60910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54472/0 (socket says 172.21.15.31:54472) 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb83fff640 1 -- 172.21.15.31:0/2080695732 learned_addr learned my addr 172.21.15.31:0/2080695732 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b05a0 con 0x55b22d8801b0 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b1a10 con 0x55b22d8679f0 2024-09-21T06:15:05.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b0c70 con 0x55b22d888390 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb83fff640 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dcd7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.987+0000 7feb91232740 10 client.0 _ll_register_callbacks cb 0x7feb78009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb83fff640 1 -- 172.21.15.31:0/2080695732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 msgr2=0x55b22dcd7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb83fff640 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dcd7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb83fff640 1 -- 172.21.15.31:0/2080695732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8801b0 msgr2=0x55b22dc60910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb83fff640 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8801b0 0x55b22dc60910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb83fff640 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b22dc611e0 con 0x55b22d8679f0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb70044070 con 0x55b22d8679f0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb83fff640 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc603d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb70005f40 tx=0x7feb70005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1061b5f7bc5c927f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb63fff640 10 client.25973 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb63fff640 1 -- 172.21.15.31:0/2080695732 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70069060 con 0x55b22d8679f0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb63fff640 1 -- 172.21.15.31:0/2080695732 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feb700325f0 con 0x55b22d8679f0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb63fff640 1 -- 172.21.15.31:0/2080695732 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7feb70035070 con 0x55b22d8679f0 2024-09-21T06:15:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb91232740 10 client.25973 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.988+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55b22dcd8eb0 con 0x55b22d8679f0 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb63fff640 1 -- 172.21.15.31:0/2080695732 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7feb7005e030 con 0x55b22d8679f0 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb91232740 10 client.25973 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb91232740 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb91232740 20 client.25973 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb91232740 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8a9070 0x55b22dcdb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb91232740 4 client.25973 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b22dbc48a0 con 0x55b22d8a9070 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.989+0000 7feb88a5a640 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8a9070 0x55b22dcdb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb63fff640 10 client.25973 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb88a5a640 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8a9070 0x55b22dcdb090 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd42ec2af43514c1 server_cookie=95b5a373b92800a4 in_seq=0 out_seq=0 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb63fff640 1 -- 172.21.15.31:0/2080695732 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b22dbc48a0 con 0x55b22d8a9070 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb63fff640 10 client.25973 handle_command_reply: tid=0 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 client.25973 shutdown 2024-09-21T06:15:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 msgr2=0x55b22dc603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc603d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb70005f40 tx=0x7feb70005c90 comp rx=0 tx=0).stop 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 shutdown_connections 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b22d8a9070 0x55b22dcdb090 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb63fff640 0 client.25973 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b22d888390 0x55b22dcd7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b22d8801b0 0x55b22dc60910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 --2- 172.21.15.31:0/2080695732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b22d8679f0 0x55b22dc603d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 >> 172.21.15.31:0/2080695732 conn(0x55b22d89d460 msgr2=0x55b22dc5d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 shutdown_connections 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.990+0000 7feb91232740 1 -- 172.21.15.31:0/2080695732 wait complete. 2024-09-21T06:15:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb91232740 20 client.25973 trim_cache size 0 max 16384 2024-09-21T06:15:05.998 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:15:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb740b6700 msgr2=0x7feb740b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb740b6700 0x7feb740b8c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0e54d0 tx=0x7feb7c00cee0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 msgr2=0x7feb84103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84103990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feb7c00eb10 tx=0x7feb7c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 shutdown_connections 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb84109e00 0x7feb84103990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feb740b6700 0x7feb740b8c10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb841094b0 0x7feb84103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 --2- 172.21.15.31:0/4129868427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84101f70 0x7feb84102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 >> 172.21.15.31:0/4129868427 conn(0x7feb84077c60 msgr2=0x7feb84107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 shutdown_connections 2024-09-21T06:15:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:05.991+0000 7feb8acbe640 1 -- 172.21.15.31:0/4129868427 wait complete. 2024-09-21T06:15:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:07 smithi110 ceph-mon[34550]: pgmap v425: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-09-21T06:15:07.788 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:07 smithi107 ceph-mon[34380]: pgmap v425: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-09-21T06:15:07.799 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:15:07.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:15:07.823 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:07 smithi031 ceph-mon[31237]: pgmap v425: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-09-21T06:15:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 -- 172.21.15.31:0/3411273946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 msgr2=0x7f03f410d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 --2- 172.21.15.31:0/3411273946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f410d810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 -- 172.21.15.31:0/3411273946 shutdown_connections 2024-09-21T06:15:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 --2- 172.21.15.31:0/3411273946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f410dd50 0x7f03f4110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 --2- 172.21.15.31:0/3411273946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f410d810 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 --2- 172.21.15.31:0/3411273946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f41047a0 0x7f03f4104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 -- 172.21.15.31:0/3411273946 >> 172.21.15.31:0/3411273946 conn(0x7f03f407a740 msgr2=0x7f03f407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 -- 172.21.15.31:0/3411273946 shutdown_connections 2024-09-21T06:15:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.485+0000 7f03f9da1640 1 -- 172.21.15.31:0/3411273946 wait complete. 2024-09-21T06:15:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 Processor -- start 2024-09-21T06:15:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 -- start start 2024-09-21T06:15:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 0x7f03f419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f410dd50 0x7f03f419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41139d0 con 0x7f03f41051d0 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4113850 con 0x7f03f41047a0 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.486+0000 7f03f9da1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4113b50 con 0x7f03f410dd50 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 0x7f03f419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 0x7f03f419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39082/0 (socket says 172.21.15.31:39082) 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 -- 172.21.15.31:0/1565027135 learned_addr learned my addr 172.21.15.31:0/1565027135 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54494/0 (socket says 172.21.15.31:54494) 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f3fff640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f410dd50 0x7f03f419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 -- 172.21.15.31:0/1565027135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f410dd50 msgr2=0x7f03f419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f410dd50 0x7f03f419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 -- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 msgr2=0x7f03f419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 -- 172.21.15.31:0/1565027135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f41a0380 con 0x7f03f41047a0 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f37fe640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 0x7f03f419a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f03e400ea60 tx=0x7f03e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f0ff9640 1 -- 172.21.15.31:0/1565027135 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e400cde0 con 0x7f03f41047a0 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.487+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41a0670 con 0x7f03f41047a0 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.488+0000 7f03f0ff9640 1 -- 172.21.15.31:0/1565027135 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e40035f0 con 0x7f03f41047a0 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.488+0000 7f03f0ff9640 1 -- 172.21.15.31:0/1565027135 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4010690 con 0x7f03f41047a0 2024-09-21T06:15:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.488+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f41a6e00 con 0x7f03f41047a0 2024-09-21T06:15:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.490+0000 7f0400de9740 1 Processor -- start 2024-09-21T06:15:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.490+0000 7f0400de9740 1 -- start start 2024-09-21T06:15:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f0400de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b5139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f0400de9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b1452f0 0x56423b513ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f0400de9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b514430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f0400de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56423b4886f0 con 0x56423b16e1b0 2024-09-21T06:15:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f0400de9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56423b477b20 con 0x56423b14d4d0 2024-09-21T06:15:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f0400de9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56423b47b930 con 0x56423b1452f0 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b5139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b514430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b514430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39088/0 (socket says 172.21.15.31:39088) 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 -- 172.21.15.31:0/3046900374 learned_addr learned my addr 172.21.15.31:0/3046900374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 -- 172.21.15.31:0/3046900374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b1452f0 msgr2=0x56423b513ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b1452f0 0x56423b513ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 -- 172.21.15.31:0/3046900374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 msgr2=0x56423b5139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b5139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.491+0000 7f03f2ffd640 1 -- 172.21.15.31:0/3046900374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423b43d530 con 0x56423b14d4d0 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.492+0000 7f03f0ff9640 1 -- 172.21.15.31:0/1565027135 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f03e4010830 con 0x7f03f41047a0 2024-09-21T06:15:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.492+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b514430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03e8033000 tx=0x7f03e8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44b15ab69cd7a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.492+0000 7f03d9ffb640 1 -- 172.21.15.31:0/3046900374 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8035040 con 0x56423b14d4d0 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.492+0000 7f03d9ffb640 1 -- 172.21.15.31:0/3046900374 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e8004030 con 0x56423b14d4d0 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.492+0000 7f03d9ffb640 1 -- 172.21.15.31:0/3046900374 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e80053d0 con 0x56423b14d4d0 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.493+0000 7f03f0ff9640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03dc0b6700 0x7f03dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.493+0000 7f0400de9740 1 -- 172.21.15.31:0/3046900374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 msgr2=0x56423b514430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.493+0000 7f0400de9740 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b514430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03e8033000 tx=0x7f03e8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.493+0000 7f03f3fff640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03dc0b6700 0x7f03dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 -- 172.21.15.31:0/3046900374 shutdown_connections 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b514430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f03f0ff9640 1 -- 172.21.15.31:0/1565027135 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f03e40e7930 con 0x7f03f41047a0 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b1452f0 0x56423b513ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 --2- 172.21.15.31:0/3046900374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b5139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 -- 172.21.15.31:0/3046900374 >> 172.21.15.31:0/3046900374 conn(0x56423b162300 msgr2=0x56423b5118a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 -- 172.21.15.31:0/3046900374 shutdown_connections 2024-09-21T06:15:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 -- 172.21.15.31:0/3046900374 wait complete. 2024-09-21T06:15:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f03f3fff640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03dc0b6700 0x7f03dc0b8c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.494+0000 7f0400de9740 1 -- start start 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 Processor -- start 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b51f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b14d4d0 0x56423b51f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b52f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56423b3afab0 con 0x56423b16e1b0 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56423b41d730 con 0x56423b1452f0 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56423b5264e0 con 0x56423b14d4d0 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f0400de9740 10 client.0 _ll_register_callbacks cb 0x56423b4716b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b51f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b51f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39104/0 (socket says 172.21.15.31:39104) 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f03f37fe640 1 -- 172.21.15.31:0/1509148618 learned_addr learned my addr 172.21.15.31:0/1509148618 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.495+0000 7f03f3fff640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b52f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b14d4d0 0x56423b51f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f37fe640 1 -- 172.21.15.31:0/1509148618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b14d4d0 msgr2=0x56423b51f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f37fe640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b14d4d0 0x56423b51f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f37fe640 1 -- 172.21.15.31:0/1509148618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 msgr2=0x56423b52f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f37fe640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b52f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f37fe640 1 -- 172.21.15.31:0/1509148618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423b1d9790 con 0x56423b1452f0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03f37fe640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b51f140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03e40e56c0 tx=0x7f03e40a9f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318a0832f4192ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03d2ffd640 1 -- 172.21.15.31:0/1509148618 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e40ad2d0 con 0x56423b1452f0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f03d2ffd640 1 -- 172.21.15.31:0/1509148618 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f03e4004000 con 0x56423b1452f0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.496+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e8004030 con 0x56423b1452f0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f03d2ffd640 1 -- 172.21.15.31:0/1509148618 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f03e40adec0 con 0x56423b1452f0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 10 client.25985 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56423b252620 con 0x56423b1452f0 2024-09-21T06:15:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f03d2ffd640 1 -- 172.21.15.31:0/1509148618 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e8004030 con 0x56423b1452f0 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f03d2ffd640 1 -- 172.21.15.31:0/1509148618 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f03e40aeaf0 con 0x56423b1452f0 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 10 client.25985 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 20 client.25985 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b12cd80 0x56423b535a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 4 client.25985 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56423b48a640 con 0x56423b12cd80 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.497+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b12cd80 0x56423b535a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.498+0000 7f03d2ffd640 10 client.25985 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.498+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b12cd80 0x56423b535a50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55332fc952414abd server_cookie=c5768bf4bde5285 in_seq=0 out_seq=0 2024-09-21T06:15:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f03d2ffd640 1 -- 172.21.15.31:0/1509148618 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56423b48a640 con 0x56423b12cd80 2024-09-21T06:15:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f03d2ffd640 10 client.25985 handle_command_reply: tid=0 2024-09-21T06:15:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 client.25985 shutdown 2024-09-21T06:15:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 msgr2=0x56423b51f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b51f140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03e40e56c0 tx=0x7f03e40a9f20 comp rx=0 tx=0).stop 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 shutdown_connections 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b12cd80 0x56423b535a50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f03d2ffd640 0 client.25985 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b16e1b0 0x56423b52f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b14d4d0 0x56423b51f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 --2- 172.21.15.31:0/1509148618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b51f140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 >> 172.21.15.31:0/1509148618 conn(0x56423b162300 msgr2=0x56423b5149e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 shutdown_connections 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 1 -- 172.21.15.31:0/1509148618 wait complete. 2024-09-21T06:15:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.499+0000 7f0400de9740 20 client.25985 trim_cache size 0 max 16384 2024-09-21T06:15:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.516+0000 7f0400de9740 1 Processor -- start 2024-09-21T06:15:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.516+0000 7f0400de9740 1 -- start start 2024-09-21T06:15:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.516+0000 7f0400de9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b515010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.516+0000 7f0400de9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b52f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f0400de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 0x56423b52fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f0400de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ab3b0 con 0x56423b14d4d0 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f0400de9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41adc30 con 0x56423b1452f0 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f0400de9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41aba20 con 0x56423b12cd80 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 0x56423b52fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b515010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 0x56423b52fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54508/0 (socket says 172.21.15.31:54508) 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b52f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f3fff640 1 -- 172.21.15.31:0/451931900 learned_addr learned my addr 172.21.15.31:0/451931900 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 -- 172.21.15.31:0/451931900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 msgr2=0x56423b52f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b52f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 -- 172.21.15.31:0/451931900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 msgr2=0x56423b52fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 0x56423b52fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 -- 172.21.15.31:0/451931900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423b535f50 con 0x56423b12cd80 2024-09-21T06:15:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f3fff640 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 0x56423b52fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.517+0000 7f03f37fe640 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b515010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f03e40ae1d0 tx=0x7f03e4003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae3898cd52a9fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.518+0000 7f03d97fa640 1 -- 172.21.15.31:0/451931900 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e4098070 con 0x56423b12cd80 2024-09-21T06:15:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.518+0000 7f03d97fa640 1 -- 172.21.15.31:0/451931900 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e40e7110 con 0x56423b12cd80 2024-09-21T06:15:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.518+0000 7f03d97fa640 1 -- 172.21.15.31:0/451931900 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e40ad2d0 con 0x56423b12cd80 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.518+0000 7f0400de9740 1 -- 172.21.15.31:0/451931900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 msgr2=0x56423b515010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.518+0000 7f0400de9740 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b515010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f03e40ae1d0 tx=0x7f03e4003e90 comp rx=0 tx=0).stop 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 -- 172.21.15.31:0/451931900 shutdown_connections 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b14d4d0 0x56423b52fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b1452f0 0x56423b52f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 --2- 172.21.15.31:0/451931900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b515010 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 -- 172.21.15.31:0/451931900 >> 172.21.15.31:0/451931900 conn(0x56423b162300 msgr2=0x56423b514900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 -- 172.21.15.31:0/451931900 shutdown_connections 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 -- 172.21.15.31:0/451931900 wait complete. 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 -- start start 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.519+0000 7f0400de9740 1 Processor -- start 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b526b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b1452f0 0x56423b5270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b59e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ac0f0 con 0x56423b1452f0 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ad560 con 0x56423b14d4d0 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41ac7c0 con 0x56423b12cd80 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f0400de9740 10 client.0 _ll_register_callbacks cb 0x7f03e4003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b526b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b1452f0 0x56423b5270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b526b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50358/0 (socket says 172.21.15.31:50358) 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f37fe640 1 -- 172.21.15.31:0/2430279178 learned_addr learned my addr 172.21.15.31:0/2430279178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f3fff640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b59e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 -- 172.21.15.31:0/2430279178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 msgr2=0x56423b526b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b526b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 -- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 msgr2=0x56423b59e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b59e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 -- 172.21.15.31:0/2430279178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56423b5278b0 con 0x56423b1452f0 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f37fe640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b526b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03f2ffd640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b1452f0 0x56423b5270a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x56423b5361c0 tx=0x7f03e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58144965576d88c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.520+0000 7f03d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f03d2ffd640 1 -- 172.21.15.31:0/2430279178 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8035040 con 0x56423b1452f0 2024-09-21T06:15:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f03d2ffd640 1 -- 172.21.15.31:0/2430279178 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f03e8005590 con 0x56423b1452f0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56423b4860f0 con 0x56423b1452f0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f03d2ffd640 1 -- 172.21.15.31:0/2430279178 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f03e8065050 con 0x56423b1452f0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 10 client.16266 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56423b59f920 con 0x56423b1452f0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f03d2ffd640 1 -- 172.21.15.31:0/2430279178 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56423b4860f0 con 0x56423b1452f0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f03d2ffd640 1 -- 172.21.15.31:0/2430279178 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f03e80323e0 con 0x56423b1452f0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 10 client.16266 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 20 client.16266 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b16e1b0 0x56423b5a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 4 client.16266 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.521+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56423b483520 con 0x56423b16e1b0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.522+0000 7f03f37fe640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b16e1b0 0x56423b5a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.522+0000 7f03d2ffd640 10 client.16266 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.522+0000 7f03f37fe640 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b16e1b0 0x56423b5a1ce0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87ac2a8c518c472d server_cookie=717ed08933601f43 in_seq=0 out_seq=0 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.543+0000 7f03d2ffd640 1 -- 172.21.15.31:0/2430279178 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56423b483520 con 0x56423b16e1b0 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.543+0000 7f03d2ffd640 10 client.16266 handle_command_reply: tid=0 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.543+0000 7f0400de9740 1 client.16266 shutdown 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.543+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b1452f0 msgr2=0x56423b5270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.543+0000 7f0400de9740 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b1452f0 0x56423b5270a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x56423b5361c0 tx=0x7f03e802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 shutdown_connections 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56423b16e1b0 0x56423b5a1ce0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56423b14d4d0 0x56423b59e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56423b1452f0 0x56423b5270a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03d2ffd640 0 client.16266 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 --2- 172.21.15.31:0/2430279178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56423b12cd80 0x56423b526b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 >> 172.21.15.31:0/2430279178 conn(0x56423b162300 msgr2=0x56423b5152c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 shutdown_connections 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 1 -- 172.21.15.31:0/2430279178 wait complete. 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f0400de9740 20 client.16266 trim_cache size 0 max 16384 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:15:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03dc0b6700 msgr2=0x7f03dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03dc0b6700 0x7f03dc0b8c10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec009500 comp rx=0 tx=0).stop 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 msgr2=0x7f03f419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 0x7f03f419a450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f03e400ea60 tx=0x7f03e400ef20 comp rx=0 tx=0).stop 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 shutdown_connections 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03f410dd50 0x7f03f419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f03dc0b6700 0x7f03dc0b8c10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f41051d0 0x7f03f419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 --2- 172.21.15.31:0/1565027135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f41047a0 0x7f03f419a450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.544+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 >> 172.21.15.31:0/1565027135 conn(0x7f03f407a740 msgr2=0x7f03f40ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.545+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 shutdown_connections 2024-09-21T06:15:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:08.545+0000 7f03f9da1640 1 -- 172.21.15.31:0/1565027135 wait complete. 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 -- 172.21.15.31:0/1972553525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51441083a0 msgr2=0x7f51441087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 --2- 172.21.15.31:0/1972553525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51441083a0 0x7f51441087b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5138009900 tx=0x7f513802a410 comp rx=0 tx=0).stop 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 -- 172.21.15.31:0/1972553525 shutdown_connections 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 --2- 172.21.15.31:0/1972553525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144108cf0 0x7f514410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 --2- 172.21.15.31:0/1972553525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51441083a0 0x7f51441087b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 --2- 172.21.15.31:0/1972553525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5144102a60 0x7f5144102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 -- 172.21.15.31:0/1972553525 >> 172.21.15.31:0/1972553525 conn(0x7f5144079730 msgr2=0x7f5144079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 -- 172.21.15.31:0/1972553525 shutdown_connections 2024-09-21T06:15:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.602+0000 7f5149b65640 1 -- 172.21.15.31:0/1972553525 wait complete. 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 Processor -- start 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 -- start start 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 0x7f514419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51441083a0 0x7f514419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 0x7f514419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5144112960 con 0x7f5144108cf0 2024-09-21T06:15:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51441127e0 con 0x7f5144102a60 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.603+0000 7f5149b65640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5144112ae0 con 0x7f51441083a0 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 0x7f514419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f51437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 0x7f514419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51441083a0 0x7f514419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 0x7f514419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54534/0 (socket says 172.21.15.31:54534) 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f51437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 0x7f514419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39148/0 (socket says 172.21.15.31:39148) 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 -- 172.21.15.31:0/3302309223 learned_addr learned my addr 172.21.15.31:0/3302309223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 -- 172.21.15.31:0/3302309223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51441083a0 msgr2=0x7f514419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51441083a0 0x7f514419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 -- 172.21.15.31:0/3302309223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 msgr2=0x7f514419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 0x7f514419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 -- 172.21.15.31:0/3302309223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441a6500 con 0x7f5144108cf0 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f51437fe640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 0x7f514419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5143fff640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 0x7f514419bcf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eb10 tx=0x7f513c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5140ff9640 1 -- 172.21.15.31:0/3302309223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c004b20 con 0x7f5144108cf0 2024-09-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5140ff9640 1 -- 172.21.15.31:0/3302309223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c004cc0 con 0x7f5144108cf0 2024-09-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51441a67f0 con 0x7f5144108cf0 2024-09-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5140ff9640 1 -- 172.21.15.31:0/3302309223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c010670 con 0x7f5144108cf0 2024-09-21T06:15:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.604+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a6cb0 con 0x7f5144108cf0 2024-09-21T06:15:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.607+0000 7f51500d9740 1 Processor -- start 2024-09-21T06:15:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f5140ff9640 1 -- 172.21.15.31:0/3302309223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f513c010810 con 0x7f5144108cf0 2024-09-21T06:15:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 -- start start 2024-09-21T06:15:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab8d350 0x5608eaf34430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab6c670 0x5608eaf34eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5608eaeaae60 con 0x5608eab8d350 2024-09-21T06:15:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5608eae9a290 con 0x5608eab64490 2024-09-21T06:15:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51500d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5608eae9e0a0 con 0x5608eab6c670 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab8d350 0x5608eaf34430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab8d350 0x5608eaf34430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54546/0 (socket says 172.21.15.31:54546) 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f5142ffd640 1 -- 172.21.15.31:0/537291272 learned_addr learned my addr 172.21.15.31:0/537291272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f5142ffd640 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab6c670 0x5608eaf34eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.608+0000 7f51437fe640 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f51437fe640 1 -- 172.21.15.31:0/537291272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab6c670 msgr2=0x5608eaf34eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f51437fe640 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab6c670 0x5608eaf34eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f51437fe640 1 -- 172.21.15.31:0/537291272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab8d350 msgr2=0x5608eaf34430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f51437fe640 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab8d350 0x5608eaf34430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f51437fe640 1 -- 172.21.15.31:0/537291272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eabf8930 con 0x5608eab64490 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5140ff9640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f512c0b66b0 0x7f512c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5142ffd640 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab6c670 0x5608eaf34eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5142ffd640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f512c0b66b0 0x7f512c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f51437fe640 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf34970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5134009fa0 tx=0x7f51340118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60d2168d7e831e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5140ff9640 1 -- 172.21.15.31:0/3302309223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f513c0e7c60 con 0x7f5144108cf0 2024-09-21T06:15:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5129ffb640 1 -- 172.21.15.31:0/537291272 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51340040f0 con 0x5608eab64490 2024-09-21T06:15:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5129ffb640 1 -- 172.21.15.31:0/537291272 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51340054d0 con 0x5608eab64490 2024-09-21T06:15:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.609+0000 7f5129ffb640 1 -- 172.21.15.31:0/537291272 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134004ce0 con 0x5608eab64490 2024-09-21T06:15:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 -- 172.21.15.31:0/537291272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 msgr2=0x5608eaf34970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f5142ffd640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f512c0b66b0 0x7f512c0b8bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51380096c0 tx=0x7f5138031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf34970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5134009fa0 tx=0x7f51340118d0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 -- 172.21.15.31:0/537291272 shutdown_connections 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab6c670 0x5608eaf34eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf34970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 --2- 172.21.15.31:0/537291272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab8d350 0x5608eaf34430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 -- 172.21.15.31:0/537291272 >> 172.21.15.31:0/537291272 conn(0x5608eab81740 msgr2=0x5608eaf32320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 -- 172.21.15.31:0/537291272 shutdown_connections 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.610+0000 7f51500d9740 1 -- 172.21.15.31:0/537291272 wait complete. 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.611+0000 7f51500d9740 1 -- start start 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.611+0000 7f51500d9740 1 Processor -- start 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf40030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab8d350 0x5608eaf4bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5608eacfb890 con 0x5608eab6c670 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5608eae34d40 con 0x5608eab64490 2024-09-21T06:15:08.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5608eaf42c10 con 0x5608eab8d350 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51500d9740 10 client.0 _ll_register_callbacks cb 0x5608eae98c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf40030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf40030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39170/0 (socket says 172.21.15.31:39170) 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf40570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54562/0 (socket says 172.21.15.31:54562) 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 -- 172.21.15.31:0/4014226863 learned_addr learned my addr 172.21.15.31:0/4014226863 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f5143fff640 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab8d350 0x5608eaf4bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 -- 172.21.15.31:0/4014226863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab8d350 msgr2=0x5608eaf4bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab8d350 0x5608eaf4bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 -- 172.21.15.31:0/4014226863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 msgr2=0x5608eaf40570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf40570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 -- 172.21.15.31:0/4014226863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eaf4c4c0 con 0x5608eab64490 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f51437fe640 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf40030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f513401b040 tx=0x7f5134003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3143cd65f18a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.612+0000 7f5122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f5122ffd640 1 -- 172.21.15.31:0/4014226863 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134028020 con 0x5608eab64490 2024-09-21T06:15:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f5122ffd640 1 -- 172.21.15.31:0/4014226863 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f5134003c20 con 0x5608eab64490 2024-09-21T06:15:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51340054d0 con 0x5608eab64490 2024-09-21T06:15:08.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f5122ffd640 1 -- 172.21.15.31:0/4014226863 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5134033050 con 0x5608eab64490 2024-09-21T06:15:08.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 10 client.26003 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:08.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5608eaf4edf0 con 0x5608eab64490 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f5122ffd640 1 -- 172.21.15.31:0/4014226863 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51340054d0 con 0x5608eab64490 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f5122ffd640 1 -- 172.21.15.31:0/4014226863 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f5134005090 con 0x5608eab64490 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 10 client.26003 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 20 client.26003 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab4bcd0 0x5608eaf52260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 4 client.26003 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.613+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608eaeacdb0 con 0x5608eab4bcd0 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.614+0000 7f5142ffd640 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab4bcd0 0x5608eaf52260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.614+0000 7f5122ffd640 10 client.26003 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.614+0000 7f5142ffd640 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab4bcd0 0x5608eaf52260 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1258226b67174a0b server_cookie=e216f9329c2c9a25 in_seq=0 out_seq=0 2024-09-21T06:15:08.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f5122ffd640 1 -- 172.21.15.31:0/4014226863 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608eaeacdb0 con 0x5608eab4bcd0 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f5122ffd640 10 client.26003 handle_command_reply: tid=0 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 client.26003 shutdown 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 msgr2=0x5608eaf40030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf40030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f513401b040 tx=0x7f5134003820 comp rx=0 tx=0).stop 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 shutdown_connections 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab4bcd0 0x5608eaf52260 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab8d350 0x5608eaf4bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f5122ffd640 0 client.26003 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 --2- 172.21.15.31:0/4014226863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab64490 0x5608eaf40030 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 >> 172.21.15.31:0/4014226863 conn(0x5608eab81740 msgr2=0x5608eaf37610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 shutdown_connections 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.615+0000 7f51500d9740 1 -- 172.21.15.31:0/4014226863 wait complete. 2024-09-21T06:15:08.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.616+0000 7f51500d9740 20 client.26003 trim_cache size 0 max 16384 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 Processor -- start 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 -- start start 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab4bcd0 0x5608eaf4be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf39540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf39a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ab3a0 con 0x5608eab6c670 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51441adc20 con 0x5608eab4bcd0 2024-09-21T06:15:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.640+0000 7f51500d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51441aba10 con 0x5608eab64490 2024-09-21T06:15:08.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f51437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab4bcd0 0x5608eaf4be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf39540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf39a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf39540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50380/0 (socket says 172.21.15.31:50380) 2024-09-21T06:15:08.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f51437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab4bcd0 0x5608eaf4be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39180/0 (socket says 172.21.15.31:39180) 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5142ffd640 1 -- 172.21.15.31:0/1915201629 learned_addr learned my addr 172.21.15.31:0/1915201629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf39a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54570/0 (socket says 172.21.15.31:54570) 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 -- 172.21.15.31:0/1915201629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 msgr2=0x5608eaf39540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf39540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 -- 172.21.15.31:0/1915201629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab4bcd0 msgr2=0x5608eaf4be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab4bcd0 0x5608eaf4be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 -- 172.21.15.31:0/1915201629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eaf37290 con 0x5608eab6c670 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5142ffd640 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf39540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f5143fff640 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf39a80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f513c0e53f0 tx=0x7f513c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230b82b68537d380 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f51297fa640 1 -- 172.21.15.31:0/1915201629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0ad600 con 0x5608eab6c670 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f51297fa640 1 -- 172.21.15.31:0/1915201629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c0ae1f0 con 0x5608eab6c670 2024-09-21T06:15:08.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.641+0000 7f51297fa640 1 -- 172.21.15.31:0/1915201629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c010b00 con 0x5608eab6c670 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 -- 172.21.15.31:0/1915201629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 msgr2=0x5608eaf39a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf39a80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f513c0e53f0 tx=0x7f513c00cee0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 -- 172.21.15.31:0/1915201629 shutdown_connections 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab6c670 0x5608eaf39a80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf39540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 --2- 172.21.15.31:0/1915201629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab4bcd0 0x5608eaf4be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.642+0000 7f51500d9740 1 -- 172.21.15.31:0/1915201629 >> 172.21.15.31:0/1915201629 conn(0x5608eab81740 msgr2=0x5608eaf3e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.643+0000 7f51500d9740 1 -- 172.21.15.31:0/1915201629 shutdown_connections 2024-09-21T06:15:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.643+0000 7f51500d9740 1 -- 172.21.15.31:0/1915201629 wait complete. 2024-09-21T06:15:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.643+0000 7f51500d9740 1 -- start start 2024-09-21T06:15:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.643+0000 7f51500d9740 1 Processor -- start 2024-09-21T06:15:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 0x5608eaf444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf44a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab6c670 0x5608eafbbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ac0e0 con 0x5608eab4bcd0 2024-09-21T06:15:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ad550 con 0x5608eab6c670 2024-09-21T06:15:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ac7b0 con 0x5608eab64490 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51500d9740 10 client.0 _ll_register_callbacks cb 0x7f5134005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 0x5608eaf444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf44a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 0x5608eaf444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54574/0 (socket says 172.21.15.31:54574) 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 -- 172.21.15.31:0/270858472 learned_addr learned my addr 172.21.15.31:0/270858472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f5143fff640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab6c670 0x5608eafbbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 -- 172.21.15.31:0/270858472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 msgr2=0x5608eaf44a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf44a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 -- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab6c670 msgr2=0x5608eafbbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab6c670 0x5608eafbbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 -- 172.21.15.31:0/270858472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eafbc040 con 0x5608eab4bcd0 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f5142ffd640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf44a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f51437fe640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 0x5608eaf444e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51340059f0 tx=0x7f5134003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d2e2bd7afe30e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.644+0000 7f5122ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f5122ffd640 1 -- 172.21.15.31:0/270858472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134028070 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f5122ffd640 1 -- 172.21.15.31:0/270858472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f5134005460 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f513c0ae1f0 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f5122ffd640 1 -- 172.21.15.31:0/270858472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f513401d070 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f51500d9740 10 client.16284 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5608eafbd260 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f5122ffd640 1 -- 172.21.15.31:0/270858472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c0ae1f0 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f5122ffd640 1 -- 172.21.15.31:0/270858472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f513400faf0 con 0x5608eab4bcd0 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f51500d9740 10 client.16284 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.645+0000 7f51500d9740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f51500d9740 20 client.16284 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f51500d9740 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab8d350 0x5608eafbf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f51500d9740 4 client.16284 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:15:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5608eaea8860 con 0x5608eab8d350 2024-09-21T06:15:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f5142ffd640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab8d350 0x5608eafbf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f5122ffd640 10 client.16284 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.646+0000 7f5142ffd640 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab8d350 0x5608eafbf9f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5ce7747d4f5a2c server_cookie=ba5a3c1e0adb8703 in_seq=0 out_seq=0 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.648+0000 7f5122ffd640 1 -- 172.21.15.31:0/270858472 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5608eaea8860 con 0x5608eab8d350 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.648+0000 7f5122ffd640 10 client.16284 handle_command_reply: tid=0 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.648+0000 7f51500d9740 1 client.16284 shutdown 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 msgr2=0x5608eaf444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 0x5608eaf444e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f51340059f0 tx=0x7f5134003900 comp rx=0 tx=0).stop 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 shutdown_connections 2024-09-21T06:15:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5608eab8d350 0x5608eafbf9f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5608eab6c670 0x5608eafbbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f5122ffd640 0 client.16284 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5608eab64490 0x5608eaf44a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 --2- 172.21.15.31:0/270858472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5608eab4bcd0 0x5608eaf444e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 >> 172.21.15.31:0/270858472 conn(0x5608eab81740 msgr2=0x5608eaf39540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 shutdown_connections 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 1 -- 172.21.15.31:0/270858472 wait complete. 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.649+0000 7f51500d9740 20 client.16284 trim_cache size 0 max 16384 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:15:08.653 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "266bf898-ce45-49ba-ac95-5434c34c696c", 2024-09-21T06:15:08.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:15:08.654 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f512c0b66b0 msgr2=0x7f512c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f512c0b66b0 0x7f512c0b8bc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51380096c0 tx=0x7f5138031400 comp rx=0 tx=0).stop 2024-09-21T06:15:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 msgr2=0x7f514419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 0x7f514419bcf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f513c00eb10 tx=0x7f513c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 shutdown_connections 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f512c0b66b0 0x7f512c0b8bc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5144108cf0 0x7f514419bcf0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51441083a0 0x7f514419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 --2- 172.21.15.31:0/3302309223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5144102a60 0x7f514419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 >> 172.21.15.31:0/3302309223 conn(0x7f5144079730 msgr2=0x7f51441043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 shutdown_connections 2024-09-21T06:15:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:08.650+0000 7f5149b65640 1 -- 172.21.15.31:0/3302309223 wait complete. 2024-09-21T06:15:09.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:09 smithi031 ceph-mon[31237]: pgmap v426: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.8 MiB/s wr, 743 op/s 2024-09-21T06:15:09.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:09 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:15:09.514 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:09 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:09.515 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:09 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:15:09.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:15:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:09 smithi110 ceph-mon[34550]: pgmap v426: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.8 MiB/s wr, 743 op/s 2024-09-21T06:15:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:09 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:15:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:09 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:09 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:15:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:09 smithi107 ceph-mon[34380]: pgmap v426: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 5.8 MiB/s wr, 743 op/s 2024-09-21T06:15:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:09 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:15:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:09 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:09 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:15:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 210.0 s, 315.2 tps, lat 320.967 ms stddev 198.108 2024-09-21T06:15:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 -- 172.21.15.31:0/4021315600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 msgr2=0x7f565c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 --2- 172.21.15.31:0/4021315600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c10d850 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5650009900 tx=0x7f565002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 -- 172.21.15.31:0/4021315600 shutdown_connections 2024-09-21T06:15:10.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 --2- 172.21.15.31:0/4021315600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c10de10 0x7f565c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 --2- 172.21.15.31:0/4021315600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c10d850 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 --2- 172.21.15.31:0/4021315600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c1049b0 0x7f565c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 -- 172.21.15.31:0/4021315600 >> 172.21.15.31:0/4021315600 conn(0x7f565c079730 msgr2=0x7f565c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 -- 172.21.15.31:0/4021315600 shutdown_connections 2024-09-21T06:15:10.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.379+0000 7f566233e640 1 -- 172.21.15.31:0/4021315600 wait complete. 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.380+0000 7f566233e640 1 Processor -- start 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.380+0000 7f566233e640 1 -- start start 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.380+0000 7f566233e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c1049b0 0x7f565c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.380+0000 7f566233e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f566233e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c10de10 0x7f565c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f566233e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f565c113a70 con 0x7f565c105300 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f566233e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1138f0 con 0x7f565c1049b0 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f566233e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f565c113bf0 con 0x7f565c10de10 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c1049b0 0x7f565c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c1049b0 0x7f565c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38500/0 (socket says 172.21.15.31:38500) 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565bfff640 1 -- 172.21.15.31:0/588957731 learned_addr learned my addr 172.21.15.31:0/588957731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f56608db640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c10de10 0x7f565c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 -- 172.21.15.31:0/588957731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c10de10 msgr2=0x7f565c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c10de10 0x7f565c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 -- 172.21.15.31:0/588957731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c1049b0 msgr2=0x7f565c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c1049b0 0x7f565c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 -- 172.21.15.31:0/588957731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c1a0470 con 0x7f565c105300 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f56608db640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c10de10 0x7f565c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.381+0000 7f565b7fe640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c19aaa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f565002a930 tx=0x7f5650031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.382+0000 7f56597fa640 1 -- 172.21.15.31:0/588957731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565003f070 con 0x7f565c105300 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.382+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c1a0700 con 0x7f565c105300 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.382+0000 7f56597fa640 1 -- 172.21.15.31:0/588957731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5650004490 con 0x7f565c105300 2024-09-21T06:15:10.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.382+0000 7f56597fa640 1 -- 172.21.15.31:0/588957731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650035070 con 0x7f565c105300 2024-09-21T06:15:10.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.382+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c1025d0 con 0x7f565c105300 2024-09-21T06:15:10.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.385+0000 7f56688b2740 1 Processor -- start 2024-09-21T06:15:10.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.385+0000 7f56688b2740 1 -- start start 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56597fa640 1 -- 172.21.15.31:0/588957731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5650043400 con 0x7f565c105300 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56688b2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b11060270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56688b2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b110607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56688b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b11060cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56688b2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563b10fd6e50 con 0x563b10c901b0 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56688b2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563b10fc6280 con 0x563b10cb9070 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56688b2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563b10fca090 con 0x563b10c98390 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f565bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b110607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f565bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b110607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45744/0 (socket says 172.21.15.31:45744) 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f565bfff640 1 -- 172.21.15.31:0/2964600405 learned_addr learned my addr 172.21.15.31:0/2964600405 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f56608db640 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b11060270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.386+0000 7f565b7fe640 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b11060cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f565bfff640 1 -- 172.21.15.31:0/2964600405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 msgr2=0x563b11060cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f565bfff640 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b11060cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f565bfff640 1 -- 172.21.15.31:0/2964600405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 msgr2=0x563b11060270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f565bfff640 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b11060270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f565bfff640 1 -- 172.21.15.31:0/2964600405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b10d24650 con 0x563b10c901b0 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f56608db640 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b11060270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:10.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f565bfff640 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b110607b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f564c009d60 tx=0x7f564c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae4accfaa8a7c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f56597fa640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56440b6680 0x7f56440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f56608db640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56440b6680 0x7f56440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f56427fc640 1 -- 172.21.15.31:0/2964600405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c004230 con 0x563b10c901b0 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.387+0000 7f56597fa640 1 -- 172.21.15.31:0/588957731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f5650107b20 con 0x7f565c105300 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.388+0000 7f56427fc640 1 -- 172.21.15.31:0/2964600405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f564c0043d0 con 0x563b10c901b0 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.388+0000 7f56427fc640 1 -- 172.21.15.31:0/2964600405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c019880 con 0x563b10c901b0 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 -- 172.21.15.31:0/2964600405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 msgr2=0x563b110607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56608db640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56440b6680 0x7f56440b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5654009fd0 tx=0x7f5654009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b110607b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f564c009d60 tx=0x7f564c011810 comp rx=0 tx=0).stop 2024-09-21T06:15:10.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 -- 172.21.15.31:0/2964600405 shutdown_connections 2024-09-21T06:15:10.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b11060cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b110607b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 --2- 172.21.15.31:0/2964600405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b11060270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 -- 172.21.15.31:0/2964600405 >> 172.21.15.31:0/2964600405 conn(0x563b10cad460 msgr2=0x563b1105e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.389+0000 7f56688b2740 1 -- 172.21.15.31:0/2964600405 shutdown_connections 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.391+0000 7f56688b2740 1 -- 172.21.15.31:0/2964600405 wait complete. 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.391+0000 7f56688b2740 1 -- start start 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.391+0000 7f56688b2740 1 Processor -- start 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b1106b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b1106be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b1106c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563b10e27700 con 0x563b10c901b0 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b1106c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f565bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b1106b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b1106be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b1106c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38504/0 (socket says 172.21.15.31:38504) 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 -- 172.21.15.31:0/1333751606 learned_addr learned my addr 172.21.15.31:0/1333751606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f565bfff640 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b1106b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45758/0 (socket says 172.21.15.31:45758) 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563b10f61110 con 0x563b10cb9070 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563b1106eaf0 con 0x563b10c98390 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56688b2740 10 client.0 _ll_register_callbacks cb 0x563b10f7a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 -- 172.21.15.31:0/1333751606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 msgr2=0x563b1106be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b1106be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 -- 172.21.15.31:0/1333751606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 msgr2=0x563b1106b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b1106b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.392+0000 7f56608db640 1 -- 172.21.15.31:0/1333751606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b10f8ec40 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f56608db640 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b1106c3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5654012910 tx=0x7f5654004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=752fe623629e7136 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f563b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f563b7fe640 1 -- 172.21.15.31:0/1333751606 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654011c90 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f563b7fe640 1 -- 172.21.15.31:0/1333751606 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f565401f420 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c0043d0 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f563b7fe640 1 -- 172.21.15.31:0/1333751606 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5654011e30 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f56688b2740 10 client.26021 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563b1107a7f0 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.393+0000 7f563b7fe640 1 -- 172.21.15.31:0/1333751606 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f564c0043d0 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f563b7fe640 1 -- 172.21.15.31:0/1333751606 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f5654027070 con 0x563b10cb9070 2024-09-21T06:15:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f56688b2740 10 client.26021 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f56688b2740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f56688b2740 20 client.26021 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f56688b2740 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10c779f0 0x563b1107de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f56688b2740 4 client.26021 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b10fd8da0 con 0x563b10c779f0 2024-09-21T06:15:10.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.394+0000 7f565bfff640 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10c779f0 0x563b1107de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.395+0000 7f563b7fe640 10 client.26021 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.395+0000 7f565bfff640 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10c779f0 0x563b1107de10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd2e28bd4e34951d server_cookie=a000e68eb6404201 in_seq=0 out_seq=0 2024-09-21T06:15:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.395+0000 7f563b7fe640 1 -- 172.21.15.31:0/1333751606 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b10fd8da0 con 0x563b10c779f0 2024-09-21T06:15:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.395+0000 7f563b7fe640 10 client.26021 handle_command_reply: tid=0 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.395+0000 7f56688b2740 1 client.26021 shutdown 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 msgr2=0x563b1106c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b1106c3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5654012910 tx=0x7f5654004ab0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 shutdown_connections 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10c779f0 0x563b1107de10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10cb9070 0x563b1106c3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f563b7fe640 0 client.26021 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c98390 0x563b1106be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 --2- 172.21.15.31:0/1333751606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c901b0 0x563b1106b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 >> 172.21.15.31:0/1333751606 conn(0x563b10cad460 msgr2=0x563b110612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 shutdown_connections 2024-09-21T06:15:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 1 -- 172.21.15.31:0/1333751606 wait complete. 2024-09-21T06:15:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.396+0000 7f56688b2740 20 client.26021 trim_cache size 0 max 16384 2024-09-21T06:15:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.415+0000 7f56688b2740 1 Processor -- start 2024-09-21T06:15:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.415+0000 7f56688b2740 1 -- start start 2024-09-21T06:15:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.415+0000 7f56688b2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c779f0 0x563b110619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56688b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1105dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56688b2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c98390 0x563b1105e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56688b2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1ab4d0 con 0x563b10c98390 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56688b2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1add50 con 0x563b10c779f0 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56688b2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1abb40 con 0x563b10c901b0 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1105dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56608db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c98390 0x563b1105e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1105dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50200/0 (socket says 172.21.15.31:50200) 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 -- 172.21.15.31:0/3009265627 learned_addr learned my addr 172.21.15.31:0/3009265627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565bfff640 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c779f0 0x563b110619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 -- 172.21.15.31:0/3009265627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c779f0 msgr2=0x563b110619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c779f0 0x563b110619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 -- 172.21.15.31:0/3009265627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c98390 msgr2=0x563b1105e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c98390 0x563b1105e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 -- 172.21.15.31:0/3009265627 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1106a860 con 0x563b10c901b0 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f56608db640 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c98390 0x563b1105e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.416+0000 7f565b7fe640 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1105dac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56500ca030 tx=0x7f565002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c394c9f5ac612cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f5641ffb640 1 -- 172.21.15.31:0/3009265627 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650048040 con 0x563b10c901b0 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f5641ffb640 1 -- 172.21.15.31:0/3009265627 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565010c040 con 0x563b10c901b0 2024-09-21T06:15:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f5641ffb640 1 -- 172.21.15.31:0/3009265627 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500cd1d0 con 0x563b10c901b0 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f56688b2740 1 -- 172.21.15.31:0/3009265627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 msgr2=0x563b1105dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f56688b2740 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1105dac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f56500ca030 tx=0x7f565002a410 comp rx=0 tx=0).stop 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f56688b2740 1 -- 172.21.15.31:0/3009265627 shutdown_connections 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f56688b2740 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c98390 0x563b1105e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f56688b2740 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1105dac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.417+0000 7f56688b2740 1 --2- 172.21.15.31:0/3009265627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c779f0 0x563b110619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.418+0000 7f56688b2740 1 -- 172.21.15.31:0/3009265627 >> 172.21.15.31:0/3009265627 conn(0x563b10cad460 msgr2=0x563b110612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.418+0000 7f56688b2740 1 -- 172.21.15.31:0/3009265627 shutdown_connections 2024-09-21T06:15:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.418+0000 7f56688b2740 1 -- 172.21.15.31:0/3009265627 wait complete. 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.418+0000 7f56688b2740 1 -- start start 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.418+0000 7f56688b2740 1 Processor -- start 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c779f0 0x563b11074720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1106fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c98390 0x563b11070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1ac210 con 0x563b10c779f0 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1ad680 con 0x563b10c98390 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1ac8e0 con 0x563b10c901b0 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56688b2740 10 client.0 _ll_register_callbacks cb 0x7f5654012840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56608db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c98390 0x563b11070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1106fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c779f0 0x563b11074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56608db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c98390 0x563b11070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38514/0 (socket says 172.21.15.31:38514) 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f56608db640 1 -- 172.21.15.31:0/2895055213 learned_addr learned my addr 172.21.15.31:0/2895055213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565b7fe640 1 -- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c98390 msgr2=0x563b11070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565b7fe640 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c98390 0x563b11070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565b7fe640 1 -- 172.21.15.31:0/2895055213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c779f0 msgr2=0x563b11074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565b7fe640 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c779f0 0x563b11074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.419+0000 7f565b7fe640 1 -- 172.21.15.31:0/2895055213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b110e7810 con 0x563b10c901b0 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f565b7fe640 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1106fc10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x563b11058490 tx=0x7f56501057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6714d521754079f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f563b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f563b7fe640 1 -- 172.21.15.31:0/2895055213 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500c91d0 con 0x563b10c901b0 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f563b7fe640 1 -- 172.21.15.31:0/2895055213 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f56500d7140 con 0x563b10c901b0 2024-09-21T06:15:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565010c040 con 0x563b10c901b0 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f563b7fe640 1 -- 172.21.15.31:0/2895055213 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f56500cd1d0 con 0x563b10c901b0 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f56688b2740 10 client.26056 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563b110e8970 con 0x563b10c901b0 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.420+0000 7f563b7fe640 1 -- 172.21.15.31:0/2895055213 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565010c040 con 0x563b10c901b0 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f563b7fe640 1 -- 172.21.15.31:0/2895055213 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f56500cece0 con 0x563b10c901b0 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f56688b2740 10 client.26056 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f56688b2740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f56688b2740 20 client.26056 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f56688b2740 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10cb9070 0x563b110eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f56688b2740 4 client.26056 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b10fd4850 con 0x563b10cb9070 2024-09-21T06:15:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.421+0000 7f565bfff640 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10cb9070 0x563b110eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f565bfff640 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10cb9070 0x563b110eaeb0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7076592291d42713 server_cookie=774da673b8a13c27 in_seq=0 out_seq=0 2024-09-21T06:15:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f563b7fe640 10 client.26056 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f563b7fe640 1 -- 172.21.15.31:0/2895055213 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x563b10fd4850 con 0x563b10cb9070 2024-09-21T06:15:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f563b7fe640 10 client.26056 handle_command_reply: tid=0 2024-09-21T06:15:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 client.26056 shutdown 2024-09-21T06:15:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 msgr2=0x563b1106fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1106fc10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x563b11058490 tx=0x7f56501057e0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 shutdown_connections 2024-09-21T06:15:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563b10cb9070 0x563b110eaeb0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563b10c98390 0x563b11070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563b10c901b0 0x563b1106fc10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 --2- 172.21.15.31:0/2895055213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563b10c779f0 0x563b11074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.422+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 >> 172.21.15.31:0/2895055213 conn(0x563b10cad460 msgr2=0x563b11076cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f563b7fe640 0 client.26056 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 shutdown_connections 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f56688b2740 1 -- 172.21.15.31:0/2895055213 wait complete. 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f56688b2740 20 client.26056 trim_cache size 0 max 16384 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "266bf898-ce45-49ba-ac95-5434c34c696c", 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:15:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:15:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56440b6680 msgr2=0x7f56440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f566233e640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56440b6680 0x7f56440b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5654009fd0 tx=0x7f5654009500 comp rx=0 tx=0).stop 2024-09-21T06:15:10.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 msgr2=0x7f565c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.423+0000 7f566233e640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c19aaa0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f565002a930 tx=0x7f5650031a20 comp rx=0 tx=0).stop 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 shutdown_connections 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f565c10de10 0x7f565c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f56440b6680 0x7f56440b8b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f565c105300 0x7f565c19aaa0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 --2- 172.21.15.31:0/588957731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565c1049b0 0x7f565c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 >> 172.21.15.31:0/588957731 conn(0x7f565c079730 msgr2=0x7f565c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 shutdown_connections 2024-09-21T06:15:10.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:10.424+0000 7f566233e640 1 -- 172.21.15.31:0/588957731 wait complete. 2024-09-21T06:15:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:10 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:10 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:10.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:10 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:11.202 INFO:tasks.cephfs.filesystem:scrub status for tag:266bf898-ce45-49ba-ac95-5434c34c696c - {'path': '/', 'tag': '266bf898-ce45-49ba-ac95-5434c34c696c', 'options': 'recursive,force'} 2024-09-21T06:15:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:11 smithi110 ceph-mon[34550]: pgmap v427: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.0 MiB/s wr, 700 op/s 2024-09-21T06:15:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:11 smithi107 ceph-mon[34380]: pgmap v427: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.0 MiB/s wr, 700 op/s 2024-09-21T06:15:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:11 smithi031 ceph-mon[31237]: pgmap v427: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 5.0 MiB/s wr, 700 op/s 2024-09-21T06:15:12.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:12 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:15:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:12 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:15:12.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:12 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:15:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:13 smithi110 ceph-mon[34550]: pgmap v428: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.2 MiB/s wr, 912 op/s 2024-09-21T06:15:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:13 smithi107 ceph-mon[34380]: pgmap v428: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.2 MiB/s wr, 912 op/s 2024-09-21T06:15:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:13 smithi031 ceph-mon[31237]: pgmap v428: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 6.2 MiB/s wr, 912 op/s 2024-09-21T06:15:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 215.0 s, 841.2 tps, lat 122.674 ms stddev 562.093 2024-09-21T06:15:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:15 smithi110 ceph-mon[34550]: pgmap v429: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.8 MiB/s wr, 737 op/s 2024-09-21T06:15:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:15 smithi107 ceph-mon[34380]: pgmap v429: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.8 MiB/s wr, 737 op/s 2024-09-21T06:15:15.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:15 smithi031 ceph-mon[31237]: pgmap v429: 97 pgs: 97 active+clean; 3.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.8 MiB/s wr, 737 op/s 2024-09-21T06:15:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:17 smithi110 ceph-mon[34550]: pgmap v430: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2024-09-21T06:15:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:17 smithi107 ceph-mon[34380]: pgmap v430: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2024-09-21T06:15:17.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:17 smithi031 ceph-mon[31237]: pgmap v430: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 26 MiB/s wr, 1.64k op/s 2024-09-21T06:15:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:19 smithi110 ceph-mon[34550]: pgmap v431: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 31 MiB/s wr, 1.65k op/s 2024-09-21T06:15:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:19 smithi107 ceph-mon[34380]: pgmap v431: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 31 MiB/s wr, 1.65k op/s 2024-09-21T06:15:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:19 smithi031 ceph-mon[31237]: pgmap v431: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 31 MiB/s wr, 1.65k op/s 2024-09-21T06:15:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 220.0 s, 4078.4 tps, lat 24.547 ms stddev 28.388 2024-09-21T06:15:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.694+0000 7f75d9590640 1 -- 172.21.15.31:0/1716806223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4118900 msgr2=0x7f75d411ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.694+0000 7f75d9590640 1 --2- 172.21.15.31:0/1716806223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4118900 0x7f75d411ad20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc02a400 comp rx=0 tx=0).stop 2024-09-21T06:15:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.696+0000 7f75d9590640 1 -- 172.21.15.31:0/1716806223 shutdown_connections 2024-09-21T06:15:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.696+0000 7f75d9590640 1 --2- 172.21.15.31:0/1716806223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d4118900 0x7f75d411ad20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.696+0000 7f75d9590640 1 --2- 172.21.15.31:0/1716806223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d410ad10 0x7f75d41183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.696+0000 7f75d9590640 1 --2- 172.21.15.31:0/1716806223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410a3c0 0x7f75d410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.696+0000 7f75d9590640 1 -- 172.21.15.31:0/1716806223 >> 172.21.15.31:0/1716806223 conn(0x7f75d4070dd0 msgr2=0x7f75d40711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.697+0000 7f75d9590640 1 -- 172.21.15.31:0/1716806223 shutdown_connections 2024-09-21T06:15:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.697+0000 7f75d9590640 1 -- 172.21.15.31:0/1716806223 wait complete. 2024-09-21T06:15:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 Processor -- start 2024-09-21T06:15:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 -- start start 2024-09-21T06:15:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d410a3c0 0x7f75d41a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 0x7f75d41a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d4118900 0x7f75d41aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75d411e450 con 0x7f75d410ad10 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75d411e2d0 con 0x7f75d4118900 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75d9590640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75d411e5d0 con 0x7f75d410a3c0 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.698+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 0x7f75d41a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 0x7f75d41a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35780/0 (socket says 172.21.15.31:35780) 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1498682796 learned_addr learned my addr 172.21.15.31:0/1498682796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d410a3c0 0x7f75d41a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75d37fe640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d4118900 0x7f75d41aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1498682796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d410a3c0 msgr2=0x7f75d41a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d410a3c0 0x7f75d41a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1498682796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d4118900 msgr2=0x7f75d41aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d4118900 0x7f75d41aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1498682796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41aad60 con 0x7f75d410ad10 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75d37fe640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d4118900 0x7f75d41aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.699+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 0x7f75d41a53d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00dbd0 tx=0x7f75bc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.700+0000 7f75d0ff9640 1 -- 172.21.15.31:0/1498682796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc017070 con 0x7f75d410ad10 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.700+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d41ab050 con 0x7f75d410ad10 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.700+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41b17e0 con 0x7f75d410ad10 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.700+0000 7f75d0ff9640 1 -- 172.21.15.31:0/1498682796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75bc010b10 con 0x7f75d410ad10 2024-09-21T06:15:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.700+0000 7f75d0ff9640 1 -- 172.21.15.31:0/1498682796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00fa00 con 0x7f75d410ad10 2024-09-21T06:15:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.704+0000 7f75e05dc740 1 Processor -- start 2024-09-21T06:15:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.704+0000 7f75e05dc740 1 -- start start 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d5221c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d5221cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e564d0 0x557d5221d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75e05dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557d521916f0 con 0x557d51e771b0 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75e05dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d52180b20 con 0x557d51e4e2f0 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75e05dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557d52184930 con 0x557d51e564d0 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d5221cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75d2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d5221cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51964/0 (socket says 172.21.15.31:51964) 2024-09-21T06:15:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d5221c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.705+0000 7f75d2ffd640 1 -- 172.21.15.31:0/1223920443 learned_addr learned my addr 172.21.15.31:0/1223920443 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75d2ffd640 1 -- 172.21.15.31:0/1223920443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e564d0 msgr2=0x557d5221d430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e564d0 0x557d5221d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75d2ffd640 1 -- 172.21.15.31:0/1223920443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 msgr2=0x557d5221c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d5221c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75d2ffd640 1 -- 172.21.15.31:0/1223920443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d52146530 con 0x557d51e4e2f0 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d5221cef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009cf0 tx=0x7f75c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36421b198335d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75c97fa640 1 -- 172.21.15.31:0/1223920443 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004230 con 0x557d51e4e2f0 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75c97fa640 1 -- 172.21.15.31:0/1223920443 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c40043d0 con 0x557d51e4e2f0 2024-09-21T06:15:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.706+0000 7f75c97fa640 1 -- 172.21.15.31:0/1223920443 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4019840 con 0x557d51e4e2f0 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 -- 172.21.15.31:0/1223920443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 msgr2=0x557d5221cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d5221cef0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009cf0 tx=0x7f75c4011810 comp rx=0 tx=0).stop 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 -- 172.21.15.31:0/1223920443 shutdown_connections 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e564d0 0x557d5221d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d5221cef0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1223920443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d5221c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 -- 172.21.15.31:0/1223920443 >> 172.21.15.31:0/1223920443 conn(0x557d51e6b300 msgr2=0x557d5221a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75d0ff9640 1 -- 172.21.15.31:0/1498682796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f75bc010cc0 con 0x7f75d410ad10 2024-09-21T06:15:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 -- 172.21.15.31:0/1223920443 shutdown_connections 2024-09-21T06:15:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.708+0000 7f75e05dc740 1 -- 172.21.15.31:0/1223920443 wait complete. 2024-09-21T06:15:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.709+0000 7f75e05dc740 1 -- start start 2024-09-21T06:15:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.709+0000 7f75e05dc740 1 Processor -- start 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d522308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e771b0 0x557d52230e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e35d80 0x557d5223ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557d520b8ab0 con 0x557d51e35d80 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557d52126730 con 0x557d51e4e2f0 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557d522338f0 con 0x557d51e771b0 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75e05dc740 10 client.0 _ll_register_callbacks cb 0x557d5217a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d522308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e35d80 0x557d5223ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d522308c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51972/0 (socket says 172.21.15.31:51972) 2024-09-21T06:15:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75d37fe640 1 -- 172.21.15.31:0/778343114 learned_addr learned my addr 172.21.15.31:0/778343114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.710+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e771b0 0x557d52230e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d0ff9640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75b40b6610 0x7f75b40b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d0ff9640 1 -- 172.21.15.31:0/1498682796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f75bc0e7400 con 0x7f75d410ad10 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d37fe640 1 -- 172.21.15.31:0/778343114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e771b0 msgr2=0x557d52230e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d37fe640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e771b0 0x557d52230e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d37fe640 1 -- 172.21.15.31:0/778343114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e35d80 msgr2=0x557d5223ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d37fe640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e35d80 0x557d5223ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d37fe640 1 -- 172.21.15.31:0/778343114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d51ee2790 con 0x557d51e4e2f0 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.711+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e771b0 0x557d52230e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75d37fe640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d522308c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831442e9421caff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75d37fe640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75b40b6610 0x7f75b40b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75c27fc640 1 -- 172.21.15.31:0/778343114 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc035040 con 0x557d51e4e2f0 2024-09-21T06:15:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75c27fc640 1 -- 172.21.15.31:0/778343114 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f75cc003e40 con 0x557d51e4e2f0 2024-09-21T06:15:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c40043d0 con 0x557d51e4e2f0 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75c27fc640 1 -- 172.21.15.31:0/778343114 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f75cc046440 con 0x557d51e4e2f0 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75e05dc740 10 client.26039 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.712+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557d51f5b620 con 0x557d51e4e2f0 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75d37fe640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75b40b6610 0x7f75b40b8b20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0096c0 tx=0x7f75cc052000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75c27fc640 1 -- 172.21.15.31:0/778343114 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c40043d0 con 0x557d51e4e2f0 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75c27fc640 1 -- 172.21.15.31:0/778343114 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f75cc044070 con 0x557d51e4e2f0 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75e05dc740 10 client.26039 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75e05dc740 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75e05dc740 20 client.26039 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75e05dc740 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d51e0b1c0 0x557d52242b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75e05dc740 4 client.26039 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d52193640 con 0x557d51e0b1c0 2024-09-21T06:15:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.713+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d51e0b1c0 0x557d52242b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.714+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d51e0b1c0 0x557d52242b10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a76ad5a41058396 server_cookie=1584a76a1a03fa5d in_seq=0 out_seq=0 2024-09-21T06:15:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.714+0000 7f75c27fc640 10 client.26039 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.715+0000 7f75c27fc640 1 -- 172.21.15.31:0/778343114 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d52193640 con 0x557d51e0b1c0 2024-09-21T06:15:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.715+0000 7f75c27fc640 10 client.26039 handle_command_reply: tid=0 2024-09-21T06:15:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 client.26039 shutdown 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 msgr2=0x557d522308c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d522308c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc003820 comp rx=0 tx=0).stop 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 shutdown_connections 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d51e0b1c0 0x557d52242b10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e35d80 0x557d5223ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e771b0 0x557d52230e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 --2- 172.21.15.31:0/778343114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d522308c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 >> 172.21.15.31:0/778343114 conn(0x557d51e6b300 msgr2=0x557d5221fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.716+0000 7f75c27fc640 0 client.26039 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.717+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 shutdown_connections 2024-09-21T06:15:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.717+0000 7f75e05dc740 1 -- 172.21.15.31:0/778343114 wait complete. 2024-09-21T06:15:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.717+0000 7f75e05dc740 20 client.26039 trim_cache size 0 max 16384 2024-09-21T06:15:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.745+0000 7f75e05dc740 1 Processor -- start 2024-09-21T06:15:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.745+0000 7f75e05dc740 1 -- start start 2024-09-21T06:15:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e35d80 0x557d5221fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 0x557d52220080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52221a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75e05dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800c7d0 con 0x557d51e771b0 2024-09-21T06:15:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75e05dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800f050 con 0x557d51e35d80 2024-09-21T06:15:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75e05dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800ce40 con 0x557d51e4e2f0 2024-09-21T06:15:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 0x557d52220080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 0x557d52220080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:46898/0 (socket says 172.21.15.31:46898) 2024-09-21T06:15:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1977928727 learned_addr learned my addr 172.21.15.31:0/1977928727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75d37fe640 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52221a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.746+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e35d80 0x557d5221fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1977928727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e35d80 msgr2=0x557d5221fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e35d80 0x557d5221fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1977928727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 msgr2=0x557d52221a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52221a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75ca7fc640 1 -- 172.21.15.31:0/1977928727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5222eda0 con 0x557d51e4e2f0 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e35d80 0x557d5221fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.747+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 0x557d52220080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0a9910 tx=0x7f75bc00f5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42bc7099b81ab2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.748+0000 7f75c8ff9640 1 -- 172.21.15.31:0/1977928727 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc020040 con 0x557d51e4e2f0 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.748+0000 7f75c8ff9640 1 -- 172.21.15.31:0/1977928727 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75bc0ace50 con 0x557d51e4e2f0 2024-09-21T06:15:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.748+0000 7f75c8ff9640 1 -- 172.21.15.31:0/1977928727 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc0ac800 con 0x557d51e4e2f0 2024-09-21T06:15:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 -- 172.21.15.31:0/1977928727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 msgr2=0x557d52220080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 0x557d52220080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0a9910 tx=0x7f75bc00f5e0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 -- 172.21.15.31:0/1977928727 shutdown_connections 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52221a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e4e2f0 0x557d52220080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 --2- 172.21.15.31:0/1977928727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e35d80 0x557d5221fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 -- 172.21.15.31:0/1977928727 >> 172.21.15.31:0/1977928727 conn(0x557d51e6b300 msgr2=0x557d5222f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 -- 172.21.15.31:0/1977928727 shutdown_connections 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.749+0000 7f75e05dc740 1 -- 172.21.15.31:0/1977928727 wait complete. 2024-09-21T06:15:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.750+0000 7f75e05dc740 1 -- start start 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.750+0000 7f75e05dc740 1 Processor -- start 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e35d80 0x557d52247930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d52247ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52237c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800d510 con 0x557d51e771b0 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800e980 con 0x557d51e4e2f0 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75b800dbe0 con 0x557d51e35d80 2024-09-21T06:15:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75e05dc740 10 client.0 _ll_register_callbacks cb 0x7f75cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52237c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52237c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35816/0 (socket says 172.21.15.31:35816) 2024-09-21T06:15:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.751+0000 7f75d37fe640 1 -- 172.21.15.31:0/3194583149 learned_addr learned my addr 172.21.15.31:0/3194583149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d52247ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e35d80 0x557d52247930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d37fe640 1 -- 172.21.15.31:0/3194583149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e35d80 msgr2=0x557d52247930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d37fe640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e35d80 0x557d52247930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d37fe640 1 -- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 msgr2=0x557d52247ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d37fe640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d52247ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d37fe640 1 -- 172.21.15.31:0/3194583149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d52238150 con 0x557d51e771b0 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75ca7fc640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d52247ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e35d80 0x557d52247930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.752+0000 7f75d37fe640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52237c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89723a2b2f0873f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.753+0000 7f75c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:15:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.753+0000 7f75c27fc640 1 -- 172.21.15.31:0/3194583149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc035040 con 0x557d51e771b0 2024-09-21T06:15:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.753+0000 7f75c27fc640 1 -- 172.21.15.31:0/3194583149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f75cc003db0 con 0x557d51e771b0 2024-09-21T06:15:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.754+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75bc0ace50 con 0x557d51e771b0 2024-09-21T06:15:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.754+0000 7f75c27fc640 1 -- 172.21.15.31:0/3194583149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f75cc0044a0 con 0x557d51e771b0 2024-09-21T06:15:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.754+0000 7f75e05dc740 10 client.16320 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.754+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557d522393e0 con 0x557d51e771b0 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75c27fc640 1 -- 172.21.15.31:0/3194583149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75bc0ace50 con 0x557d51e771b0 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75c27fc640 1 -- 172.21.15.31:0/3194583149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f75cc044040 con 0x557d51e771b0 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75e05dc740 10 client.16320 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75e05dc740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75e05dc740 20 client.16320 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75e05dc740 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d522ad510 0x557d522af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75e05dc740 4 client.16320 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.755+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d5218f0f0 con 0x557d522ad510 2024-09-21T06:15:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.756+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d522ad510 0x557d522af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.757+0000 7f75c27fc640 10 client.16320 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.757+0000 7f75d2ffd640 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d522ad510 0x557d522af930 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297b4be095787e3e server_cookie=cb515585e449abc7 in_seq=0 out_seq=0 2024-09-21T06:15:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:21 smithi110 ceph-mon[34550]: pgmap v432: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 MiB/s wr, 1.71k op/s 2024-09-21T06:15:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75c27fc640 1 -- 172.21.15.31:0/3194583149 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d5218f0f0 con 0x557d522ad510 2024-09-21T06:15:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75c27fc640 10 client.16320 handle_command_reply: tid=0 2024-09-21T06:15:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 client.16320 shutdown 2024-09-21T06:15:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 msgr2=0x557d52237c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52237c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc005850 comp rx=0 tx=0).stop 2024-09-21T06:15:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 shutdown_connections 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557d522ad510 0x557d522af930 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d51e771b0 0x557d52237c10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557d51e4e2f0 0x557d52247ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 --2- 172.21.15.31:0/3194583149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d51e35d80 0x557d52247930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 >> 172.21.15.31:0/3194583149 conn(0x557d51e6b300 msgr2=0x557d5222efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75c27fc640 0 client.16320 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.778+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 shutdown_connections 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.779+0000 7f75e05dc740 1 -- 172.21.15.31:0/3194583149 wait complete. 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.779+0000 7f75e05dc740 20 client.16320 trim_cache size 0 max 16384 2024-09-21T06:15:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:15:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:15:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:15:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75b40b6610 msgr2=0x7f75b40b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75b40b6610 0x7f75b40b8b20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0096c0 tx=0x7f75cc052000 comp rx=0 tx=0).stop 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 msgr2=0x7f75d41a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 0x7f75d41a53d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00dbd0 tx=0x7f75bc00b630 comp rx=0 tx=0).stop 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 shutdown_connections 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75d4118900 0x7f75d41aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75b40b6610 0x7f75b40b8b20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75d410ad10 0x7f75d41a53d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 --2- 172.21.15.31:0/1498682796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75d410a3c0 0x7f75d41a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.780+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 >> 172.21.15.31:0/1498682796 conn(0x7f75d4070dd0 msgr2=0x7f75d4118d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.781+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 shutdown_connections 2024-09-21T06:15:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:21.781+0000 7f75d9590640 1 -- 172.21.15.31:0/1498682796 wait complete. 2024-09-21T06:15:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:21 smithi107 ceph-mon[34380]: pgmap v432: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 MiB/s wr, 1.71k op/s 2024-09-21T06:15:21.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:21 smithi031 ceph-mon[31237]: pgmap v432: 97 pgs: 97 active+clean; 3.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 32 MiB/s wr, 1.71k op/s 2024-09-21T06:15:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:23 smithi110 ceph-mon[34550]: pgmap v433: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 35 MiB/s wr, 1.92k op/s 2024-09-21T06:15:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:23 smithi107 ceph-mon[34380]: pgmap v433: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 35 MiB/s wr, 1.92k op/s 2024-09-21T06:15:23.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:23 smithi031 ceph-mon[31237]: pgmap v433: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 35 MiB/s wr, 1.92k op/s 2024-09-21T06:15:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 225.0 s, 3220.2 tps, lat 31.079 ms stddev 33.259 2024-09-21T06:15:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:25 smithi110 ceph-mon[34550]: pgmap v434: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-09-21T06:15:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:25 smithi107 ceph-mon[34380]: pgmap v434: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-09-21T06:15:25.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:25 smithi031 ceph-mon[31237]: pgmap v434: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-09-21T06:15:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:27 smithi110 ceph-mon[34550]: pgmap v435: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 1.81k op/s 2024-09-21T06:15:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:27 smithi107 ceph-mon[34380]: pgmap v435: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 1.81k op/s 2024-09-21T06:15:27.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:27 smithi031 ceph-mon[31237]: pgmap v435: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 MiB/s wr, 1.81k op/s 2024-09-21T06:15:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:29 smithi110 ceph-mon[34550]: pgmap v436: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 939 op/s 2024-09-21T06:15:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:29 smithi107 ceph-mon[34380]: pgmap v436: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 939 op/s 2024-09-21T06:15:29.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:29 smithi031 ceph-mon[31237]: pgmap v436: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 939 op/s 2024-09-21T06:15:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 230.0 s, 3969.4 tps, lat 25.258 ms stddev 26.943 2024-09-21T06:15:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:31 smithi110 ceph-mon[34550]: pgmap v437: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 759 op/s 2024-09-21T06:15:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:31 smithi107 ceph-mon[34380]: pgmap v437: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 759 op/s 2024-09-21T06:15:31.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:31 smithi031 ceph-mon[31237]: pgmap v437: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.4 MiB/s wr, 759 op/s 2024-09-21T06:15:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:33 smithi110 ceph-mon[34550]: pgmap v438: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 880 op/s 2024-09-21T06:15:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:33 smithi107 ceph-mon[34380]: pgmap v438: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 880 op/s 2024-09-21T06:15:33.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:33 smithi031 ceph-mon[31237]: pgmap v438: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.5 MiB/s wr, 880 op/s 2024-09-21T06:15:34.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:34 smithi031 ceph-mon[31237]: pgmap v439: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 612 op/s 2024-09-21T06:15:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.944+0000 7f610ce0a640 1 -- 172.21.15.31:0/168640691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 msgr2=0x7f61081105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.944+0000 7f610ce0a640 1 --2- 172.21.15.31:0/168640691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081105a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6100009ca0 tx=0x7f610002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.945+0000 7f610ce0a640 1 -- 172.21.15.31:0/168640691 shutdown_connections 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.945+0000 7f610ce0a640 1 --2- 172.21.15.31:0/168640691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081105a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.945+0000 7f610ce0a640 1 --2- 172.21.15.31:0/168640691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108109420 0x7f6108109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.945+0000 7f610ce0a640 1 --2- 172.21.15.31:0/168640691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108103af0 0x7f6108103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.945+0000 7f610ce0a640 1 -- 172.21.15.31:0/168640691 >> 172.21.15.31:0/168640691 conn(0x7f61080ff570 msgr2=0x7f61081019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.946+0000 7f610ce0a640 1 -- 172.21.15.31:0/168640691 shutdown_connections 2024-09-21T06:15:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.946+0000 7f610ce0a640 1 -- 172.21.15.31:0/168640691 wait complete. 2024-09-21T06:15:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.947+0000 7f610ce0a640 1 Processor -- start 2024-09-21T06:15:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.947+0000 7f610ce0a640 1 -- start start 2024-09-21T06:15:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610ce0a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 0x7f61081a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610ce0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108109420 0x7f61081a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610ce0a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610ce0a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6108113a80 con 0x7f6108109d70 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 0x7f61081a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610ce0a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6108113900 con 0x7f6108109420 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610ce0a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6108113c00 con 0x7f6108103af0 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 0x7f61081a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40778/0 (socket says 172.21.15.31:40778) 2024-09-21T06:15:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f610659c640 1 -- 172.21.15.31:0/1891012906 learned_addr learned my addr 172.21.15.31:0/1891012906 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:34.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.948+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108109420 0x7f61081a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f6106d9d640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fdd9b640 1 -- 172.21.15.31:0/1891012906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 msgr2=0x7f61081a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 0x7f61081a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fdd9b640 1 -- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 msgr2=0x7f61081ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fdd9b640 1 -- 172.21.15.31:0/1891012906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61081a2520 con 0x7f6108109420 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f6106d9d640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f610659c640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 0x7f61081a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108109420 0x7f61081a1c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f610806c8b0 tx=0x7f60f000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.949+0000 7f60fffff640 1 -- 172.21.15.31:0/1891012906 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f000cc40 con 0x7f6108109420 2024-09-21T06:15:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.950+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61081acdc0 con 0x7f6108109420 2024-09-21T06:15:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.950+0000 7f60fffff640 1 -- 172.21.15.31:0/1891012906 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f000cde0 con 0x7f6108109420 2024-09-21T06:15:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.950+0000 7f60fffff640 1 -- 172.21.15.31:0/1891012906 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0018800 con 0x7f6108109420 2024-09-21T06:15:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.950+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081ad280 con 0x7f6108109420 2024-09-21T06:15:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.953+0000 7f6113e50740 1 Processor -- start 2024-09-21T06:15:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.953+0000 7f6113e50740 1 -- start start 2024-09-21T06:15:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6113e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6113e50740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6113e50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6113e50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b71742e6f0 con 0x55b7170f34d0 2024-09-21T06:15:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6113e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b71741db20 con 0x55b7171141b0 2024-09-21T06:15:34.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6113e50740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b717421930 con 0x55b7170eb2f0 2024-09-21T06:15:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174b9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40780/0 (socket says 172.21.15.31:40780) 2024-09-21T06:15:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f610659c640 1 -- 172.21.15.31:0/14464309 learned_addr learned my addr 172.21.15.31:0/14464309 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.954+0000 7f6106d9d640 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.955+0000 7f6106d9d640 1 -- 172.21.15.31:0/14464309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 msgr2=0x55b7174b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.955+0000 7f6106d9d640 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174b9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.955+0000 7f6106d9d640 1 -- 172.21.15.31:0/14464309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 msgr2=0x55b7174ba430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.955+0000 7f6106d9d640 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174ba430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.955+0000 7f6106d9d640 1 -- 172.21.15.31:0/14464309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7173e3530 con 0x55b7171141b0 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.955+0000 7f6106d9d640 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b99b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f610002aea0 tx=0x7f61000093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55117b94094bd2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.956+0000 7f60fcd99640 1 -- 172.21.15.31:0/14464309 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100030620 con 0x55b7171141b0 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.956+0000 7f60fffff640 1 -- 172.21.15.31:0/1891012906 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f60f00189a0 con 0x7f6108109420 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.956+0000 7f60fcd99640 1 -- 172.21.15.31:0/14464309 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6100040070 con 0x55b7171141b0 2024-09-21T06:15:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.956+0000 7f60fcd99640 1 -- 172.21.15.31:0/14464309 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100004460 con 0x55b7171141b0 2024-09-21T06:15:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.957+0000 7f6113e50740 1 -- 172.21.15.31:0/14464309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 msgr2=0x55b7174b99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.957+0000 7f6113e50740 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b99b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f610002aea0 tx=0x7f61000093a0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.957+0000 7f60fffff640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60f40b6700 0x7f60f40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f6113e50740 1 -- 172.21.15.31:0/14464309 shutdown_connections 2024-09-21T06:15:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f6113e50740 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f60fffff640 1 -- 172.21.15.31:0/1891012906 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60f00e79f0 con 0x7f6108109420 2024-09-21T06:15:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f6113e50740 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f6113e50740 1 --2- 172.21.15.31:0/14464309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b99b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f6113e50740 1 -- 172.21.15.31:0/14464309 >> 172.21.15.31:0/14464309 conn(0x55b717108300 msgr2=0x55b7174b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f610659c640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60f40b6700 0x7f60f40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.958+0000 7f6113e50740 1 -- 172.21.15.31:0/14464309 shutdown_connections 2024-09-21T06:15:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.959+0000 7f6113e50740 1 -- 172.21.15.31:0/14464309 wait complete. 2024-09-21T06:15:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.959+0000 7f610659c640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60f40b6700 0x7f60f40b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55b7174b8630 tx=0x7f60f8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.959+0000 7f6113e50740 1 -- start start 2024-09-21T06:15:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.959+0000 7f6113e50740 1 Processor -- start 2024-09-21T06:15:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7170f34d0 0x55b7174c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7171141b0 0x55b7174d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b717355ab0 con 0x55b7171141b0 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b7173c3730 con 0x55b7170f34d0 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b7174c8120 con 0x55b7170eb2f0 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6106d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7171141b0 0x55b7174d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40790/0 (socket says 172.21.15.31:40790) 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f610659c640 1 -- 172.21.15.31:0/84620165 learned_addr learned my addr 172.21.15.31:0/84620165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6106d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7171141b0 0x55b7174d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56844/0 (socket says 172.21.15.31:56844) 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7170f34d0 0x55b7174c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.960+0000 7f6113e50740 10 client.0 _ll_register_callbacks cb 0x55b7174176b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f610659c640 1 -- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7170f34d0 msgr2=0x55b7174c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f610659c640 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7170f34d0 0x55b7174c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f610659c640 1 -- 172.21.15.31:0/84620165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7171141b0 msgr2=0x55b7174d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f610659c640 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7171141b0 0x55b7174d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f610659c640 1 -- 172.21.15.31:0/84620165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b71717f790 con 0x55b7170eb2f0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f610659c640 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174c51d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60f800f7f0 tx=0x7f60f8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58431f17dc8c1df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.961+0000 7f60edffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.962+0000 7f60edffb640 1 -- 172.21.15.31:0/84620165 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f800fe30 con 0x55b7170eb2f0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.962+0000 7f60edffb640 1 -- 172.21.15.31:0/84620165 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60f8011bd0 con 0x55b7170eb2f0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.962+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6100040070 con 0x55b7170eb2f0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.963+0000 7f60edffb640 1 -- 172.21.15.31:0/84620165 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f60f8011dd0 con 0x55b7170eb2f0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.963+0000 7f6113e50740 10 client.26083 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.963+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b7171f8620 con 0x55b7170eb2f0 2024-09-21T06:15:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.963+0000 7f60edffb640 1 -- 172.21.15.31:0/84620165 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6100040070 con 0x55b7170eb2f0 2024-09-21T06:15:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f60edffb640 1 -- 172.21.15.31:0/84620165 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f60f8027070 con 0x55b7170eb2f0 2024-09-21T06:15:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f6113e50740 10 client.26083 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f6113e50740 10 client.26083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f6113e50740 20 client.26083 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f6113e50740 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b7170d2d80 0x55b7174d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f6113e50740 4 client.26083 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.964+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b717430640 con 0x55b7170d2d80 2024-09-21T06:15:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.965+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b7170d2d80 0x55b7174d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.965+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b7170d2d80 0x55b7174d7340 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c6e9a2e110032f server_cookie=9d68a7cd8e21588e in_seq=0 out_seq=0 2024-09-21T06:15:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.965+0000 7f60edffb640 10 client.26083 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.966+0000 7f60edffb640 1 -- 172.21.15.31:0/84620165 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b717430640 con 0x55b7170d2d80 2024-09-21T06:15:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.966+0000 7f60edffb640 10 client.26083 handle_command_reply: tid=0 2024-09-21T06:15:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.966+0000 7f6113e50740 1 client.26083 shutdown 2024-09-21T06:15:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.966+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 msgr2=0x55b7174c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.966+0000 7f6113e50740 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174c51d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60f800f7f0 tx=0x7f60f8004730 comp rx=0 tx=0).stop 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 shutdown_connections 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b7170d2d80 0x55b7174d7340 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7171141b0 0x55b7174d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7170f34d0 0x55b7174c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 --2- 172.21.15.31:0/84620165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170eb2f0 0x55b7174c51d0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f60f800f7f0 tx=0x7f60f8004730 comp rx=0 tx=0).stop 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 >> 172.21.15.31:0/84620165 conn(0x55b717108300 msgr2=0x55b7174ba9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f60edffb640 0 client.26083 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 shutdown_connections 2024-09-21T06:15:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 1 -- 172.21.15.31:0/84620165 wait complete. 2024-09-21T06:15:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.967+0000 7f6113e50740 20 client.26083 trim_cache size 0 max 16384 2024-09-21T06:15:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.992+0000 7f6113e50740 1 Processor -- start 2024-09-21T06:15:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.992+0000 7f6113e50740 1 -- start start 2024-09-21T06:15:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.992+0000 7f6113e50740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170f34d0 0x55b7174c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.992+0000 7f6113e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f6113e50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 0x55b7174d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f6113e50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b3bf0 con 0x55b7170a81c0 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f6113e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b63c0 con 0x55b7171141b0 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f6113e50740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b41b0 con 0x55b7170f34d0 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170f34d0 0x55b7174c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f6106d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 0x55b7174d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f610659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170f34d0 0x55b7174c08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40800/0 (socket says 172.21.15.31:40800) 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f6106d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 0x55b7174d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56858/0 (socket says 172.21.15.31:56858) 2024-09-21T06:15:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f610659c640 1 -- 172.21.15.31:0/1037167293 learned_addr learned my addr 172.21.15.31:0/1037167293 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.993+0000 7f60fdd9b640 1 -- 172.21.15.31:0/1037167293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170f34d0 msgr2=0x55b7174c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.994+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170f34d0 0x55b7174c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.994+0000 7f60fdd9b640 1 -- 172.21.15.31:0/1037167293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 msgr2=0x55b7174d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.994+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 0x55b7174d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.994+0000 7f60fdd9b640 1 -- 172.21.15.31:0/1037167293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7170d2fb0 con 0x55b7171141b0 2024-09-21T06:15:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.994+0000 7f6106d9d640 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 0x55b7174d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.995+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b7800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60f00e5180 tx=0x7f60f00a9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d51c7755ab4eaa4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.995+0000 7f60effff640 1 -- 172.21.15.31:0/1037167293 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00b2360 con 0x55b7171141b0 2024-09-21T06:15:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.995+0000 7f60effff640 1 -- 172.21.15.31:0/1037167293 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f00b72a0 con 0x55b7171141b0 2024-09-21T06:15:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.995+0000 7f60effff640 1 -- 172.21.15.31:0/1037167293 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00aee10 con 0x55b7171141b0 2024-09-21T06:15:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 -- 172.21.15.31:0/1037167293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 msgr2=0x55b7174b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b7800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60f00e5180 tx=0x7f60f00a9040 comp rx=0 tx=0).stop 2024-09-21T06:15:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 -- 172.21.15.31:0/1037167293 shutdown_connections 2024-09-21T06:15:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170a81c0 0x55b7174d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7171141b0 0x55b7174b7800 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60f00e5180 tx=0x7f60f00a9040 comp rx=0 tx=0).stop 2024-09-21T06:15:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 --2- 172.21.15.31:0/1037167293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170f34d0 0x55b7174c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:34.996+0000 7f6113e50740 1 -- 172.21.15.31:0/1037167293 >> 172.21.15.31:0/1037167293 conn(0x55b717108300 msgr2=0x55b7174b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.002+0000 7f6113e50740 1 -- 172.21.15.31:0/1037167293 shutdown_connections 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.006+0000 7f6113e50740 1 -- 172.21.15.31:0/1037167293 wait complete. 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.006+0000 7f6113e50740 1 -- start start 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.006+0000 7f6113e50740 1 Processor -- start 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170a81c0 0x55b7174cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7174bf6c0 0x55b7174bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b4880 con 0x55b7170f34d0 2024-09-21T06:15:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b5cf0 con 0x55b7174bf6c0 2024-09-21T06:15:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61081b4f50 con 0x55b7170a81c0 2024-09-21T06:15:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.008+0000 7f6113e50740 10 client.0 _ll_register_callbacks cb 0x7f60f8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.009+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.009+0000 7f60fdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174cdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56872/0 (socket says 172.21.15.31:56872) 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.009+0000 7f60fdd9b640 1 -- 172.21.15.31:0/2761141889 learned_addr learned my addr 172.21.15.31:0/2761141889 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.009+0000 7f6106d9d640 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7174bf6c0 0x55b7174bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.009+0000 7f60fdd9b640 1 -- 172.21.15.31:0/2761141889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170a81c0 msgr2=0x55b7174cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.010+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170a81c0 0x55b7174cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.010+0000 7f60fdd9b640 1 -- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7174bf6c0 msgr2=0x55b7174bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.010+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7174bf6c0 0x55b7174bfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.010+0000 7f60fdd9b640 1 -- 172.21.15.31:0/2761141889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7174c0370 con 0x55b7170f34d0 2024-09-21T06:15:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.010+0000 7f60fdd9b640 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174cdc90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ad580 tx=0x7f60f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca8f3623d642e36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.011+0000 7f60edffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:15:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.011+0000 7f60edffb640 1 -- 172.21.15.31:0/2761141889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00b2360 con 0x55b7170f34d0 2024-09-21T06:15:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.011+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f00b72a0 con 0x55b7170f34d0 2024-09-21T06:15:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.011+0000 7f60edffb640 1 -- 172.21.15.31:0/2761141889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60f00aebb0 con 0x55b7170f34d0 2024-09-21T06:15:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.012+0000 7f60edffb640 1 -- 172.21.15.31:0/2761141889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f60f00aedb0 con 0x55b7170f34d0 2024-09-21T06:15:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.012+0000 7f6113e50740 10 client.16335 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.012+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b717552d10 con 0x55b7170f34d0 2024-09-21T06:15:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f60edffb640 1 -- 172.21.15.31:0/2761141889 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f00b72a0 con 0x55b7170f34d0 2024-09-21T06:15:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f60edffb640 1 -- 172.21.15.31:0/2761141889 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f60f00b34f0 con 0x55b7170f34d0 2024-09-21T06:15:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f6113e50740 10 client.16335 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f6113e50740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f6113e50740 20 client.16335 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f6113e50740 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b717553180 0x55b7175555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f6113e50740 4 client.16335 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.013+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b71742c0f0 con 0x55b717553180 2024-09-21T06:15:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.014+0000 7f610659c640 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b717553180 0x55b7175555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.016+0000 7f610659c640 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b717553180 0x55b7175555d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26adc410198f8799 server_cookie=bc29e09d520e543a in_seq=0 out_seq=0 2024-09-21T06:15:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.016+0000 7f60edffb640 10 client.16335 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:35.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:34 smithi110 ceph-mon[34550]: pgmap v439: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 612 op/s 2024-09-21T06:15:35.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:34 smithi107 ceph-mon[34380]: pgmap v439: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 612 op/s 2024-09-21T06:15:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f60edffb640 1 -- 172.21.15.31:0/2761141889 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b71742c0f0 con 0x55b717553180 2024-09-21T06:15:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f60edffb640 10 client.16335 handle_command_reply: tid=0 2024-09-21T06:15:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 client.16335 shutdown 2024-09-21T06:15:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 msgr2=0x55b7174cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174cdc90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60f00ad580 tx=0x7f60f00057f0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 shutdown_connections 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b717553180 0x55b7175555d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7174bf6c0 0x55b7174bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7170f34d0 0x55b7174cdc90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 --2- 172.21.15.31:0/2761141889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b7170a81c0 0x55b7174cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 >> 172.21.15.31:0/2761141889 conn(0x55b717108300 msgr2=0x55b7174c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.048+0000 7f60edffb640 0 client.16335 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 shutdown_connections 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f6113e50740 1 -- 172.21.15.31:0/2761141889 wait complete. 2024-09-21T06:15:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f6113e50740 20 client.16335 trim_cache size 0 max 16384 2024-09-21T06:15:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:15:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:15:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:15:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:15:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60f40b6700 msgr2=0x7f60f40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f610ce0a640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60f40b6700 0x7f60f40b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55b7174b8630 tx=0x7f60f8005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108109420 msgr2=0x7f61081a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.049+0000 7f610ce0a640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108109420 0x7f61081a1c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f610806c8b0 tx=0x7f60f000ee50 comp rx=0 tx=0).stop 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.050+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 shutdown_connections 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.050+0000 7f610ce0a640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60f40b6700 0x7f60f40b8c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.050+0000 7f610ce0a640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6108109d70 0x7f61081ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.050+0000 7f610ce0a640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6108109420 0x7f61081a1c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.050+0000 7f610ce0a640 1 --2- 172.21.15.31:0/1891012906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6108103af0 0x7f61081a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.050+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 >> 172.21.15.31:0/1891012906 conn(0x7f61080ff570 msgr2=0x7f61081069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.051+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 shutdown_connections 2024-09-21T06:15:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:35.051+0000 7f610ce0a640 1 -- 172.21.15.31:0/1891012906 wait complete. 2024-09-21T06:15:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 235.0 s, 3606.7 tps, lat 27.675 ms stddev 31.106 2024-09-21T06:15:37.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:36 smithi107 ceph-mon[34380]: pgmap v440: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.6 MiB/s wr, 823 op/s 2024-09-21T06:15:37.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:36 smithi031 ceph-mon[31237]: pgmap v440: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.6 MiB/s wr, 823 op/s 2024-09-21T06:15:37.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:36 smithi110 ceph-mon[34550]: pgmap v440: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.6 MiB/s wr, 823 op/s 2024-09-21T06:15:39.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:38 smithi031 ceph-mon[31237]: pgmap v441: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.1 MiB/s wr, 676 op/s 2024-09-21T06:15:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:38 smithi110 ceph-mon[34550]: pgmap v441: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.1 MiB/s wr, 676 op/s 2024-09-21T06:15:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:38 smithi107 ceph-mon[34380]: pgmap v441: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.1 MiB/s wr, 676 op/s 2024-09-21T06:15:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 240.0 s, 4198.2 tps, lat 23.701 ms stddev 27.631 2024-09-21T06:15:41.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:40 smithi031 ceph-mon[31237]: pgmap v442: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-09-21T06:15:41.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:15:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:40 smithi110 ceph-mon[34550]: pgmap v442: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-09-21T06:15:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:40 smithi107 ceph-mon[34380]: pgmap v442: 97 pgs: 97 active+clean; 3.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-09-21T06:15:41.603 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:42.803 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:42 smithi031 ceph-mon[31237]: pgmap v443: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 907 op/s 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:15:42.994 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:15:42.998 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:42.999 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:43.000 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.708+0000 7fdd800f0640 1 -- 172.21.15.31:0/2215084178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 msgr2=0x7fdd781098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.708+0000 7fdd800f0640 1 --2- 172.21.15.31:0/2215084178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 0x7fdd781098e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00dc90 tx=0x7fdd6c032790 comp rx=0 tx=0).stop 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.710+0000 7fdd800f0640 1 -- 172.21.15.31:0/2215084178 shutdown_connections 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.710+0000 7fdd800f0640 1 --2- 172.21.15.31:0/2215084178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78109e20 0x7fdd78110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.710+0000 7fdd800f0640 1 --2- 172.21.15.31:0/2215084178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 0x7fdd781098e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.710+0000 7fdd800f0640 1 --2- 172.21.15.31:0/2215084178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78103b90 0x7fdd78103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.710+0000 7fdd800f0640 1 -- 172.21.15.31:0/2215084178 >> 172.21.15.31:0/2215084178 conn(0x7fdd780ff5e0 msgr2=0x7fdd78101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.711+0000 7fdd800f0640 1 -- 172.21.15.31:0/2215084178 shutdown_connections 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.711+0000 7fdd800f0640 1 -- 172.21.15.31:0/2215084178 wait complete. 2024-09-21T06:15:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.712+0000 7fdd800f0640 1 Processor -- start 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.712+0000 7fdd800f0640 1 -- start start 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.713+0000 7fdd800f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78103b90 0x7fdd781a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.713+0000 7fdd800f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 0x7fdd781a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.713+0000 7fdd800f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 0x7fdd781ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.713+0000 7fdd800f0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781229a0 con 0x7fdd781094d0 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.713+0000 7fdd800f0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78122820 con 0x7fdd78103b90 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.713+0000 7fdd800f0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78122b20 con 0x7fdd78109e20 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7de8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78103b90 0x7fdd781a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 0x7fdd781ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 0x7fdd781ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51680/0 (socket says 172.21.15.31:51680) 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3592161400 learned_addr learned my addr 172.21.15.31:0/3592161400 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3592161400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78103b90 msgr2=0x7fdd781a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78103b90 0x7fdd781a90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 msgr2=0x7fdd781a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.714+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 0x7fdd781a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.715+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3592161400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781af050 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.715+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 0x7fdd781ae920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000cad0 tx=0x7fdd7000ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.716+0000 7fdd6affd640 1 -- 172.21.15.31:0/3592161400 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000ad60 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.716+0000 7fdd6affd640 1 -- 172.21.15.31:0/3592161400 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70005020 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.716+0000 7fdd6affd640 1 -- 172.21.15.31:0/3592161400 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000f6b0 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.716+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781af340 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.717+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781b5aa0 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.724+0000 7fdd86664740 1 Processor -- start 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.727+0000 7fdd86664740 1 -- start start 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.727+0000 7fdd6affd640 1 -- 172.21.15.31:0/3592161400 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdd70004a40 con 0x7fdd78109e20 2024-09-21T06:15:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.727+0000 7fdd86664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.728+0000 7fdd6affd640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd640b6880 0x7fdd640b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd6affd640 1 -- 172.21.15.31:0/3592161400 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdd700e74c0 con 0x7fdd78109e20 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd86664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd86664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd7d68b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd7d68b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51684/0 (socket says 172.21.15.31:51684) 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd7d68b640 1 -- 172.21.15.31:0/2445524687 learned_addr learned my addr 172.21.15.31:0/2445524687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd86664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55923a926e50 con 0x55923a5e01b0 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55923a916280 con 0x55923a5e8390 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.729+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55923a91a090 con 0x55923a609070 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7d68b640 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 -- 172.21.15.31:0/2445524687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 msgr2=0x55923a9b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 -- 172.21.15.31:0/2445524687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 msgr2=0x55923a9b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 -- 172.21.15.31:0/2445524687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55923a674650 con 0x55923a5e8390 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.730+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9b0cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd700e4bf0 tx=0x7fdd700acb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8951c74c8ad3569 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.731+0000 7fdd63fff640 1 -- 172.21.15.31:0/2445524687 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001d040 con 0x55923a5e8390 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.731+0000 7fdd63fff640 1 -- 172.21.15.31:0/2445524687 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd700b1dd0 con 0x55923a5e8390 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.731+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd640b6880 0x7fdd640b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.732+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 msgr2=0x55923a9b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.732+0000 7fdd86664740 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9b0cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd700e4bf0 tx=0x7fdd700acb20 comp rx=0 tx=0).stop 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 shutdown_connections 2024-09-21T06:15:43.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9b0cf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 --2- 172.21.15.31:0/2445524687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 >> 172.21.15.31:0/2445524687 conn(0x55923a5fd460 msgr2=0x55923a9ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 shutdown_connections 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 -- 172.21.15.31:0/2445524687 wait complete. 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.733+0000 7fdd86664740 1 -- start start 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.734+0000 7fdd86664740 1 Processor -- start 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55923a777700 con 0x55923a5e01b0 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55923a8b1110 con 0x55923a5e8390 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55923a9beaf0 con 0x55923a609070 2024-09-21T06:15:43.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd86664740 10 client.0 _ll_register_callbacks cb 0x55923a8ca990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.735+0000 7fdd7d68b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.736+0000 7fdd7d68b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9bbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53718/0 (socket says 172.21.15.31:53718) 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.736+0000 7fdd7d68b640 1 -- 172.21.15.31:0/3843409089 learned_addr learned my addr 172.21.15.31:0/3843409089 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.736+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.737+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.737+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 msgr2=0x55923a9bbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.737+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9bbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.737+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3843409089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 msgr2=0x55923a9bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.737+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.737+0000 7fdd7e68d640 1 -- 172.21.15.31:0/3843409089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55923a8dec40 con 0x55923a609070 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9bc3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000caa0 tx=0x7fdd700acf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8211f90accf521 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd617fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd617fa640 1 -- 172.21.15.31:0/3843409089 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001d070 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd640b6880 0x7fdd640b8d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009a30 tx=0x7fdd74009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd700b1dd0 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd617fa640 1 -- 172.21.15.31:0/3843409089 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdd700b2f60 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.738+0000 7fdd617fa640 1 -- 172.21.15.31:0/3843409089 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdd700b6d10 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd86664740 10 client.26095 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55923a9db850 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd617fa640 1 -- 172.21.15.31:0/3843409089 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd700b1dd0 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd617fa640 1 -- 172.21.15.31:0/3843409089 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdd70094a30 con 0x55923a609070 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd86664740 10 client.26095 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd86664740 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd86664740 20 client.26095 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.739+0000 7fdd86664740 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a5c79f0 0x55923a9debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.740+0000 7fdd86664740 4 client.26095 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:43.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.740+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55923a928da0 con 0x55923a5c79f0 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.741+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a5c79f0 0x55923a9debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.742+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a5c79f0 0x55923a9debb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b16f57f1447eee server_cookie=540bd5535996c172 in_seq=0 out_seq=0 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.742+0000 7fdd617fa640 10 client.26095 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.742+0000 7fdd617fa640 1 -- 172.21.15.31:0/3843409089 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55923a928da0 con 0x55923a5c79f0 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.743+0000 7fdd617fa640 10 client.26095 handle_command_reply: tid=0 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.743+0000 7fdd86664740 1 client.26095 shutdown 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.744+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 msgr2=0x55923a9bc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.744+0000 7fdd86664740 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9bc3d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000caa0 tx=0x7fdd700acf40 comp rx=0 tx=0).stop 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 shutdown_connections 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a5c79f0 0x55923a9debb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9bc3d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e8390 0x55923a9bbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 --2- 172.21.15.31:0/3843409089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 >> 172.21.15.31:0/3843409089 conn(0x55923a5fd460 msgr2=0x55923a9b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd617fa640 0 client.26095 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 shutdown_connections 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 1 -- 172.21.15.31:0/3843409089 wait complete. 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.745+0000 7fdd86664740 20 client.26095 trim_cache size 0 max 16384 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.791+0000 7fdd86664740 1 Processor -- start 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.791+0000 7fdd86664740 1 -- start start 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd86664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e01b0 0x55923a9b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd86664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e8390 0x55923a9bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd86664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd86664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00c820 con 0x55923a5e8390 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd86664740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00f0a0 con 0x55923a5e01b0 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd86664740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00ce90 con 0x55923a609070 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd7e68d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd7e68d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51702/0 (socket says 172.21.15.31:51702) 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd7e68d640 1 -- 172.21.15.31:0/2048485232 learned_addr learned my addr 172.21.15.31:0/2048485232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd7d68b640 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e8390 0x55923a9bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.792+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e01b0 0x55923a9b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd7de8c640 1 -- 172.21.15.31:0/2048485232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 msgr2=0x55923a9ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd7de8c640 1 -- 172.21.15.31:0/2048485232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e8390 msgr2=0x55923a9bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e8390 0x55923a9bd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd7de8c640 1 -- 172.21.15.31:0/2048485232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55923a9bc9b0 con 0x55923a5e01b0 2024-09-21T06:15:43.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e01b0 0x55923a9b1c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55923a9df2c0 tx=0x7fdd74011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c2788917e65231 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.793+0000 7fdd637fe640 1 -- 172.21.15.31:0/2048485232 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7402c070 con 0x55923a5e01b0 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.794+0000 7fdd637fe640 1 -- 172.21.15.31:0/2048485232 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd74019040 con 0x55923a5e01b0 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 -- 172.21.15.31:0/2048485232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e01b0 msgr2=0x55923a9b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e01b0 0x55923a9b1c60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55923a9df2c0 tx=0x7fdd74011000 comp rx=0 tx=0).stop 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 -- 172.21.15.31:0/2048485232 shutdown_connections 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a609070 0x55923a9ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e8390 0x55923a9bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 --2- 172.21.15.31:0/2048485232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a5e01b0 0x55923a9b1c60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.795+0000 7fdd86664740 1 -- 172.21.15.31:0/2048485232 >> 172.21.15.31:0/2048485232 conn(0x55923a5fd460 msgr2=0x55923a9bce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.796+0000 7fdd86664740 1 -- 172.21.15.31:0/2048485232 shutdown_connections 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.796+0000 7fdd86664740 1 -- 172.21.15.31:0/2048485232 wait complete. 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.797+0000 7fdd86664740 1 -- start start 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.797+0000 7fdd86664740 1 Processor -- start 2024-09-21T06:15:43.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 0x55923a9c0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a609070 0x55923a9c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00d560 con 0x55923a5e01b0 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00e9d0 con 0x55923a609070 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5c00dc30 con 0x55923a5e8390 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd86664740 10 client.0 _ll_register_callbacks cb 0x7fdd70097070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd7d68b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 0x55923a9c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd7d68b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 0x55923a9c0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51710/0 (socket says 172.21.15.31:51710) 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.798+0000 7fdd7d68b640 1 -- 172.21.15.31:0/2632636178 learned_addr learned my addr 172.21.15.31:0/2632636178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a609070 0x55923a9c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7d68b640 1 -- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a609070 msgr2=0x55923a9c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7d68b640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a609070 0x55923a9c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7d68b640 1 -- 172.21.15.31:0/2632636178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 msgr2=0x55923a9bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7d68b640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7d68b640 1 -- 172.21.15.31:0/2632636178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55923aa48550 con 0x55923a5e8390 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7e68d640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a609070 0x55923a9c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd7d68b640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 0x55923a9c0280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c03a000 tx=0x7fdd6c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48cd14f5fd034262 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.799+0000 7fdd617fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.801+0000 7fdd617fa640 1 -- 172.21.15.31:0/2632636178 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c018400 con 0x55923a5e8390 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.801+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd74019040 con 0x55923a5e8390 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.801+0000 7fdd617fa640 1 -- 172.21.15.31:0/2632636178 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fdd6c0185a0 con 0x55923a5e8390 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.804+0000 7fdd617fa640 1 -- 172.21.15.31:0/2632636178 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdd6c00d410 con 0x55923a5e8390 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.804+0000 7fdd86664740 10 client.26101 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.804+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55923aa49780 con 0x55923a5e8390 2024-09-21T06:15:43.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.804+0000 7fdd617fa640 1 -- 172.21.15.31:0/2632636178 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd74019040 con 0x55923a5e8390 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd617fa640 1 -- 172.21.15.31:0/2632636178 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdd6c043540 con 0x55923a5e8390 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd86664740 10 client.26101 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd86664740 10 client.26101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd86664740 20 client.26101 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd86664740 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a59cda0 0x55923aa4bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd86664740 4 client.26101 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.805+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55923a924850 con 0x55923a59cda0 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.808+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a59cda0 0x55923aa4bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.809+0000 7fdd7de8c640 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a59cda0 0x55923aa4bf70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e78f920b6a8d97 server_cookie=fd41e361ec7748 in_seq=0 out_seq=0 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.809+0000 7fdd617fa640 10 client.26101 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.809+0000 7fdd617fa640 1 -- 172.21.15.31:0/2632636178 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55923a924850 con 0x55923a59cda0 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.810+0000 7fdd617fa640 10 client.26101 handle_command_reply: tid=0 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.810+0000 7fdd86664740 1 client.26101 shutdown 2024-09-21T06:15:43.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 msgr2=0x55923a9c0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 0x55923a9c0280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c03a000 tx=0x7fdd6c004270 comp rx=0 tx=0).stop 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 shutdown_connections 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55923a59cda0 0x55923aa4bf70 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923a609070 0x55923a9c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55923a5e8390 0x55923a9c0280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 --2- 172.21.15.31:0/2632636178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55923a5e01b0 0x55923a9bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 >> 172.21.15.31:0/2632636178 conn(0x55923a5fd460 msgr2=0x55923a9b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.811+0000 7fdd617fa640 0 client.26101 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.812+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 shutdown_connections 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.812+0000 7fdd86664740 1 -- 172.21.15.31:0/2632636178 wait complete. 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.812+0000 7fdd86664740 20 client.26101 trim_cache size 0 max 16384 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd640b6880 msgr2=0x7fdd640b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd640b6880 0x7fdd640b8d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009a30 tx=0x7fdd74009110 comp rx=0 tx=0).stop 2024-09-21T06:15:43.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 msgr2=0x7fdd781ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 0x7fdd781ae920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000cad0 tx=0x7fdd7000ced0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 shutdown_connections 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd78109e20 0x7fdd781ae920 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdd640b6880 0x7fdd640b8d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd781094d0 0x7fdd781a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 --2- 172.21.15.31:0/3592161400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd78103b90 0x7fdd781a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.813+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 >> 172.21.15.31:0/3592161400 conn(0x7fdd780ff5e0 msgr2=0x7fdd781072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.815+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 shutdown_connections 2024-09-21T06:15:43.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:42.815+0000 7fdd800f0640 1 -- 172.21.15.31:0/3592161400 wait complete. 2024-09-21T06:15:43.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:42 smithi110 ceph-mon[34550]: pgmap v443: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 907 op/s 2024-09-21T06:15:43.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:42 smithi107 ceph-mon[34380]: pgmap v443: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 907 op/s 2024-09-21T06:15:43.825 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:15:43.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:15:44.258 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:44.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:44 smithi031 ceph-mon[31237]: pgmap v444: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.4 MiB/s wr, 684 op/s 2024-09-21T06:15:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.868+0000 7fe31cc99640 1 -- 172.21.15.31:0/434971247 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310030c50 con 0x7fe3181096c0 2024-09-21T06:15:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.868+0000 7fe31ec9d640 1 -- 172.21.15.31:0/434971247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3181096c0 msgr2=0x7fe318109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.868+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/434971247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3181096c0 0x7fe318109ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe310009ca0 tx=0x7fe31002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.869+0000 7fe31ec9d640 1 -- 172.21.15.31:0/434971247 shutdown_connections 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.869+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/434971247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe318102020 0x7fe318110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.869+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/434971247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181016d0 0x7fe318101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.869+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/434971247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3181096c0 0x7fe318109ad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.869+0000 7fe31ec9d640 1 -- 172.21.15.31:0/434971247 >> 172.21.15.31:0/434971247 conn(0x7fe318078680 msgr2=0x7fe318078ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.870+0000 7fe31ec9d640 1 -- 172.21.15.31:0/434971247 shutdown_connections 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.870+0000 7fe31ec9d640 1 -- 172.21.15.31:0/434971247 wait complete. 2024-09-21T06:15:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 Processor -- start 2024-09-21T06:15:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 -- start start 2024-09-21T06:15:44.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3181016d0 0x7fe31819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318102020 0x7fe31819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 0x7fe31819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3181138f0 con 0x7fe3181016d0 2024-09-21T06:15:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe318113770 con 0x7fe318102020 2024-09-21T06:15:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.871+0000 7fe31ec9d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe318113a70 con 0x7fe3181096c0 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 0x7fe31819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 0x7fe31819fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51714/0 (socket says 172.21.15.31:51714) 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 -- 172.21.15.31:0/4130366623 learned_addr learned my addr 172.21.15.31:0/4130366623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31dc9b640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3181016d0 0x7fe31819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31d49a640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318102020 0x7fe31819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 -- 172.21.15.31:0/4130366623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318102020 msgr2=0x7fe31819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318102020 0x7fe31819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 -- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3181016d0 msgr2=0x7fe31819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3181016d0 0x7fe31819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.872+0000 7fe31e49c640 1 -- 172.21.15.31:0/4130366623 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3181a02a0 con 0x7fe3181096c0 2024-09-21T06:15:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.873+0000 7fe31e49c640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 0x7fe31819fba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe31400bfa0 tx=0x7fe31400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.873+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314018070 con 0x7fe3181096c0 2024-09-21T06:15:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.873+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3181a0590 con 0x7fe3181096c0 2024-09-21T06:15:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.873+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe318103070 con 0x7fe3181096c0 2024-09-21T06:15:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.873+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31400de60 con 0x7fe3181096c0 2024-09-21T06:15:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.873+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314004c60 con 0x7fe3181096c0 2024-09-21T06:15:44.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.875+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc0052c0 con 0x7fe3181096c0 2024-09-21T06:15:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.877+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe314004e60 con 0x7fe3181096c0 2024-09-21T06:15:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.878+0000 7fe30effd640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3040b67b0 0x7fe3040b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.878+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe3140eb130 con 0x7fe3181096c0 2024-09-21T06:15:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.878+0000 7fe31dc9b640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3040b67b0 0x7fe3040b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:44.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.879+0000 7fe31dc9b640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3040b67b0 0x7fe3040b8cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe310006010 tx=0x7fe310000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:44.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:44.882+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3140b1660 con 0x7fe3181096c0 2024-09-21T06:15:45.080 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:44 smithi110 ceph-mon[34550]: pgmap v444: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.4 MiB/s wr, 684 op/s 2024-09-21T06:15:45.090 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:44 smithi107 ceph-mon[34380]: pgmap v444: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.4 MiB/s wr, 684 op/s 2024-09-21T06:15:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 245.0 s, 3002.0 tps, lat 33.357 ms stddev 37.560 2024-09-21T06:15:45.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.281+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2fc005880 con 0x7fe3181096c0 2024-09-21T06:15:45.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.282+0000 7fe30effd640 1 -- 172.21.15.31:0/4130366623 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fe3140b5a40 con 0x7fe3181096c0 2024-09-21T06:15:45.284 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:15:45.284 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:15:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:15:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3040b67b0 msgr2=0x7fe3040b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3040b67b0 0x7fe3040b8cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe310006010 tx=0x7fe310000f40 comp rx=0 tx=0).stop 2024-09-21T06:15:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 msgr2=0x7fe31819fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 0x7fe31819fba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe31400bfa0 tx=0x7fe31400dc10 comp rx=0 tx=0).stop 2024-09-21T06:15:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 shutdown_connections 2024-09-21T06:15:45.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3181096c0 0x7fe31819fba0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318102020 0x7fe31819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3040b67b0 0x7fe3040b8cc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 --2- 172.21.15.31:0/4130366623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3181016d0 0x7fe31819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 >> 172.21.15.31:0/4130366623 conn(0x7fe318078680 msgr2=0x7fe318100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 shutdown_connections 2024-09-21T06:15:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:45.286+0000 7fe31ec9d640 1 -- 172.21.15.31:0/4130366623 wait complete. 2024-09-21T06:15:45.892 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:45 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4130366623' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:46.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:45 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4130366623' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:46.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:45 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4130366623' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:46.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:15:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:47.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:46 smithi031 ceph-mon[31237]: pgmap v445: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-21T06:15:47.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:46 smithi110 ceph-mon[34550]: pgmap v445: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-21T06:15:47.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:46 smithi107 ceph-mon[34380]: pgmap v445: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 -- 172.21.15.31:0/3088497642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810ead0 msgr2=0x7f33c81174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3088497642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810ead0 0x7f33c81174d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f33b800a970 tx=0x7f33b802b670 comp rx=0 tx=0).stop 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 -- 172.21.15.31:0/3088497642 shutdown_connections 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3088497642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810ead0 0x7f33c81174d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3088497642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c810e180 0x7f33c810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3088497642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810aee0 0x7f33c810b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 -- 172.21.15.31:0/3088497642 >> 172.21.15.31:0/3088497642 conn(0x7f33c8070ec0 msgr2=0x7f33c80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 -- 172.21.15.31:0/3088497642 shutdown_connections 2024-09-21T06:15:47.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.324+0000 7f33cdf99640 1 -- 172.21.15.31:0/3088497642 wait complete. 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.325+0000 7f33cdf99640 1 Processor -- start 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 -- start start 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810aee0 0x7f33c8117160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 0x7f33c8115460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c810ead0 0x7f33c81159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8119690 con 0x7f33c810e180 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8119510 con 0x7f33c810ead0 2024-09-21T06:15:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33cdf99640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8119810 con 0x7f33c810aee0 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 0x7f33c8115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 0x7f33c8115460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32896/0 (socket says 172.21.15.31:32896) 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33c6ffd640 1 -- 172.21.15.31:0/3420454256 learned_addr learned my addr 172.21.15.31:0/3420454256 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810aee0 0x7f33c8117160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.326+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c810ead0 0x7f33c81159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.327+0000 7f33c6ffd640 1 -- 172.21.15.31:0/3420454256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810aee0 msgr2=0x7f33c8117160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.327+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810aee0 0x7f33c8117160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.327+0000 7f33c6ffd640 1 -- 172.21.15.31:0/3420454256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c810ead0 msgr2=0x7f33c81159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.327+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c810ead0 0x7f33c81159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.327+0000 7f33c6ffd640 1 -- 172.21.15.31:0/3420454256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c81160d0 con 0x7f33c810e180 2024-09-21T06:15:47.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.327+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 0x7f33c8115460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f33c000ea50 tx=0x7f33c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.328+0000 7f33c4ff9640 1 -- 172.21.15.31:0/3420454256 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000cd40 con 0x7f33c810e180 2024-09-21T06:15:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.328+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c81b56b0 con 0x7f33c810e180 2024-09-21T06:15:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.328+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c81b5b60 con 0x7f33c810e180 2024-09-21T06:15:47.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.328+0000 7f33c4ff9640 1 -- 172.21.15.31:0/3420454256 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c0005020 con 0x7f33c810e180 2024-09-21T06:15:47.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.328+0000 7f33c4ff9640 1 -- 172.21.15.31:0/3420454256 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c00116b0 con 0x7f33c810e180 2024-09-21T06:15:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 Processor -- start 2024-09-21T06:15:47.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 -- start start 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 0x5651eff502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb801b0 0x5651eff50800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efb88390 0x5651eff50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5651efec6ea0 con 0x5651efb801b0 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651efeb62d0 con 0x5651efb88390 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33d450d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651efeba0e0 con 0x5651efba9070 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33c7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 0x5651eff502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33c7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 0x5651eff502c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51736/0 (socket says 172.21.15.31:51736) 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.332+0000 7f33c7fff640 1 -- 172.21.15.31:0/3548155771 learned_addr learned my addr 172.21.15.31:0/3548155771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:47.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb801b0 0x5651eff50800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c77fe640 1 -- 172.21.15.31:0/3548155771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 msgr2=0x5651eff502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 0x5651eff502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c77fe640 1 -- 172.21.15.31:0/3548155771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efb88390 msgr2=0x5651eff50d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efb88390 0x5651eff50d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c77fe640 1 -- 172.21.15.31:0/3548155771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651efc14650 con 0x5651efb801b0 2024-09-21T06:15:47.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.333+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 0x5651eff502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:47.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.334+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb801b0 0x5651eff50800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009530 tx=0x7f33bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97b20cdff63a095 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:47.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.334+0000 7f33adffb640 1 -- 172.21.15.31:0/3548155771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc01e070 con 0x5651efb801b0 2024-09-21T06:15:47.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.334+0000 7f33adffb640 1 -- 172.21.15.31:0/3548155771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc019ae0 con 0x5651efb801b0 2024-09-21T06:15:47.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.334+0000 7f33adffb640 1 -- 172.21.15.31:0/3548155771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00e9e0 con 0x5651efb801b0 2024-09-21T06:15:47.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.334+0000 7f33c4ff9640 1 -- 172.21.15.31:0/3420454256 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f33c0011850 con 0x7f33c810e180 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.335+0000 7f33c4ff9640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33b00b67c0 0x7f33b00b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.335+0000 7f33d450d740 1 -- 172.21.15.31:0/3548155771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb801b0 msgr2=0x5651eff50800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.335+0000 7f33d450d740 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb801b0 0x5651eff50800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009530 tx=0x7f33bc011810 comp rx=0 tx=0).stop 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 -- 172.21.15.31:0/3548155771 shutdown_connections 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efb88390 0x5651eff50d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb801b0 0x5651eff50800 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 --2- 172.21.15.31:0/3548155771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efba9070 0x5651eff502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 -- 172.21.15.31:0/3548155771 >> 172.21.15.31:0/3548155771 conn(0x5651efb9d460 msgr2=0x5651eff4e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:47.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33c4ff9640 1 -- 172.21.15.31:0/3420454256 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f33c00e7680 con 0x7f33c810e180 2024-09-21T06:15:47.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33b00b67c0 0x7f33b00b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 -- 172.21.15.31:0/3548155771 shutdown_connections 2024-09-21T06:15:47.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 -- 172.21.15.31:0/3548155771 wait complete. 2024-09-21T06:15:47.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.336+0000 7f33d450d740 1 -- start start 2024-09-21T06:15:47.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33b00b67c0 0x7f33b00b8cd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33b800a940 tx=0x7f33b8030f80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:47.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 Processor -- start 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff5be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5651efd17700 con 0x5651efb88390 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651efe51160 con 0x5651efba9070 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.337+0000 7f33d450d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5651eff5eaa0 con 0x5651efb801b0 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33d450d740 10 client.0 _ll_register_callbacks cb 0x5651efe6a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51738/0 (socket says 172.21.15.31:51738) 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 -- 172.21.15.31:0/3124452085 learned_addr learned my addr 172.21.15.31:0/3124452085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 -- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 msgr2=0x5651eff5c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 -- 172.21.15.31:0/3124452085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 msgr2=0x5651eff5be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff5be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 -- 172.21.15.31:0/3124452085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651efe7ec90 con 0x5651efb801b0 2024-09-21T06:15:47.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.338+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5b920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005890 tx=0x7f33bc019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9739efe8dc85fb04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:47.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.339+0000 7f33a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:47.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.339+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3124452085 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc029020 con 0x5651efb801b0 2024-09-21T06:15:47.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.339+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3124452085 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f33bc005040 con 0x5651efb801b0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.339+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33bc019ae0 con 0x5651efb801b0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.340+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3124452085 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f33bc01e040 con 0x5651efb801b0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.340+0000 7f33d450d740 10 client.26119 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.340+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5651eff6a780 con 0x5651efb801b0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.340+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3124452085 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc019ae0 con 0x5651efb801b0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.340+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3124452085 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f33bc00eaf0 con 0x5651efb801b0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33d450d740 10 client.26119 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33d450d740 10 client.26119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33d450d740 20 client.26119 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33d450d740 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb679f0 0x5651eff6dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33d450d740 4 client.26119 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651efec8df0 con 0x5651efb679f0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb679f0 0x5651eff6dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.341+0000 7f33a6ffd640 10 client.26119 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:47.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.342+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb679f0 0x5651eff6dae0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a6578880e2de10 server_cookie=a95190a72e9c0024 in_seq=0 out_seq=0 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3124452085 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651efec8df0 con 0x5651efb679f0 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33a6ffd640 10 client.26119 handle_command_reply: tid=0 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 client.26119 shutdown 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 msgr2=0x5651eff5b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5b920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005890 tx=0x7f33bc019660 comp rx=0 tx=0).stop 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 shutdown_connections 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb679f0 0x5651eff6dae0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff5be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.343+0000 7f33d450d740 1 --2- 172.21.15.31:0/3124452085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5b920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.344+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 >> 172.21.15.31:0/3124452085 conn(0x5651efb9d460 msgr2=0x5651eff53450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.344+0000 7f33a6ffd640 0 client.26119 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.344+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 shutdown_connections 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.344+0000 7f33d450d740 1 -- 172.21.15.31:0/3124452085 wait complete. 2024-09-21T06:15:47.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.344+0000 7f33d450d740 20 client.26119 trim_cache size 0 max 16384 2024-09-21T06:15:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.365+0000 7f33d450d740 1 Processor -- start 2024-09-21T06:15:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.365+0000 7f33d450d740 1 -- start start 2024-09-21T06:15:47.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33d450d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33d450d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff51a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33d450d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33d450d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33b400c7c0 con 0x5651efb88390 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33d450d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33b400f040 con 0x5651efba9070 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff51a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff51a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32936/0 (socket says 172.21.15.31:32936) 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33c6ffd640 1 -- 172.21.15.31:0/2319563851 learned_addr learned my addr 172.21.15.31:0/2319563851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33d450d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33b400ce30 con 0x5651efb801b0 2024-09-21T06:15:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.366+0000 7f33c77fe640 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.367+0000 7f33c6ffd640 1 -- 172.21.15.31:0/2319563851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 msgr2=0x5651eff5d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.367+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.367+0000 7f33c6ffd640 1 -- 172.21.15.31:0/2319563851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 msgr2=0x5651eff4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.367+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.367+0000 7f33c6ffd640 1 -- 172.21.15.31:0/2319563851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651efb67c10 con 0x5651efb88390 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.367+0000 7f33c6ffd640 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff51a00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ad4f0 tx=0x7f33c0005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0124cd2229d62ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.368+0000 7f33ad7fa640 1 -- 172.21.15.31:0/2319563851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c001f030 con 0x5651efb88390 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.368+0000 7f33ad7fa640 1 -- 172.21.15.31:0/2319563851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c00e5030 con 0x5651efb88390 2024-09-21T06:15:47.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.368+0000 7f33ad7fa640 1 -- 172.21.15.31:0/2319563851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c00ac490 con 0x5651efb88390 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.369+0000 7f33d450d740 1 -- 172.21.15.31:0/2319563851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 msgr2=0x5651eff51a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.369+0000 7f33d450d740 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff51a00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ad4f0 tx=0x7f33c0005b40 comp rx=0 tx=0).stop 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.370+0000 7f33d450d740 1 -- 172.21.15.31:0/2319563851 shutdown_connections 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.370+0000 7f33d450d740 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.370+0000 7f33d450d740 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff51a00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.370+0000 7f33d450d740 1 --2- 172.21.15.31:0/2319563851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff5d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.370+0000 7f33d450d740 1 -- 172.21.15.31:0/2319563851 >> 172.21.15.31:0/2319563851 conn(0x5651efb9d460 msgr2=0x5651eff512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:47.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.372+0000 7f33d450d740 1 -- 172.21.15.31:0/2319563851 shutdown_connections 2024-09-21T06:15:47.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.377+0000 7f33d450d740 1 -- 172.21.15.31:0/2319563851 wait complete. 2024-09-21T06:15:47.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.377+0000 7f33d450d740 1 -- start start 2024-09-21T06:15:47.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.377+0000 7f33d450d740 1 Processor -- start 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33b400d500 con 0x5651efb88390 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33b400e970 con 0x5651efba9070 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33b400dbd0 con 0x5651efb801b0 2024-09-21T06:15:47.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33d450d740 10 client.0 _ll_register_callbacks cb 0x7f33bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:47.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33c77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff642b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51752/0 (socket says 172.21.15.31:51752) 2024-09-21T06:15:47.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.378+0000 7f33c77fe640 1 -- 172.21.15.31:0/3693652726 learned_addr learned my addr 172.21.15.31:0/3693652726 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:47.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.379+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.382+0000 7f33c7fff640 1 -- 172.21.15.31:0/3693652726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 msgr2=0x5651eff642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.382+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.382+0000 7f33c7fff640 1 -- 172.21.15.31:0/3693652726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 msgr2=0x5651eff5f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.382+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.382+0000 7f33c7fff640 1 -- 172.21.15.31:0/3693652726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651eff60230 con 0x5651efba9070 2024-09-21T06:15:47.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.386+0000 7f33c7fff640 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5fcf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33b8031040 tx=0x7f33b8032400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85dca140184cb15a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:47.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.386+0000 7f33a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.389+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3693652726 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b80053d0 con 0x5651efba9070 2024-09-21T06:15:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.390+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3693652726 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f33b8005570 con 0x5651efba9070 2024-09-21T06:15:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.390+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c00e5030 con 0x5651efba9070 2024-09-21T06:15:47.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.391+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3693652726 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f33b80046c0 con 0x5651efba9070 2024-09-21T06:15:47.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.391+0000 7f33d450d740 10 client.26090 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:47.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.391+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5651effe90a0 con 0x5651efba9070 2024-09-21T06:15:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.392+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3693652726 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c00e5030 con 0x5651efba9070 2024-09-21T06:15:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.392+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3693652726 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f33b8052020 con 0x5651efba9070 2024-09-21T06:15:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.393+0000 7f33d450d740 10 client.26090 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.393+0000 7f33d450d740 10 client.26090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.393+0000 7f33d450d740 20 client.26090 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.393+0000 7f33d450d740 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb3cda0 0x5651effeb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.393+0000 7f33d450d740 4 client.26090 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.393+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651efec48a0 con 0x5651efb3cda0 2024-09-21T06:15:47.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.398+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb3cda0 0x5651effeb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:47.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.414+0000 7f33c77fe640 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb3cda0 0x5651effeb810 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3a9b71927c5fdd server_cookie=5d83a8b2e763c343 in_seq=0 out_seq=0 2024-09-21T06:15:47.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.414+0000 7f33a6ffd640 10 client.26090 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:47.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.414+0000 7f33a6ffd640 1 -- 172.21.15.31:0/3693652726 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651efec48a0 con 0x5651efb3cda0 2024-09-21T06:15:47.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.414+0000 7f33a6ffd640 10 client.26090 handle_command_reply: tid=0 2024-09-21T06:15:47.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 client.26090 shutdown 2024-09-21T06:15:47.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 msgr2=0x5651eff5fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5fcf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33b8031040 tx=0x7f33b8032400 comp rx=0 tx=0).stop 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 shutdown_connections 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5651efb3cda0 0x5651effeb810 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651efba9070 0x5651eff5fcf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5651efb88390 0x5651eff5f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 --2- 172.21.15.31:0/3693652726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5651efb801b0 0x5651eff642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33a6ffd640 0 client.26090 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.415+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 >> 172.21.15.31:0/3693652726 conn(0x5651efb9d460 msgr2=0x5651eff512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:47.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.416+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 shutdown_connections 2024-09-21T06:15:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.417+0000 7f33d450d740 1 -- 172.21.15.31:0/3693652726 wait complete. 2024-09-21T06:15:47.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.417+0000 7f33d450d740 20 client.26090 trim_cache size 0 max 16384 2024-09-21T06:15:47.420 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33b00b67c0 msgr2=0x7f33b00b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33b00b67c0 0x7f33b00b8cd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f33b800a940 tx=0x7f33b8030f80 comp rx=0 tx=0).stop 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 msgr2=0x7f33c8115460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 0x7f33c8115460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f33c000ea50 tx=0x7f33c000ef10 comp rx=0 tx=0).stop 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 shutdown_connections 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33c810ead0 0x7f33c81159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f33b00b67c0 0x7f33b00b8cd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c810e180 0x7f33c8115460 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 --2- 172.21.15.31:0/3420454256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33c810aee0 0x7f33c8117160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:47.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.421+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 >> 172.21.15.31:0/3420454256 conn(0x7f33c8070ec0 msgr2=0x7f33c810c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:47.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.422+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 shutdown_connections 2024-09-21T06:15:47.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:47.422+0000 7f33cdf99640 1 -- 172.21.15.31:0/3420454256 wait complete. 2024-09-21T06:15:48.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:47 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3580111216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b4105cf0 msgr2=0x7ff6b4106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3580111216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b4105cf0 0x7ff6b4106100 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4009900 tx=0x7ff6a402a410 comp rx=0 tx=0).stop 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3580111216 shutdown_connections 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3580111216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b410bf70 0x7ff6b41127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3580111216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b410b620 0x7ff6b410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3580111216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b4105cf0 0x7ff6b4106100 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3580111216 >> 172.21.15.31:0/3580111216 conn(0x7ff6b41016f0 msgr2=0x7ff6b4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3580111216 shutdown_connections 2024-09-21T06:15:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.190+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3580111216 wait complete. 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 Processor -- start 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 -- start start 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 0x7ff6b419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b410b620 0x7ff6b419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b410bf70 0x7ff6b41a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4115c80 con 0x7ff6b410bf70 2024-09-21T06:15:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4115b00 con 0x7ff6b410b620 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b91ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b4115e00 con 0x7ff6b4105cf0 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.191+0000 7ff6b2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 0x7ff6b419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 0x7ff6b419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51778/0 (socket says 172.21.15.31:51778) 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 -- 172.21.15.31:0/3689163276 learned_addr learned my addr 172.21.15.31:0/3689163276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b359e640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b410bf70 0x7ff6b41a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b259c640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b410b620 0x7ff6b419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 -- 172.21.15.31:0/3689163276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b410b620 msgr2=0x7ff6b419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b410b620 0x7ff6b419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 -- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b410bf70 msgr2=0x7ff6b41a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b410bf70 0x7ff6b41a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.192+0000 7ff6b2d9d640 1 -- 172.21.15.31:0/3689163276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b41a2630 con 0x7ff6b4105cf0 2024-09-21T06:15:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.193+0000 7ff6b2d9d640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 0x7ff6b419c700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005ab0 tx=0x7ff6a4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.193+0000 7ff69bfff640 1 -- 172.21.15.31:0/3689163276 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a403f070 con 0x7ff6b4105cf0 2024-09-21T06:15:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.193+0000 7ff69bfff640 1 -- 172.21.15.31:0/3689163276 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6a4031b60 con 0x7ff6b4105cf0 2024-09-21T06:15:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.193+0000 7ff69bfff640 1 -- 172.21.15.31:0/3689163276 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4035030 con 0x7ff6b4105cf0 2024-09-21T06:15:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.194+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b41a28c0 con 0x7ff6b4105cf0 2024-09-21T06:15:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.194+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b41a2d60 con 0x7ff6b4105cf0 2024-09-21T06:15:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.198+0000 7ff6c0202740 1 Processor -- start 2024-09-21T06:15:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.198+0000 7ff6c0202740 1 -- start start 2024-09-21T06:15:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff69bfff640 1 -- 172.21.15.31:0/3689163276 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff6a4042070 con 0x7ff6b4105cf0 2024-09-21T06:15:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f4674f2f0 0x563f46b1def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6c0202740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f46a926f0 con 0x563f4674f2f0 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6c0202740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f46a81b20 con 0x563f467574d0 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6c0202740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f46a85930 con 0x563f467781b0 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b1d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51782/0 (socket says 172.21.15.31:51782) 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 -- 172.21.15.31:0/59965314 learned_addr learned my addr 172.21.15.31:0/59965314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 -- 172.21.15.31:0/59965314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 msgr2=0x563f46b1e430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b1e430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 -- 172.21.15.31:0/59965314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f4674f2f0 msgr2=0x563f46b1def0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f4674f2f0 0x563f46b1def0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.199+0000 7ff6b359e640 1 -- 172.21.15.31:0/59965314 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f46a47530 con 0x563f467781b0 2024-09-21T06:15:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.200+0000 7ff6b359e640 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b1d9b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009f80 tx=0x7ff6ac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1071b694f14d5897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.200+0000 7ff69bfff640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff69c0b6700 0x7ff69c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.200+0000 7ff6997fa640 1 -- 172.21.15.31:0/59965314 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac02e070 con 0x563f467781b0 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.200+0000 7ff6997fa640 1 -- 172.21.15.31:0/59965314 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac003e20 con 0x563f467781b0 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.201+0000 7ff6997fa640 1 -- 172.21.15.31:0/59965314 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac004850 con 0x563f467781b0 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.201+0000 7ff69bfff640 1 -- 172.21.15.31:0/3689163276 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff6a4031d10 con 0x7ff6b4105cf0 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.201+0000 7ff6b2d9d640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff69c0b6700 0x7ff69c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 -- 172.21.15.31:0/59965314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 msgr2=0x563f46b1d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b1d9b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009f80 tx=0x7ff6ac0118d0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 -- 172.21.15.31:0/59965314 shutdown_connections 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b1e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f4674f2f0 0x563f46b1def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 --2- 172.21.15.31:0/59965314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b1d9b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 -- 172.21.15.31:0/59965314 >> 172.21.15.31:0/59965314 conn(0x563f4676c300 msgr2=0x563f46b1b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 -- 172.21.15.31:0/59965314 shutdown_connections 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6b2d9d640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff69c0b6700 0x7ff69c0b8c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x563f46b1c630 tx=0x7ff6a402a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.202+0000 7ff6c0202740 1 -- 172.21.15.31:0/59965314 wait complete. 2024-09-21T06:15:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.204+0000 7ff6c0202740 1 -- start start 2024-09-21T06:15:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.205+0000 7ff6c0202740 1 Processor -- start 2024-09-21T06:15:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b21c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467781b0 0x563f46b22710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f469b9ab0 con 0x563f467781b0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f46a27730 con 0x563f467574d0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f46b2c0c0 con 0x563f4674f2f0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6c0202740 10 client.0 _ll_register_callbacks cb 0x563f46a7b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b21c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b21c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51784/0 (socket says 172.21.15.31:51784) 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 -- 172.21.15.31:0/1872314555 learned_addr learned my addr 172.21.15.31:0/1872314555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467781b0 0x563f46b22710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 -- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 msgr2=0x563f46b221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 -- 172.21.15.31:0/1872314555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467781b0 msgr2=0x563f46b22710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467781b0 0x563f46b22710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.207+0000 7ff6b359e640 1 -- 172.21.15.31:0/1872314555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f467e3790 con 0x563f4674f2f0 2024-09-21T06:15:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b21c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009fd0 tx=0x7ff6ac005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85fffcf23a7148f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff696ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff696ff5640 1 -- 172.21.15.31:0/1872314555 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac02e070 con 0x563f4674f2f0 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff696ff5640 1 -- 172.21.15.31:0/1872314555 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff6ac004c40 con 0x563f4674f2f0 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac003e20 con 0x563f4674f2f0 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff696ff5640 1 -- 172.21.15.31:0/1872314555 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac004e40 con 0x563f4674f2f0 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff6c0202740 10 client.26137 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563f4685c620 con 0x563f4674f2f0 2024-09-21T06:15:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.208+0000 7ff696ff5640 1 -- 172.21.15.31:0/1872314555 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac003e20 con 0x563f4674f2f0 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff696ff5640 1 -- 172.21.15.31:0/1872314555 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff6ac005230 con 0x563f4674f2f0 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6c0202740 10 client.26137 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6c0202740 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6c0202740 20 client.26137 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f46736d80 0x563f46b3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6c0202740 4 client.26137 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f46a94640 con 0x563f46736d80 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.209+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f46736d80 0x563f46b3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.210+0000 7ff696ff5640 10 client.26137 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.210+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f46736d80 0x563f46b3b220 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744e8627cd5aff1c server_cookie=39ff8c7688bffc8b in_seq=0 out_seq=0 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.210+0000 7ff696ff5640 1 -- 172.21.15.31:0/1872314555 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f46a94640 con 0x563f46736d80 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.210+0000 7ff696ff5640 10 client.26137 handle_command_reply: tid=0 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.210+0000 7ff6c0202740 1 client.26137 shutdown 2024-09-21T06:15:48.214 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 msgr2=0x563f46b21c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b21c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009fd0 tx=0x7ff6ac005bd0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 shutdown_connections 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f46736d80 0x563f46b3b220 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467781b0 0x563f46b22710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467574d0 0x563f46b221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1872314555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b21c90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 >> 172.21.15.31:0/1872314555 conn(0x563f4676c300 msgr2=0x563f46b1e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff696ff5640 0 client.26137 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 shutdown_connections 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 1 -- 172.21.15.31:0/1872314555 wait complete. 2024-09-21T06:15:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.211+0000 7ff6c0202740 20 client.26137 trim_cache size 0 max 16384 2024-09-21T06:15:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.251+0000 7ff6c0202740 1 Processor -- start 2024-09-21T06:15:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 -- start start 2024-09-21T06:15:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4674f2f0 0x563f46b1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b2d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b2d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41ab580 con 0x563f467574d0 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41ade00 con 0x563f4674f2f0 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6c0202740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41abbf0 con 0x563f467781b0 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b2d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b2d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32986/0 (socket says 172.21.15.31:32986) 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.254+0000 7ff6b259c640 1 -- 172.21.15.31:0/1798177281 learned_addr learned my addr 172.21.15.31:0/1798177281 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.255+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b2d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.255+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4674f2f0 0x563f46b1f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.256+0000 7ff6b359e640 1 -- 172.21.15.31:0/1798177281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4674f2f0 msgr2=0x563f46b1f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.256+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4674f2f0 0x563f46b1f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.256+0000 7ff6b359e640 1 -- 172.21.15.31:0/1798177281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 msgr2=0x563f46b2d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.256+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b2d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.256+0000 7ff6b359e640 1 -- 172.21.15.31:0/1798177281 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f46b201c0 con 0x563f467781b0 2024-09-21T06:15:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.256+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.261+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b2d820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009250 tx=0x7ff6ac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427f5a2863f3d25e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.263+0000 7ff698ff9640 1 -- 172.21.15.31:0/1798177281 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac02e070 con 0x563f467781b0 2024-09-21T06:15:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.263+0000 7ff698ff9640 1 -- 172.21.15.31:0/1798177281 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac02f040 con 0x563f467781b0 2024-09-21T06:15:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.263+0000 7ff698ff9640 1 -- 172.21.15.31:0/1798177281 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00f830 con 0x563f467781b0 2024-09-21T06:15:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.264+0000 7ff6c0202740 1 -- 172.21.15.31:0/1798177281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 msgr2=0x563f46b2d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.264+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b2d820 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009250 tx=0x7ff6ac0118d0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.268+0000 7ff6c0202740 1 -- 172.21.15.31:0/1798177281 shutdown_connections 2024-09-21T06:15:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.268+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f467781b0 0x563f46b2d820 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.268+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.268+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1798177281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f4674f2f0 0x563f46b1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.268+0000 7ff6c0202740 1 -- 172.21.15.31:0/1798177281 >> 172.21.15.31:0/1798177281 conn(0x563f4676c300 msgr2=0x563f46b1e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.272+0000 7ff6c0202740 1 -- 172.21.15.31:0/1798177281 shutdown_connections 2024-09-21T06:15:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.275+0000 7ff6c0202740 1 -- 172.21.15.31:0/1798177281 wait complete. 2024-09-21T06:15:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.275+0000 7ff6c0202740 1 -- start start 2024-09-21T06:15:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 Processor -- start 2024-09-21T06:15:48.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:47 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:15:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b3d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467781b0 0x563f46b3ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41ac2c0 con 0x563f467574d0 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41ad730 con 0x563f467781b0 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b41ac990 con 0x563f4674f2f0 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.276+0000 7ff6c0202740 10 client.0 _ll_register_callbacks cb 0x7ff6ac009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b3d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b3d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51802/0 (socket says 172.21.15.31:51802) 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b359e640 1 -- 172.21.15.31:0/1969145026 learned_addr learned my addr 172.21.15.31:0/1969145026 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467781b0 0x563f46b3ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b259c640 1 -- 172.21.15.31:0/1969145026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 msgr2=0x563f46b3d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b3d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b259c640 1 -- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467781b0 msgr2=0x563f46b3ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467781b0 0x563f46b3ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.277+0000 7ff6b259c640 1 -- 172.21.15.31:0/1969145026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f46b9daa0 con 0x563f467574d0 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.278+0000 7ff6b259c640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b3d8b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x563f46b0d6b0 tx=0x7ff6a8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8824c8802d88f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.278+0000 7ff696ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:15:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.279+0000 7ff696ff5640 1 -- 172.21.15.31:0/1969145026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a801c720 con 0x563f467574d0 2024-09-21T06:15:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.279+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac02f040 con 0x563f467574d0 2024-09-21T06:15:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.279+0000 7ff696ff5640 1 -- 172.21.15.31:0/1969145026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff6a801c8c0 con 0x563f467574d0 2024-09-21T06:15:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.279+0000 7ff696ff5640 1 -- 172.21.15.31:0/1969145026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff6a801cac0 con 0x563f467574d0 2024-09-21T06:15:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.279+0000 7ff6c0202740 10 client.16383 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.279+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563f46b9edc0 con 0x563f467574d0 2024-09-21T06:15:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.280+0000 7ff696ff5640 1 -- 172.21.15.31:0/1969145026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac02f040 con 0x563f467574d0 2024-09-21T06:15:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.280+0000 7ff696ff5640 1 -- 172.21.15.31:0/1969145026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff6a80223d0 con 0x563f467574d0 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6c0202740 10 client.16383 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6c0202740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6c0202740 20 client.16383 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f4670c1c0 0x563f46bb89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6c0202740 4 client.16383 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f46a900f0 con 0x563f4670c1c0 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.281+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f4670c1c0 0x563f46bb89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.282+0000 7ff696ff5640 10 client.16383 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.282+0000 7ff6b359e640 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f4670c1c0 0x563f46bb89b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c42f69c2f8e1fda server_cookie=c13dbe4bf43cfb07 in_seq=0 out_seq=0 2024-09-21T06:15:48.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:47 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:15:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.309+0000 7ff696ff5640 1 -- 172.21.15.31:0/1969145026 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f46a900f0 con 0x563f4670c1c0 2024-09-21T06:15:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.309+0000 7ff696ff5640 10 client.16383 handle_command_reply: tid=0 2024-09-21T06:15:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.309+0000 7ff6c0202740 1 client.16383 shutdown 2024-09-21T06:15:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 msgr2=0x563f46b3d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b3d8b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x563f46b0d6b0 tx=0x7ff6a8010800 comp rx=0 tx=0).stop 2024-09-21T06:15:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 shutdown_connections 2024-09-21T06:15:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f4670c1c0 0x563f46bb89b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f467781b0 0x563f46b3ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f467574d0 0x563f46b3d8b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 --2- 172.21.15.31:0/1969145026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f4674f2f0 0x563f46b3d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff696ff5640 0 client.16383 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 >> 172.21.15.31:0/1969145026 conn(0x563f4676c300 msgr2=0x563f46b323d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 shutdown_connections 2024-09-21T06:15:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 1 -- 172.21.15.31:0/1969145026 wait complete. 2024-09-21T06:15:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.310+0000 7ff6c0202740 20 client.16383 trim_cache size 0 max 16384 2024-09-21T06:15:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:15:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:15:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:15:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.311+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff69c0b6700 msgr2=0x7ff69c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.311+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff69c0b6700 0x7ff69c0b8c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x563f46b1c630 tx=0x7ff6a402a410 comp rx=0 tx=0).stop 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.311+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 msgr2=0x7ff6b419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.311+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 0x7ff6b419c700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4005ab0 tx=0x7ff6a4003c10 comp rx=0 tx=0).stop 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 shutdown_connections 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff69c0b6700 0x7ff69c0b8c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6b410bf70 0x7ff6b41a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6b410b620 0x7ff6b419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 --2- 172.21.15.31:0/3689163276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6b4105cf0 0x7ff6b419c700 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 >> 172.21.15.31:0/3689163276 conn(0x7ff6b41016f0 msgr2=0x7ff6b410ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 shutdown_connections 2024-09-21T06:15:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:15:48.312+0000 7ff6b91ba640 1 -- 172.21.15.31:0/3689163276 wait complete. 2024-09-21T06:15:49.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:48 smithi031 ceph-mon[31237]: pgmap v446: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-21T06:15:49.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:15:49.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:15:49.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:15:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:48 smithi110 ceph-mon[34550]: pgmap v446: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-21T06:15:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:15:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:15:49.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:15:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:48 smithi107 ceph-mon[34380]: pgmap v446: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-21T06:15:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:15:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:15:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:15:49.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:15:49.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:15:49.688 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 250.0 s, 2417.2 tps, lat 41.364 ms stddev 46.181 2024-09-21T06:15:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.321+0000 7f89dfd93640 1 -- 172.21.15.31:0/3127950601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8109ec0 msgr2=0x7f89d81106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.321+0000 7f89dfd93640 1 --2- 172.21.15.31:0/3127950601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8109ec0 0x7f89d81106f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009be0 tx=0x7f89d002a6f0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.323+0000 7f89dfd93640 1 -- 172.21.15.31:0/3127950601 shutdown_connections 2024-09-21T06:15:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.323+0000 7f89dfd93640 1 --2- 172.21.15.31:0/3127950601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8109ec0 0x7f89d81106f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.323+0000 7f89dfd93640 1 --2- 172.21.15.31:0/3127950601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8109570 0x7f89d8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.323+0000 7f89dfd93640 1 --2- 172.21.15.31:0/3127950601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8103c30 0x7f89d8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.323+0000 7f89dfd93640 1 -- 172.21.15.31:0/3127950601 >> 172.21.15.31:0/3127950601 conn(0x7f89d807b8d0 msgr2=0x7f89d807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.324+0000 7f89dfd93640 1 -- 172.21.15.31:0/3127950601 shutdown_connections 2024-09-21T06:15:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.324+0000 7f89dfd93640 1 -- 172.21.15.31:0/3127950601 wait complete. 2024-09-21T06:15:50.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 Processor -- start 2024-09-21T06:15:50.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 -- start start 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 0x7f89d819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8109570 0x7f89d819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 0x7f89d819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8113ae0 con 0x7f89d8109570 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8113960 con 0x7f89d8103c30 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.325+0000 7f89dfd93640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8113c60 con 0x7f89d8109ec0 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 0x7f89d819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89de330640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 0x7f89d819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 0x7f89d819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60642/0 (socket says 172.21.15.31:60642) 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89de330640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 0x7f89d819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:56932/0 (socket says 172.21.15.31:56932) 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 -- 172.21.15.31:0/2983961968 learned_addr learned my addr 172.21.15.31:0/2983961968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 -- 172.21.15.31:0/2983961968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 msgr2=0x7f89d819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 0x7f89d819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 -- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8109570 msgr2=0x7f89d819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8109570 0x7f89d819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89ddb2f640 1 -- 172.21.15.31:0/2983961968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d81a01b0 con 0x7f89d8103c30 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.326+0000 7f89de330640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 0x7f89d819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.327+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 0x7f89d819a430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f89c800d980 tx=0x7f89c800de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.327+0000 7f89ceffd640 1 -- 172.21.15.31:0/2983961968 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8017030 con 0x7f89d8103c30 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.327+0000 7f89ceffd640 1 -- 172.21.15.31:0/2983961968 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c8004020 con 0x7f89d8103c30 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.327+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d81a04a0 con 0x7f89d8103c30 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.327+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81a0a30 con 0x7f89d8103c30 2024-09-21T06:15:50.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.327+0000 7f89ceffd640 1 -- 172.21.15.31:0/2983961968 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8005630 con 0x7f89d8103c30 2024-09-21T06:15:50.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.331+0000 7f89e6307740 1 Processor -- start 2024-09-21T06:15:50.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.331+0000 7f89e6307740 1 -- start start 2024-09-21T06:15:50.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89e6307740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b0b4350 0x560e1b45b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89e6307740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b45b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89e6307740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b45beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89e6307740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560e1b3d1e60 con 0x560e1b0b4350 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89e6307740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560e1b3c1290 con 0x560e1b093670 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89e6307740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560e1b3c50a0 con 0x560e1b08b490 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b45beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89de330640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b0b4350 0x560e1b45b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b45beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60648/0 (socket says 172.21.15.31:60648) 2024-09-21T06:15:50.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 -- 172.21.15.31:0/2818315355 learned_addr learned my addr 172.21.15.31:0/2818315355 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 -- 172.21.15.31:0/2818315355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 msgr2=0x560e1b45b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:15:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b45b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 -- 172.21.15.31:0/2818315355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b0b4350 msgr2=0x560e1b45b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.332+0000 7f89dd32e640 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b0b4350 0x560e1b45b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.333+0000 7f89dd32e640 1 -- 172.21.15.31:0/2818315355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1b11f930 con 0x560e1b093670 2024-09-21T06:15:50.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.333+0000 7f89dd32e640 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b45beb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f89d4009a20 tx=0x7f89d4009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23545c9261c21fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:50.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.333+0000 7f89ceffd640 1 -- 172.21.15.31:0/2983961968 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f89c80041d0 con 0x7f89d8103c30 2024-09-21T06:15:50.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.334+0000 7f89c3fff640 1 -- 172.21.15.31:0/2818315355 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d400ecb0 con 0x560e1b093670 2024-09-21T06:15:50.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.334+0000 7f89c3fff640 1 -- 172.21.15.31:0/2818315355 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d400ee50 con 0x560e1b093670 2024-09-21T06:15:50.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.334+0000 7f89c3fff640 1 -- 172.21.15.31:0/2818315355 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4004a90 con 0x560e1b093670 2024-09-21T06:15:50.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89ceffd640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89c40b6700 0x7f89c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89de330640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89c40b6700 0x7f89c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 -- 172.21.15.31:0/2818315355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 msgr2=0x560e1b45beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b45beb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f89d4009a20 tx=0x7f89d4009ee0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89ceffd640 1 -- 172.21.15.31:0/2983961968 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f89c80e6f60 con 0x7f89d8103c30 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 -- 172.21.15.31:0/2818315355 shutdown_connections 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b45beb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b45b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 --2- 172.21.15.31:0/2818315355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b0b4350 0x560e1b45b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.335+0000 7f89e6307740 1 -- 172.21.15.31:0/2818315355 >> 172.21.15.31:0/2818315355 conn(0x560e1b0a8740 msgr2=0x560e1b459320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.336+0000 7f89e6307740 1 -- 172.21.15.31:0/2818315355 shutdown_connections 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.336+0000 7f89e6307740 1 -- 172.21.15.31:0/2818315355 wait complete. 2024-09-21T06:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.336+0000 7f89de330640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89c40b6700 0x7f89c40b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89d0031040 tx=0x7f89d0009380 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.336+0000 7f89e6307740 1 -- start start 2024-09-21T06:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.336+0000 7f89e6307740 1 Processor -- start 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b467050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b467590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b0b4350 0x560e1b472f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560e1b222890 con 0x560e1b08b490 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560e1b35bd40 con 0x560e1b093670 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560e1b469c10 con 0x560e1b0b4350 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.338+0000 7f89e6307740 10 client.0 _ll_register_callbacks cb 0x560e1b3bfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b467590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89ddb2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b467050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b467590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60650/0 (socket says 172.21.15.31:60650) 2024-09-21T06:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 -- 172.21.15.31:0/753281010 learned_addr learned my addr 172.21.15.31:0/753281010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 -- 172.21.15.31:0/753281010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b0b4350 msgr2=0x560e1b472f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b0b4350 0x560e1b472f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 -- 172.21.15.31:0/753281010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 msgr2=0x560e1b467050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b467050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 -- 172.21.15.31:0/753281010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1b473580 con 0x560e1b093670 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.339+0000 7f89dd32e640 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b467590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f89d4005650 tx=0x7f89d4005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6feab7f1785ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.340+0000 7f89c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.340+0000 7f89c17fa640 1 -- 172.21.15.31:0/753281010 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4004a10 con 0x560e1b093670 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.340+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89d400ee50 con 0x560e1b093670 2024-09-21T06:15:50.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.340+0000 7f89c17fa640 1 -- 172.21.15.31:0/753281010 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f89d4004bb0 con 0x560e1b093670 2024-09-21T06:15:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.351+0000 7f89c17fa640 1 -- 172.21.15.31:0/753281010 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f89d4004dd0 con 0x560e1b093670 2024-09-21T06:15:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.356+0000 7f89e6307740 10 client.26117 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:50.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.356+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560e1b475e90 con 0x560e1b093670 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.358+0000 7f89c17fa640 1 -- 172.21.15.31:0/753281010 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d4023b30 con 0x560e1b093670 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.358+0000 7f89c17fa640 1 -- 172.21.15.31:0/753281010 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f89d4018980 con 0x560e1b093670 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.360+0000 7f89e6307740 10 client.26117 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.360+0000 7f89e6307740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.360+0000 7f89e6307740 20 client.26117 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.360+0000 7f89e6307740 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b072cd0 0x560e1b4791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.360+0000 7f89e6307740 4 client.26117 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:50.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.360+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f89d400ee50 con 0x560e1b072cd0 2024-09-21T06:15:50.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.361+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b072cd0 0x560e1b4791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.365+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b072cd0 0x560e1b4791f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1e0fca6cc6e835 server_cookie=96c3822e2023917c in_seq=0 out_seq=0 2024-09-21T06:15:50.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.365+0000 7f89c17fa640 10 client.26117 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.368+0000 7f89c17fa640 1 -- 172.21.15.31:0/753281010 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f89d400ee50 con 0x560e1b072cd0 2024-09-21T06:15:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.368+0000 7f89c17fa640 10 client.26117 handle_command_reply: tid=0 2024-09-21T06:15:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.368+0000 7f89e6307740 1 client.26117 shutdown 2024-09-21T06:15:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.369+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 msgr2=0x560e1b467590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.369+0000 7f89e6307740 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b467590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f89d4005650 tx=0x7f89d4005250 comp rx=0 tx=0).stop 2024-09-21T06:15:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 shutdown_connections 2024-09-21T06:15:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b072cd0 0x560e1b4791f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b0b4350 0x560e1b472f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b467590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 --2- 172.21.15.31:0/753281010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b467050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89c17fa640 0 client.26117 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:50.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 >> 172.21.15.31:0/753281010 conn(0x560e1b0a8740 msgr2=0x560e1b45e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:50.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 shutdown_connections 2024-09-21T06:15:50.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.371+0000 7f89e6307740 1 -- 172.21.15.31:0/753281010 wait complete. 2024-09-21T06:15:50.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.372+0000 7f89e6307740 20 client.26117 trim_cache size 0 max 16384 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.410+0000 7f89e6307740 1 Processor -- start 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.410+0000 7f89e6307740 1 -- start start 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.410+0000 7f89e6307740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b072cd0 0x560e1b460c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.410+0000 7f89e6307740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b458860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89e6307740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b458da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89e6307740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc00c740 con 0x560e1b08b490 2024-09-21T06:15:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89e6307740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc00efc0 con 0x560e1b093670 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89e6307740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc00cdb0 con 0x560e1b072cd0 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b458860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b458860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55684/0 (socket says 172.21.15.31:55684) 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 -- 172.21.15.31:0/3240219378 learned_addr learned my addr 172.21.15.31:0/3240219378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89de330640 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b458da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b072cd0 0x560e1b460c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 -- 172.21.15.31:0/3240219378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b072cd0 msgr2=0x560e1b460c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b072cd0 0x560e1b460c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 -- 172.21.15.31:0/3240219378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 msgr2=0x560e1b458da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b458da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 -- 172.21.15.31:0/3240219378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1b465db0 con 0x560e1b08b490 2024-09-21T06:15:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.411+0000 7f89dd32e640 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b458860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f89d4005ec0 tx=0x7f89d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b70e971b621a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:50.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.412+0000 7f89c37fe640 1 -- 172.21.15.31:0/3240219378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4039070 con 0x560e1b08b490 2024-09-21T06:15:50.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.412+0000 7f89c37fe640 1 -- 172.21.15.31:0/3240219378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d40044e0 con 0x560e1b08b490 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.412+0000 7f89c37fe640 1 -- 172.21.15.31:0/3240219378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d4010a90 con 0x560e1b08b490 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 -- 172.21.15.31:0/3240219378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 msgr2=0x560e1b458860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b458860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f89d4005ec0 tx=0x7f89d4005fd0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 -- 172.21.15.31:0/3240219378 shutdown_connections 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b458da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b08b490 0x560e1b458860 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 --2- 172.21.15.31:0/3240219378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b072cd0 0x560e1b460c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 -- 172.21.15.31:0/3240219378 >> 172.21.15.31:0/3240219378 conn(0x560e1b0a8740 msgr2=0x560e1b460540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 -- 172.21.15.31:0/3240219378 shutdown_connections 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.413+0000 7f89e6307740 1 -- 172.21.15.31:0/3240219378 wait complete. 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.414+0000 7f89e6307740 1 -- start start 2024-09-21T06:15:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.414+0000 7f89e6307740 1 Processor -- start 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.414+0000 7f89e6307740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b072cd0 0x560e1b46b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.414+0000 7f89e6307740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b46bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.414+0000 7f89e6307740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b4e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89e6307740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc00d480 con 0x560e1b072cd0 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89e6307740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc00e8f0 con 0x560e1b093670 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89e6307740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc00db50 con 0x560e1b08b490 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89e6307740 10 client.0 _ll_register_callbacks cb 0x7f89d4005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b4e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89ddb2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b072cd0 0x560e1b46b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b4e2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60670/0 (socket says 172.21.15.31:60670) 2024-09-21T06:15:50.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 -- 172.21.15.31:0/3880595098 learned_addr learned my addr 172.21.15.31:0/3880595098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89dd32e640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b46bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 -- 172.21.15.31:0/3880595098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 msgr2=0x560e1b46bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b46bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 -- 172.21.15.31:0/3880595098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b072cd0 msgr2=0x560e1b46b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b072cd0 0x560e1b46b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 -- 172.21.15.31:0/3880595098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1b4e3010 con 0x560e1b093670 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b072cd0 0x560e1b46b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:50.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89de330640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b4e2a80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009be0 tx=0x7f89d002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=168fe981aa175e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:50.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.415+0000 7f89c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:50.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.417+0000 7f89c17fa640 1 -- 172.21.15.31:0/3880595098 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0036040 con 0x560e1b093670 2024-09-21T06:15:50.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.417+0000 7f89c17fa640 1 -- 172.21.15.31:0/3880595098 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f89d0004040 con 0x560e1b093670 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.417+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89d40044e0 con 0x560e1b093670 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89c17fa640 1 -- 172.21.15.31:0/3880595098 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f89d0004240 con 0x560e1b093670 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 10 client.26123 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560e1b4e4290 con 0x560e1b093670 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89c17fa640 1 -- 172.21.15.31:0/3880595098 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89d40044e0 con 0x560e1b093670 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89c17fa640 1 -- 172.21.15.31:0/3880595098 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f89d004c040 con 0x560e1b093670 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 10 client.26123 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 20 client.26123 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b0b4350 0x560e1b4e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 4 client.26123 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:15:50.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.418+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e1b3d3db0 con 0x560e1b0b4350 2024-09-21T06:15:50.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.419+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b0b4350 0x560e1b4e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:50.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.419+0000 7f89c17fa640 10 client.26123 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:50.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.419+0000 7f89ddb2f640 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b0b4350 0x560e1b4e6a10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7b479bc74a5498 server_cookie=f3256bc2eef9bae9 in_seq=0 out_seq=0 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89c17fa640 1 -- 172.21.15.31:0/3880595098 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e1b3d3db0 con 0x560e1b0b4350 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89c17fa640 10 client.26123 handle_command_reply: tid=0 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89e6307740 1 client.26123 shutdown 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 msgr2=0x560e1b4e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89e6307740 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b4e2a80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009be0 tx=0x7f89d002a6f0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 shutdown_connections 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.421+0000 7f89e6307740 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560e1b0b4350 0x560e1b4e6a10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89c17fa640 0 client.26123 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560e1b093670 0x560e1b4e2a80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560e1b08b490 0x560e1b46bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 1 --2- 172.21.15.31:0/3880595098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e1b072cd0 0x560e1b46b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 >> 172.21.15.31:0/3880595098 conn(0x560e1b0a8740 msgr2=0x560e1b45c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 shutdown_connections 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 1 -- 172.21.15.31:0/3880595098 wait complete. 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.422+0000 7f89e6307740 20 client.26123 trim_cache size 0 max 16384 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa", 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:15:50.425 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89c40b6700 msgr2=0x7f89c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89c40b6700 0x7f89c40b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89d0031040 tx=0x7f89d0009380 comp rx=0 tx=0).stop 2024-09-21T06:15:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 msgr2=0x7f89d819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 0x7f89d819a430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f89c800d980 tx=0x7f89c800de40 comp rx=0 tx=0).stop 2024-09-21T06:15:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 shutdown_connections 2024-09-21T06:15:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89d8109ec0 0x7f89d819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f89c40b6700 0x7f89c40b8c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89d8109570 0x7f89d819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 --2- 172.21.15.31:0/2983961968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89d8103c30 0x7f89d819a430 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 >> 172.21.15.31:0/2983961968 conn(0x7f89d807b8d0 msgr2=0x7f89d81054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 shutdown_connections 2024-09-21T06:15:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:50.423+0000 7f89dfd93640 1 -- 172.21.15.31:0/2983961968 wait complete. 2024-09-21T06:15:51.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:50 smithi031 ceph-mon[31237]: pgmap v447: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 738 op/s 2024-09-21T06:15:51.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:50 smithi110 ceph-mon[34550]: pgmap v447: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 738 op/s 2024-09-21T06:15:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:50 smithi107 ceph-mon[34380]: pgmap v447: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 738 op/s 2024-09-21T06:15:51.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:15:51.850 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:51 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:15:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:52.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:51 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:52.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:51 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:15:52.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:51 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:15:52.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:51 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:52.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:51 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:15:52.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:51 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:15:52.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:51 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:52.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:51 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:15:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.561+0000 7f7428fc0640 1 -- 172.21.15.31:0/1969701447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424113eb0 msgr2=0x7f74241142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 --2- 172.21.15.31:0/1969701447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424113eb0 0x7f74241142c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7414009900 tx=0x7f741402a410 comp rx=0 tx=0).stop 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 -- 172.21.15.31:0/1969701447 shutdown_connections 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 --2- 172.21.15.31:0/1969701447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424114800 0x7f742411b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 --2- 172.21.15.31:0/1969701447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424113eb0 0x7f74241142c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 --2- 172.21.15.31:0/1969701447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f742410e5c0 0x7f742410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 -- 172.21.15.31:0/1969701447 >> 172.21.15.31:0/1969701447 conn(0x7f742410a3c0 msgr2=0x7f742410c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 -- 172.21.15.31:0/1969701447 shutdown_connections 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.562+0000 7f7428fc0640 1 -- 172.21.15.31:0/1969701447 wait complete. 2024-09-21T06:15:52.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.564+0000 7f7428fc0640 1 Processor -- start 2024-09-21T06:15:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.564+0000 7f7428fc0640 1 -- start start 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.564+0000 7f7428fc0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f742410e5c0 0x7f7424072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.564+0000 7f7428fc0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 0x7f742406f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.564+0000 7f7428fc0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424114800 0x7f742406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7428fc0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f742411e4e0 con 0x7f7424114800 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7428fc0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f742411e360 con 0x7f742410e5c0 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7428fc0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f742411e660 con 0x7f7424113eb0 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 0x7f742406f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 0x7f742406f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:56976/0 (socket says 172.21.15.31:56976) 2024-09-21T06:15:52.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 -- 172.21.15.31:0/331524951 learned_addr learned my addr 172.21.15.31:0/331524951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f742259c640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f742410e5c0 0x7f7424072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 -- 172.21.15.31:0/331524951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f742410e5c0 msgr2=0x7f7424072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f742410e5c0 0x7f7424072dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 -- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424114800 msgr2=0x7f742406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424114800 0x7f742406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.565+0000 7f7421d9b640 1 -- 172.21.15.31:0/331524951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74241b5250 con 0x7f7424113eb0 2024-09-21T06:15:52.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.566+0000 7f7421d9b640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 0x7f742406f0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f741402aa00 tx=0x7f74140305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.566+0000 7f74137fe640 1 -- 172.21.15.31:0/331524951 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741403f030 con 0x7f7424113eb0 2024-09-21T06:15:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.566+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241b5420 con 0x7f7424113eb0 2024-09-21T06:15:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.566+0000 7f74137fe640 1 -- 172.21.15.31:0/331524951 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7414030e30 con 0x7f7424113eb0 2024-09-21T06:15:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.566+0000 7f74137fe640 1 -- 172.21.15.31:0/331524951 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414035070 con 0x7f7424113eb0 2024-09-21T06:15:52.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.566+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241b59b0 con 0x7f7424113eb0 2024-09-21T06:15:52.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.570+0000 7f74137fe640 1 -- 172.21.15.31:0/331524951 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7414042030 con 0x7f7424113eb0 2024-09-21T06:15:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.571+0000 7f74137fe640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f740c0b6730 0x7f740c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.571+0000 7f742f534740 1 Processor -- start 2024-09-21T06:15:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.571+0000 7f74137fe640 1 -- 172.21.15.31:0/331524951 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f74141081a0 con 0x7f7424113eb0 2024-09-21T06:15:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.571+0000 7f742259c640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f740c0b6730 0x7f740c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.571+0000 7f742f534740 1 -- start start 2024-09-21T06:15:52.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742f534740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be2e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742f534740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742f534740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2ea0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742f534740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f3be25fe50 con 0x55f3bdf21390 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742f534740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f3be24f280 con 0x55f3bdf191b0 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742f534740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f3be253090 con 0x55f3bdf42070 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be2e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2e9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60686/0 (socket says 172.21.15.31:60686) 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 -- 172.21.15.31:0/1244259406 learned_addr learned my addr 172.21.15.31:0/1244259406 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7422d9d640 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2ea0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f742259c640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f740c0b6730 0x7f740c0b8c40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7424072b20 tx=0x7f7418005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 -- 172.21.15.31:0/1244259406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 msgr2=0x55f3be2e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be2e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 -- 172.21.15.31:0/1244259406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 msgr2=0x55f3be2ea0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2ea0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.572+0000 7f7421d9b640 1 -- 172.21.15.31:0/1244259406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3bdfad650 con 0x55f3bdf191b0 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.573+0000 7f7421d9b640 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2e9b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74140cb410 tx=0x7f74140ced60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b13d4e999c3f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.574+0000 7f7410ff9640 1 -- 172.21.15.31:0/1244259406 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414049070 con 0x55f3bdf191b0 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.574+0000 7f7410ff9640 1 -- 172.21.15.31:0/1244259406 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74140d39b0 con 0x55f3bdf191b0 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.574+0000 7f7410ff9640 1 -- 172.21.15.31:0/1244259406 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74140cce20 con 0x55f3bdf191b0 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 -- 172.21.15.31:0/1244259406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 msgr2=0x55f3be2e9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2e9b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74140cb410 tx=0x7f74140ced60 comp rx=0 tx=0).stop 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 -- 172.21.15.31:0/1244259406 shutdown_connections 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2ea0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2e9b90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 --2- 172.21.15.31:0/1244259406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be2e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 -- 172.21.15.31:0/1244259406 >> 172.21.15.31:0/1244259406 conn(0x55f3bdf36460 msgr2=0x55f3be2e7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 -- 172.21.15.31:0/1244259406 shutdown_connections 2024-09-21T06:15:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.575+0000 7f742f534740 1 -- 172.21.15.31:0/1244259406 wait complete. 2024-09-21T06:15:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.576+0000 7f742f534740 1 -- start start 2024-09-21T06:15:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.576+0000 7f742f534740 1 Processor -- start 2024-09-21T06:15:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.576+0000 7f742f534740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.576+0000 7f742f534740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742f534740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be300dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742f534740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f3be0b0700 con 0x55f3bdf21390 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60688/0 (socket says 172.21.15.31:60688) 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742259c640 1 -- 172.21.15.31:0/3407635759 learned_addr learned my addr 172.21.15.31:0/3407635759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f3be1ea110 con 0x55f3bdf191b0 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f3be2f7bf0 con 0x55f3bdf42070 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f7422d9d640 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be300dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.577+0000 7f742f534740 10 client.0 _ll_register_callbacks cb 0x55f3be203990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7422d9d640 1 -- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 msgr2=0x55f3be2f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7422d9d640 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7422d9d640 1 -- 172.21.15.31:0/3407635759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 msgr2=0x55f3be2f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7422d9d640 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7422d9d640 1 -- 172.21.15.31:0/3407635759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3be217c40 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7422d9d640 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be300dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f741c0108d0 tx=0x7f741c010d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c9be84d6d1e3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7409ffb640 1 -- 172.21.15.31:0/3407635759 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c017070 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74140d39b0 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.578+0000 7f7409ffb640 1 -- 172.21.15.31:0/3407635759 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f741c004d20 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f7409ffb640 1 -- 172.21.15.31:0/3407635759 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f741c00ea40 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 10 client.26167 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f3be303900 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f7409ffb640 1 -- 172.21.15.31:0/3407635759 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74140d39b0 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f7409ffb640 1 -- 172.21.15.31:0/3407635759 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f741c004400 con 0x55f3bdf42070 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 10 client.26167 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 20 client.26167 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bdf009f0 0x55f3be306e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 4 client.26167 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.579+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3be261da0 con 0x55f3bdf009f0 2024-09-21T06:15:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.580+0000 7f742259c640 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bdf009f0 0x55f3be306e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.581+0000 7f742259c640 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bdf009f0 0x55f3be306e40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e035e1bd621aafc0 server_cookie=b32f8b9da032b904 in_seq=0 out_seq=0 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.581+0000 7f7409ffb640 10 client.26167 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.581+0000 7f7409ffb640 1 -- 172.21.15.31:0/3407635759 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3be261da0 con 0x55f3bdf009f0 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.581+0000 7f7409ffb640 10 client.26167 handle_command_reply: tid=0 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.581+0000 7f742f534740 1 client.26167 shutdown 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 msgr2=0x55f3be300dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be300dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f741c0108d0 tx=0x7f741c010d90 comp rx=0 tx=0).stop 2024-09-21T06:15:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 shutdown_connections 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bdf009f0 0x55f3be306e40 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf42070 0x55f3be300dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf21390 0x55f3be2f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 --2- 172.21.15.31:0/3407635759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf191b0 0x55f3be2f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 >> 172.21.15.31:0/3407635759 conn(0x55f3bdf36460 msgr2=0x55f3be2ea680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f7409ffb640 0 client.26167 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 shutdown_connections 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 1 -- 172.21.15.31:0/3407635759 wait complete. 2024-09-21T06:15:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.582+0000 7f742f534740 20 client.26167 trim_cache size 0 max 16384 2024-09-21T06:15:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 Processor -- start 2024-09-21T06:15:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 -- start start 2024-09-21T06:15:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be300c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be301190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f74241bc590 con 0x55f3bdf191b0 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74241bee10 con 0x55f3bdf42070 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.609+0000 7f742f534740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74241bcc00 con 0x55f3bdf21390 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be301190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be301190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60694/0 (socket says 172.21.15.31:60694) 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 -- 172.21.15.31:0/51619264 learned_addr learned my addr 172.21.15.31:0/51619264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7421d9b640 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be300c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f742259c640 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 -- 172.21.15.31:0/51619264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 msgr2=0x55f3be300c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be300c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 -- 172.21.15.31:0/51619264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 msgr2=0x55f3be2f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 -- 172.21.15.31:0/51619264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3be2f5ea0 con 0x55f3bdf42070 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7421d9b640 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be300c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.610+0000 7f7422d9d640 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be301190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f741c004850 tx=0x7f741c0049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e6ebfad9819e394 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.611+0000 7f740bfff640 1 -- 172.21.15.31:0/51619264 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c032070 con 0x55f3bdf42070 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.611+0000 7f740bfff640 1 -- 172.21.15.31:0/51619264 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f741c01f020 con 0x55f3bdf42070 2024-09-21T06:15:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.611+0000 7f740bfff640 1 -- 172.21.15.31:0/51619264 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c017050 con 0x55f3bdf42070 2024-09-21T06:15:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 -- 172.21.15.31:0/51619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 msgr2=0x55f3be301190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be301190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f741c004850 tx=0x7f741c0049c0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 -- 172.21.15.31:0/51619264 shutdown_connections 2024-09-21T06:15:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be301190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be300c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 --2- 172.21.15.31:0/51619264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.612+0000 7f742f534740 1 -- 172.21.15.31:0/51619264 >> 172.21.15.31:0/51619264 conn(0x55f3bdf36460 msgr2=0x55f3be2f6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.613+0000 7f742f534740 1 -- 172.21.15.31:0/51619264 shutdown_connections 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.613+0000 7f742f534740 1 -- 172.21.15.31:0/51619264 wait complete. 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.613+0000 7f742f534740 1 -- start start 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.613+0000 7f742f534740 1 Processor -- start 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.613+0000 7f742f534740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742f534740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be2f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742f534740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be2f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742f534740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f74241bd2d0 con 0x55f3bdf191b0 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742f534740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74241be740 con 0x55f3bdf42070 2024-09-21T06:15:52.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742f534740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f74241bd9a0 con 0x55f3bdf21390 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742f534740 10 client.0 _ll_register_callbacks cb 0x7f741c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be2f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be2f92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:57000/0 (socket says 172.21.15.31:57000) 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f742259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55736/0 (socket says 172.21.15.31:55736) 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 -- 172.21.15.31:0/59519603 learned_addr learned my addr 172.21.15.31:0/59519603 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7422d9d640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be2f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 -- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 msgr2=0x55f3be2f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be2f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 -- 172.21.15.31:0/59519603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 msgr2=0x55f3be2f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 -- 172.21.15.31:0/59519603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3be370970 con 0x55f3bdf21390 2024-09-21T06:15:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7422d9d640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be2f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:52.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.614+0000 7f7421d9b640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be2f92e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74140cebd0 tx=0x7f74140306f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c5a64b313d26d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:52.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f7409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:52.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f7409ffb640 1 -- 172.21.15.31:0/59519603 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414049040 con 0x55f3bdf21390 2024-09-21T06:15:52.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f7409ffb640 1 -- 172.21.15.31:0/59519603 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f741407e1d0 con 0x55f3bdf21390 2024-09-21T06:15:52.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741c01f020 con 0x55f3bdf21390 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f7409ffb640 1 -- 172.21.15.31:0/59519603 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f74140d39b0 con 0x55f3bdf21390 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f742f534740 10 client.26173 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.615+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f3be371be0 con 0x55f3bdf21390 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f7409ffb640 1 -- 172.21.15.31:0/59519603 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f741c01f020 con 0x55f3bdf21390 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f7409ffb640 1 -- 172.21.15.31:0/59519603 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f74140d3040 con 0x55f3bdf21390 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f742f534740 10 client.26173 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f742f534740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f742f534740 20 client.26173 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f742f534740 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bded5da0 0x55f3be374160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f742f534740 4 client.26173 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:15:52.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.616+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3be25d850 con 0x55f3bded5da0 2024-09-21T06:15:52.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.617+0000 7f742259c640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bded5da0 0x55f3be374160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:52.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742259c640 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bded5da0 0x55f3be374160 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953cbc7137fef1ef server_cookie=bcf5268b119955d7 in_seq=0 out_seq=0 2024-09-21T06:15:52.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f7409ffb640 10 client.26173 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:52.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f7409ffb640 1 -- 172.21.15.31:0/59519603 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55f3be25d850 con 0x55f3bded5da0 2024-09-21T06:15:52.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f7409ffb640 10 client.26173 handle_command_reply: tid=0 2024-09-21T06:15:52.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 client.26173 shutdown 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 msgr2=0x55f3be2f92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be2f92e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74140cebd0 tx=0x7f74140306f0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 shutdown_connections 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f3bded5da0 0x55f3be374160 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f7409ffb640 0 client.26173 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f3bdf42070 0x55f3be2f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.618+0000 7f742f534740 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f3bdf21390 0x55f3be2f92e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.619+0000 7f742f534740 1 --2- 172.21.15.31:0/59519603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f3bdf191b0 0x55f3be2f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.619+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 >> 172.21.15.31:0/59519603 conn(0x55f3bdf36460 msgr2=0x55f3be2f6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.619+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 shutdown_connections 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.619+0000 7f742f534740 1 -- 172.21.15.31:0/59519603 wait complete. 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.619+0000 7f742f534740 20 client.26173 trim_cache size 0 max 16384 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:15:52.623 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.624 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.625 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.626 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:15:52.627 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.628 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:15:52.630 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:15:52.631 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f740c0b6730 msgr2=0x7f740c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f740c0b6730 0x7f740c0b8c40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7424072b20 tx=0x7f7418005f90 comp rx=0 tx=0).stop 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 msgr2=0x7f742406f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 0x7f742406f0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f741402aa00 tx=0x7f74140305d0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 shutdown_connections 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f740c0b6730 0x7f740c0b8c40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7424114800 0x7f742406f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7424113eb0 0x7f742406f0b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 --2- 172.21.15.31:0/331524951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f742410e5c0 0x7f7424072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 >> 172.21.15.31:0/331524951 conn(0x7f742410a3c0 msgr2=0x7f7424111d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.620+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 shutdown_connections 2024-09-21T06:15:52.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:52.621+0000 7f7428fc0640 1 -- 172.21.15.31:0/331524951 wait complete. 2024-09-21T06:15:53.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:52 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:53.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:52 smithi031 ceph-mon[31237]: pgmap v448: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-21T06:15:53.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:52 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:15:53.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:52 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:53.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:52 smithi110 ceph-mon[34550]: pgmap v448: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-21T06:15:53.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:52 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:15:53.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:52 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:15:53.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:52 smithi107 ceph-mon[34380]: pgmap v448: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-21T06:15:53.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:52 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:15:53.898 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:15:53.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:15:54.296 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:54.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 -- 172.21.15.31:0/2999681518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb01025d0 msgr2=0x7f7eb01029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 --2- 172.21.15.31:0/2999681518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb01025d0 0x7f7eb01029e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009900 tx=0x7f7ea802a480 comp rx=0 tx=0).stop 2024-09-21T06:15:54.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 -- 172.21.15.31:0/2999681518 shutdown_connections 2024-09-21T06:15:54.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 --2- 172.21.15.31:0/2999681518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0110180 0x7f7eb01125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 --2- 172.21.15.31:0/2999681518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb0102f20 0x7f7eb010fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 --2- 172.21.15.31:0/2999681518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb01025d0 0x7f7eb01029e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 -- 172.21.15.31:0/2999681518 >> 172.21.15.31:0/2999681518 conn(0x7f7eb0079730 msgr2=0x7f7eb0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 -- 172.21.15.31:0/2999681518 shutdown_connections 2024-09-21T06:15:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.906+0000 7f7eb862e640 1 -- 172.21.15.31:0/2999681518 wait complete. 2024-09-21T06:15:54.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 Processor -- start 2024-09-21T06:15:54.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 -- start start 2024-09-21T06:15:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 0x7f7eb019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb0102f20 0x7f7eb019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0110180 0x7f7eb01a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0115cc0 con 0x7f7eb0102f20 2024-09-21T06:15:54.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0115b40 con 0x7f7eb01025d0 2024-09-21T06:15:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.907+0000 7f7eb862e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0115e40 con 0x7f7eb0110180 2024-09-21T06:15:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb5bc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb0102f20 0x7f7eb019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 0x7f7eb019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 0x7f7eb019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60712/0 (socket says 172.21.15.31:60712) 2024-09-21T06:15:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb5bc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb0102f20 0x7f7eb019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55742/0 (socket says 172.21.15.31:55742) 2024-09-21T06:15:54.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 -- 172.21.15.31:0/3439493807 learned_addr learned my addr 172.21.15.31:0/3439493807 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 -- 172.21.15.31:0/3439493807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0110180 msgr2=0x7f7eb01a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0110180 0x7f7eb01a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 -- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb0102f20 msgr2=0x7f7eb019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb0102f20 0x7f7eb019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 -- 172.21.15.31:0/3439493807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01a2620 con 0x7f7eb01025d0 2024-09-21T06:15:54.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.908+0000 7f7eb63ca640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 0x7f7eb019c6b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005ab0 tx=0x7f7ea8003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.909+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea803f070 con 0x7f7eb01025d0 2024-09-21T06:15:54.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.909+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea8031b50 con 0x7f7eb01025d0 2024-09-21T06:15:54.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.909+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8035030 con 0x7f7eb01025d0 2024-09-21T06:15:54.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.911+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01a28b0 con 0x7f7eb01025d0 2024-09-21T06:15:54.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.912+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01a2d10 con 0x7f7eb01025d0 2024-09-21T06:15:54.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.913+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb01029e0 con 0x7f7eb01025d0 2024-09-21T06:15:54.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.915+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7ea8042070 con 0x7f7eb01025d0 2024-09-21T06:15:54.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.916+0000 7f7eaf7fe640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e980b6880 0x7f7e980b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:54.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.916+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7ea8108520 con 0x7f7eb01025d0 2024-09-21T06:15:54.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.916+0000 7f7eb5bc9640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e980b6880 0x7f7e980b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.917+0000 7f7eb5bc9640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e980b6880 0x7f7e980b8d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0104050 tx=0x7f7ea400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:54.919+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ea8049070 con 0x7f7eb01025d0 2024-09-21T06:15:55.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:54 smithi031 ceph-mon[31237]: pgmap v449: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 659 op/s 2024-09-21T06:15:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 255.0 s, 2834.2 tps, lat 35.459 ms stddev 41.412 2024-09-21T06:15:55.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:54 smithi110 ceph-mon[34550]: pgmap v449: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 659 op/s 2024-09-21T06:15:55.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:54 smithi107 ceph-mon[34380]: pgmap v449: 97 pgs: 97 active+clean; 3.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.9 MiB/s wr, 659 op/s 2024-09-21T06:15:55.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.339+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7eb01a9330 con 0x7f7eb01025d0 2024-09-21T06:15:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.345+0000 7f7eaf7fe640 1 -- 172.21.15.31:0/3439493807 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f7ea80cead0 con 0x7f7eb01025d0 2024-09-21T06:15:55.347 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:15:55.347 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:15:55.347 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:15:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.351+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e980b6880 msgr2=0x7f7e980b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.351+0000 7f7eb862e640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e980b6880 0x7f7e980b8d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0104050 tx=0x7f7ea400b410 comp rx=0 tx=0).stop 2024-09-21T06:15:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.351+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 msgr2=0x7f7eb019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.351+0000 7f7eb862e640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 0x7f7eb019c6b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005ab0 tx=0x7f7ea8003c20 comp rx=0 tx=0).stop 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 shutdown_connections 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7eb0110180 0x7f7eb01a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7e980b6880 0x7f7e980b8d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7eb0102f20 0x7f7eb019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 --2- 172.21.15.31:0/3439493807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eb01025d0 0x7f7eb019c6b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 >> 172.21.15.31:0/3439493807 conn(0x7f7eb0079730 msgr2=0x7f7eb0109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 shutdown_connections 2024-09-21T06:15:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:55.354+0000 7f7eb862e640 1 -- 172.21.15.31:0/3439493807 wait complete. 2024-09-21T06:15:56.104 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:55 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3439493807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:56.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:15:56.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:55 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3439493807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:56.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:55 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3439493807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:15:56.621 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:15:57.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:56 smithi031 ceph-mon[31237]: pgmap v450: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 9.6 MiB/s wr, 967 op/s 2024-09-21T06:15:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 -- 172.21.15.31:0/2936581654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 msgr2=0x7fc0a010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 --2- 172.21.15.31:0/2936581654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 0x7fc0a010f550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009860 tx=0x7fc09c02a340 comp rx=0 tx=0).stop 2024-09-21T06:15:57.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 -- 172.21.15.31:0/2936581654 shutdown_connections 2024-09-21T06:15:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 --2- 172.21.15.31:0/2936581654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 0x7fc0a010f550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 --2- 172.21.15.31:0/2936581654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a01083d0 0x7fc0a01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 --2- 172.21.15.31:0/2936581654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a0102ae0 0x7fc0a0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 -- 172.21.15.31:0/2936581654 >> 172.21.15.31:0/2936581654 conn(0x7fc0a0079790 msgr2=0x7fc0a0079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 -- 172.21.15.31:0/2936581654 shutdown_connections 2024-09-21T06:15:57.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.221+0000 7fc0a8032640 1 -- 172.21.15.31:0/2936581654 wait complete. 2024-09-21T06:15:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 Processor -- start 2024-09-21T06:15:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 -- start start 2024-09-21T06:15:57.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a0102ae0 0x7fc0a010b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a01083d0 0x7fc0a0109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 0x7fc0a0109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a01129e0 con 0x7fc0a0102ae0 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0112860 con 0x7fc0a01083d0 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.222+0000 7fc0a8032640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0112b60 con 0x7fc0a0108d20 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a5dce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a0102ae0 0x7fc0a010b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a5dce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a0102ae0 0x7fc0a010b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55760/0 (socket says 172.21.15.31:55760) 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a5dce640 1 -- 172.21.15.31:0/3341221757 learned_addr learned my addr 172.21.15.31:0/3341221757 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a01083d0 0x7fc0a0109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 0x7fc0a0109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a55cd640 1 -- 172.21.15.31:0/3341221757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 msgr2=0x7fc0a0109b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 0x7fc0a0109b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a55cd640 1 -- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a0102ae0 msgr2=0x7fc0a010b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a0102ae0 0x7fc0a010b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.223+0000 7fc0a55cd640 1 -- 172.21.15.31:0/3341221757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a010a3a0 con 0x7fc0a01083d0 2024-09-21T06:15:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.224+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a01083d0 0x7fc0a0109640 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0a006a890 tx=0x7fc09000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.226+0000 7fc096ffd640 1 -- 172.21.15.31:0/3341221757 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09000ccd0 con 0x7fc0a01083d0 2024-09-21T06:15:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.226+0000 7fc096ffd640 1 -- 172.21.15.31:0/3341221757 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0900035f0 con 0x7fc0a01083d0 2024-09-21T06:15:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.226+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a01ace40 con 0x7fc0a01083d0 2024-09-21T06:15:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.226+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a01ad290 con 0x7fc0a01083d0 2024-09-21T06:15:57.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.226+0000 7fc096ffd640 1 -- 172.21.15.31:0/3341221757 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090010690 con 0x7fc0a01083d0 2024-09-21T06:15:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.229+0000 7fc0ae5a6740 1 Processor -- start 2024-09-21T06:15:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 -- start start 2024-09-21T06:15:57.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b267e070 0x5650b2a256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a25be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 0x5650b2a26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650b299bea0 con 0x5650b267e070 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650b298b2d0 con 0x5650b26551b0 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650b298f0e0 con 0x5650b265d390 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc096ffd640 1 -- 172.21.15.31:0/3341221757 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc090010830 con 0x7fc0a01083d0 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 0x5650b2a26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 0x5650b2a26120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:57048/0 (socket says 172.21.15.31:57048) 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.230+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1268973638 learned_addr learned my addr 172.21.15.31:0/1268973638 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b267e070 0x5650b2a256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1268973638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 msgr2=0x5650b2a25be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a25be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1268973638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b267e070 msgr2=0x5650b2a256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b267e070 0x5650b2a256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1268973638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b26e9650 con 0x5650b265d390 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b267e070 0x5650b2a256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.231+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 0x5650b2a26120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0900e4bf0 tx=0x7fc0900ed9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a467b3be9711903 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.232+0000 7fc08bfff640 1 -- 172.21.15.31:0/1268973638 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900fa070 con 0x5650b265d390 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.232+0000 7fc08bfff640 1 -- 172.21.15.31:0/1268973638 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0900e7db0 con 0x5650b265d390 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.232+0000 7fc08bfff640 1 -- 172.21.15.31:0/1268973638 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900f59e0 con 0x5650b265d390 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.232+0000 7fc096ffd640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc08c0b67d0 0x7fc08c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.232+0000 7fc096ffd640 1 -- 172.21.15.31:0/3341221757 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc0900e74f0 con 0x7fc0a01083d0 2024-09-21T06:15:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc08c0b67d0 0x7fc08c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1268973638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 msgr2=0x5650b2a26120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 0x5650b2a26120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc0900e4bf0 tx=0x7fc0900ed9d0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1268973638 shutdown_connections 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b265d390 0x5650b2a26120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a25be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1268973638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b267e070 0x5650b2a256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.233+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1268973638 >> 172.21.15.31:0/1268973638 conn(0x5650b2672460 msgr2=0x5650b2a23590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.234+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1268973638 shutdown_connections 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.234+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1268973638 wait complete. 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.234+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc08c0b67d0 0x7fc08c0b8ce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009830 tx=0x7fc09c02c040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.234+0000 7fc0ae5a6740 1 -- start start 2024-09-21T06:15:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.234+0000 7fc0ae5a6740 1 Processor -- start 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.235+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 0x5650b2a31320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a3d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650b27ec700 con 0x5650b263c9f0 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650b2926160 con 0x5650b26551b0 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650b2a33e80 con 0x5650b267e070 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0ae5a6740 10 client.0 _ll_register_callbacks cb 0x5650b293f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 0x5650b2a31320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 0x5650b2a31320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:57060/0 (socket says 172.21.15.31:57060) 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1159976168 learned_addr learned my addr 172.21.15.31:0/1159976168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a3d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 msgr2=0x5650b2a30de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1159976168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 msgr2=0x5650b2a3d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a3d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 -- 172.21.15.31:0/1159976168 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b2953c90 con 0x5650b267e070 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 0x5650b2a31320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc0900aa5f0 tx=0x7fc090009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e040790fd099a524 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.236+0000 7fc0897fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0897fa640 1 -- 172.21.15.31:0/1159976168 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900a99d0 con 0x5650b267e070 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0897fa640 1 -- 172.21.15.31:0/1159976168 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc0900a95f0 con 0x5650b267e070 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0900e7db0 con 0x5650b267e070 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0897fa640 1 -- 172.21.15.31:0/1159976168 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc0900acaf0 con 0x5650b267e070 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0ae5a6740 10 client.26188 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5650b2a3fc50 con 0x5650b267e070 2024-09-21T06:15:57.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0897fa640 1 -- 172.21.15.31:0/1159976168 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0900e7db0 con 0x5650b267e070 2024-09-21T06:15:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0897fa640 1 -- 172.21.15.31:0/1159976168 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc0900b6910 con 0x5650b267e070 2024-09-21T06:15:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0ae5a6740 10 client.26188 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0ae5a6740 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.237+0000 7fc0ae5a6740 20 client.26188 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.238+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b2611da0 0x5650b2a431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.238+0000 7fc0ae5a6740 4 client.26188 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.238+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b299ddf0 con 0x5650b2611da0 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.238+0000 7fc0a5dce640 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b2611da0 0x5650b2a431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.239+0000 7fc0a5dce640 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b2611da0 0x5650b2a431e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a911ba837193ef20 server_cookie=97314a966d6cf9a2 in_seq=0 out_seq=0 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.239+0000 7fc0897fa640 10 client.26188 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.239+0000 7fc0897fa640 1 -- 172.21.15.31:0/1159976168 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650b299ddf0 con 0x5650b2611da0 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.239+0000 7fc0897fa640 10 client.26188 handle_command_reply: tid=0 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 client.26188 shutdown 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 msgr2=0x5650b2a31320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 0x5650b2a31320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc0900aa5f0 tx=0x7fc090009f90 comp rx=0 tx=0).stop 2024-09-21T06:15:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 shutdown_connections 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b2611da0 0x5650b2a431e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0897fa640 0 client.26188 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b267e070 0x5650b2a31320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/1159976168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a30de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 >> 172.21.15.31:0/1159976168 conn(0x5650b2672460 msgr2=0x5650b2a28830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 shutdown_connections 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/1159976168 wait complete. 2024-09-21T06:15:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.240+0000 7fc0ae5a6740 20 client.26188 trim_cache size 0 max 16384 2024-09-21T06:15:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.260+0000 7fc0ae5a6740 1 Processor -- start 2024-09-21T06:15:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.261+0000 7fc0ae5a6740 1 -- start start 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.261+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b2611da0 0x5650b2a32cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.261+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b263c9f0 0x5650b2a26de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.261+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 0x5650b2a22bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc08400c6d0 con 0x5650b26551b0 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 0x5650b2a22bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 0x5650b2a22bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55776/0 (socket says 172.21.15.31:55776) 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b263c9f0 0x5650b2a26de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 -- 172.21.15.31:0/910060231 learned_addr learned my addr 172.21.15.31:0/910060231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc08400ef50 con 0x5650b2611da0 2024-09-21T06:15:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc08400cd40 con 0x5650b263c9f0 2024-09-21T06:15:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a5dce640 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b2611da0 0x5650b2a32cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 -- 172.21.15.31:0/910060231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b263c9f0 msgr2=0x5650b2a26de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b263c9f0 0x5650b2a26de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 -- 172.21.15.31:0/910060231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b2611da0 msgr2=0x5650b2a32cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b2611da0 0x5650b2a32cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a65cf640 1 -- 172.21.15.31:0/910060231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b2a2f2e0 con 0x5650b26551b0 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.262+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b263c9f0 0x5650b2a26de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.263+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 0x5650b2a22bc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005eb0 tx=0x7fc09c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef79cb288c607e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.263+0000 7fc08b7fe640 1 -- 172.21.15.31:0/910060231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c037040 con 0x5650b26551b0 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.263+0000 7fc08b7fe640 1 -- 172.21.15.31:0/910060231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c0307d0 con 0x5650b26551b0 2024-09-21T06:15:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.263+0000 7fc08b7fe640 1 -- 172.21.15.31:0/910060231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c032880 con 0x5650b26551b0 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.264+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 msgr2=0x5650b2a22bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.264+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 0x5650b2a22bc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc09c005eb0 tx=0x7fc09c03d000 comp rx=0 tx=0).stop 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.264+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 shutdown_connections 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.265+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b26551b0 0x5650b2a22bc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.265+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b263c9f0 0x5650b2a26de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.265+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/910060231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b2611da0 0x5650b2a32cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.265+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 >> 172.21.15.31:0/910060231 conn(0x5650b2672460 msgr2=0x5650b2a266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.265+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 shutdown_connections 2024-09-21T06:15:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.265+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/910060231 wait complete. 2024-09-21T06:15:57.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.266+0000 7fc0ae5a6740 1 -- start start 2024-09-21T06:15:57.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.266+0000 7fc0ae5a6740 1 Processor -- start 2024-09-21T06:15:57.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.266+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b2611da0 0x5650b2a39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a34de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0ae5a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a35320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc08400d410 con 0x5650b263c9f0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc08400e880 con 0x5650b26551b0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0ae5a6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc08400dae0 con 0x5650b2611da0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0ae5a6740 10 client.0 _ll_register_callbacks cb 0x7fc0900f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a34de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a55cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a34de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55782/0 (socket says 172.21.15.31:55782) 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a55cd640 1 -- 172.21.15.31:0/3233225924 learned_addr learned my addr 172.21.15.31:0/3233225924 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a35320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a5dce640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b2611da0 0x5650b2a39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a65cf640 1 -- 172.21.15.31:0/3233225924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b2611da0 msgr2=0x5650b2a39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b2611da0 0x5650b2a39930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a65cf640 1 -- 172.21.15.31:0/3233225924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 msgr2=0x5650b2a34de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:15:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a34de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a65cf640 1 -- 172.21.15.31:0/3233225924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b2a35a90 con 0x5650b26551b0 2024-09-21T06:15:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.267+0000 7fc0a55cd640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a34de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:15:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.268+0000 7fc0a65cf640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a35320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc09c030d90 tx=0x7fc09c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3884f1ff5e446fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:15:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.268+0000 7fc0897fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:15:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.269+0000 7fc0897fa640 1 -- 172.21.15.31:0/3233225924 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c059020 con 0x5650b26551b0 2024-09-21T06:15:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.269+0000 7fc0897fa640 1 -- 172.21.15.31:0/3233225924 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc09c003ca0 con 0x5650b26551b0 2024-09-21T06:15:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.269+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc09c0307d0 con 0x5650b26551b0 2024-09-21T06:15:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.270+0000 7fc0897fa640 1 -- 172.21.15.31:0/3233225924 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc09c037030 con 0x5650b26551b0 2024-09-21T06:15:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.270+0000 7fc0ae5a6740 10 client.26159 fetch_fsmap learned FSMap version 19 2024-09-21T06:15:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.270+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5650b2aadac0 con 0x5650b26551b0 2024-09-21T06:15:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.270+0000 7fc0897fa640 1 -- 172.21.15.31:0/3233225924 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c0307d0 con 0x5650b26551b0 2024-09-21T06:15:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0897fa640 1 -- 172.21.15.31:0/3233225924 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc09c04a410 con 0x5650b26551b0 2024-09-21T06:15:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0ae5a6740 10 client.26159 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:15:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0ae5a6740 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:15:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0ae5a6740 20 client.26159 populate_metadata read hostname 'smithi031' 2024-09-21T06:15:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b267e070 0x5650b2ab0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:15:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0ae5a6740 4 client.26159 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:15:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.271+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650b29998a0 con 0x5650b267e070 2024-09-21T06:15:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.272+0000 7fc0a5dce640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b267e070 0x5650b2ab0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.272+0000 7fc0a5dce640 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b267e070 0x5650b2ab0150 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff68e6268250631 server_cookie=4d3ba11ed6b0c5c8 in_seq=0 out_seq=0 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.272+0000 7fc0897fa640 10 client.26159 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.272+0000 7fc0897fa640 1 -- 172.21.15.31:0/3233225924 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650b29998a0 con 0x5650b267e070 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.272+0000 7fc0897fa640 10 client.26159 handle_command_reply: tid=0 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.272+0000 7fc0ae5a6740 1 client.26159 shutdown 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 msgr2=0x5650b2a35320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a35320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc09c030d90 tx=0x7fc09c0038a0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 shutdown_connections 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b267e070 0x5650b2ab0150 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b26551b0 0x5650b2a35320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b263c9f0 0x5650b2a34de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 --2- 172.21.15.31:0/3233225924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b2611da0 0x5650b2a39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 >> 172.21.15.31:0/3233225924 conn(0x5650b2672460 msgr2=0x5650b2a27040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.273+0000 7fc0897fa640 0 client.26159 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:15:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.274+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 shutdown_connections 2024-09-21T06:15:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.274+0000 7fc0ae5a6740 1 -- 172.21.15.31:0/3233225924 wait complete. 2024-09-21T06:15:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.274+0000 7fc0ae5a6740 20 client.26159 trim_cache size 0 max 16384 2024-09-21T06:15:57.276 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:15:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc08c0b67d0 msgr2=0x7fc08c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc08c0b67d0 0x7fc08c0b8ce0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009830 tx=0x7fc09c02c040 comp rx=0 tx=0).stop 2024-09-21T06:15:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a01083d0 msgr2=0x7fc0a0109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a01083d0 0x7fc0a0109640 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0a006a890 tx=0x7fc09000ee50 comp rx=0 tx=0).stop 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 shutdown_connections 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0a0108d20 0x7fc0a0109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0a01083d0 0x7fc0a0109640 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc08c0b67d0 0x7fc08c0b8ce0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 --2- 172.21.15.31:0/3341221757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0a0102ae0 0x7fc0a010b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 >> 172.21.15.31:0/3341221757 conn(0x7fc0a0079790 msgr2=0x7fc0a01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 shutdown_connections 2024-09-21T06:15:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:15:57.275+0000 7fc0a8032640 1 -- 172.21.15.31:0/3341221757 wait complete. 2024-09-21T06:15:57.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:56 smithi110 ceph-mon[34550]: pgmap v450: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 9.6 MiB/s wr, 967 op/s 2024-09-21T06:15:57.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:56 smithi107 ceph-mon[34380]: pgmap v450: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 9.6 MiB/s wr, 967 op/s 2024-09-21T06:15:59.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:15:58 smithi031 ceph-mon[31237]: pgmap v451: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-21T06:15:59.198 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:15:59.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:15:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:15:58 smithi110 ceph-mon[34550]: pgmap v451: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-21T06:15:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:15:58 smithi107 ceph-mon[34380]: pgmap v451: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-21T06:15:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 260.0 s, 3346.0 tps, lat 29.721 ms stddev 36.217 2024-09-21T06:16:00.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.233+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/501213170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 msgr2=0x7fa0f01061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.233+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/501213170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009900 tx=0x7fa0e002a410 comp rx=0 tx=0).stop 2024-09-21T06:16:00.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/501213170 shutdown_connections 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/501213170 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 0x7fa0f0112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/501213170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0f010b6f0 0x7fa0f010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/501213170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f01061c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/501213170 >> 172.21.15.31:0/501213170 conn(0x7fa0f01017a0 msgr2=0x7fa0f0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/501213170 shutdown_connections 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.234+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/501213170 wait complete. 2024-09-21T06:16:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.236+0000 7fa0f4e0f640 1 Processor -- start 2024-09-21T06:16:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.236+0000 7fa0f4e0f640 1 -- start start 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0f4e0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0f4e0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0f010b6f0 0x7fa0f019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0f4e0f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 0x7fa0f01a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0ee59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0ee59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f019c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46484/0 (socket says 172.21.15.31:46484) 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0f010b6f0 0x7fa0f019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0ee59c640 1 -- 172.21.15.31:0/2159930602 learned_addr learned my addr 172.21.15.31:0/2159930602 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0f4e0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f0115d50 con 0x7fa0f010b6f0 2024-09-21T06:16:00.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f0115bd0 con 0x7fa0f0105db0 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f0115ed0 con 0x7fa0f010c040 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.237+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 0x7fa0f01a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.238+0000 7fa0eed9d640 1 -- 172.21.15.31:0/2159930602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 msgr2=0x7fa0f019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.238+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.238+0000 7fa0eed9d640 1 -- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0f010b6f0 msgr2=0x7fa0f019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.238+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0f010b6f0 0x7fa0f019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.238+0000 7fa0eed9d640 1 -- 172.21.15.31:0/2159930602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f01a2690 con 0x7fa0f010c040 2024-09-21T06:16:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.238+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.239+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 0x7fa0f01a1f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fd0 tx=0x7fa0e800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.239+0000 7fa0df7fe640 1 -- 172.21.15.31:0/2159930602 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800ce20 con 0x7fa0f010c040 2024-09-21T06:16:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.240+0000 7fa0df7fe640 1 -- 172.21.15.31:0/2159930602 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e8005020 con 0x7fa0f010c040 2024-09-21T06:16:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.240+0000 7fa0df7fe640 1 -- 172.21.15.31:0/2159930602 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8010690 con 0x7fa0f010c040 2024-09-21T06:16:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.240+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f01a2980 con 0x7fa0f010c040 2024-09-21T06:16:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.240+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f0077800 con 0x7fa0f010c040 2024-09-21T06:16:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.244+0000 7fa0df7fe640 1 -- 172.21.15.31:0/2159930602 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa0e80054d0 con 0x7fa0f010c040 2024-09-21T06:16:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.245+0000 7fa0fb383740 1 Processor -- start 2024-09-21T06:16:00.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.245+0000 7fa0fb383740 1 -- start start 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.245+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.245+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f5eb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.245+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f223670 0x55662f5ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0fb383740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55662f561e60 con 0x55662f223670 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0fb383740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55662f551290 con 0x55662f244350 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0fb383740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55662f5550a0 con 0x55662f21b490 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46488/0 (socket says 172.21.15.31:46488) 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0edd9b640 1 -- 172.21.15.31:0/2982361615 learned_addr learned my addr 172.21.15.31:0/2982361615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:00.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f223670 0x55662f5ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.246+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f5eb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.247+0000 7fa0edd9b640 1 -- 172.21.15.31:0/2982361615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 msgr2=0x55662f5eb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.247+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f5eb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.247+0000 7fa0edd9b640 1 -- 172.21.15.31:0/2982361615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f223670 msgr2=0x55662f5ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.247+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f223670 0x55662f5ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.247+0000 7fa0edd9b640 1 -- 172.21.15.31:0/2982361615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662f2af930 con 0x55662f244350 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.247+0000 7fa0df7fe640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0d80b6800 0x7fa0d80b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.248+0000 7fa0df7fe640 1 -- 172.21.15.31:0/2159930602 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa0e80e7dd0 con 0x7fa0f010c040 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.248+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0d80b6800 0x7fa0d80b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.248+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5eb430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011440 tx=0x7fa0e4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757dd7aedbc0a659 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.249+0000 7fa0dcff9640 1 -- 172.21.15.31:0/2982361615 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4004030 con 0x55662f244350 2024-09-21T06:16:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.249+0000 7fa0dcff9640 1 -- 172.21.15.31:0/2982361615 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e40041d0 con 0x55662f244350 2024-09-21T06:16:00.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.249+0000 7fa0dcff9640 1 -- 172.21.15.31:0/2982361615 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400f620 con 0x55662f244350 2024-09-21T06:16:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.250+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0d80b6800 0x7fa0d80b8d10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005f70 tx=0x7fa0e0005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.250+0000 7fa0fb383740 1 -- 172.21.15.31:0/2982361615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 msgr2=0x55662f5eb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.250+0000 7fa0fb383740 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5eb430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011440 tx=0x7fa0e4005850 comp rx=0 tx=0).stop 2024-09-21T06:16:00.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 -- 172.21.15.31:0/2982361615 shutdown_connections 2024-09-21T06:16:00.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f223670 0x55662f5ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f5eb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 --2- 172.21.15.31:0/2982361615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5eb430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 -- 172.21.15.31:0/2982361615 >> 172.21.15.31:0/2982361615 conn(0x55662f238740 msgr2=0x55662f5e9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:00.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 -- 172.21.15.31:0/2982361615 shutdown_connections 2024-09-21T06:16:00.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.251+0000 7fa0fb383740 1 -- 172.21.15.31:0/2982361615 wait complete. 2024-09-21T06:16:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.252+0000 7fa0fb383740 1 -- start start 2024-09-21T06:16:00.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.252+0000 7fa0fb383740 1 Processor -- start 2024-09-21T06:16:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.253+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.253+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f602f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0fb383740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55662f3b2890 con 0x55662f21b490 2024-09-21T06:16:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0fb383740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55662f4ebd40 con 0x55662f244350 2024-09-21T06:16:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0fb383740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55662f5f9c10 con 0x55662f223670 2024-09-21T06:16:00.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42870/0 (socket says 172.21.15.31:42870) 2024-09-21T06:16:00.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0ee59c640 1 -- 172.21.15.31:0/3157966420 learned_addr learned my addr 172.21.15.31:0/3157966420 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:00.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f602f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.254+0000 7fa0fb383740 10 client.0 _ll_register_callbacks cb 0x55662f54fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0edd9b640 1 -- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 msgr2=0x55662f602f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f602f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0edd9b640 1 -- 172.21.15.31:0/3157966420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 msgr2=0x55662f5f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0edd9b640 1 -- 172.21.15.31:0/3157966420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662f603520 con 0x55662f223670 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5f7570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40058c0 tx=0x7fa0e4003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e07b92921ff787 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/3157966420 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4005240 con 0x55662f223670 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e40041d0 con 0x55662f223670 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.255+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/3157966420 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa0e40053e0 con 0x55662f223670 2024-09-21T06:16:00.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.256+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/3157966420 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa0e40055e0 con 0x55662f223670 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.256+0000 7fa0fb383740 10 client.26206 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.256+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55662f605e10 con 0x55662f223670 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.256+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/3157966420 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e40041d0 con 0x55662f223670 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f602f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/3157966420 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa0e4030750 con 0x55662f223670 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0fb383740 10 client.26206 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0fb383740 10 client.26206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0fb383740 20 client.26206 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0fb383740 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f202cd0 0x55662f6094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0fb383740 4 client.26206 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662f563db0 con 0x55662f202cd0 2024-09-21T06:16:00.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.257+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f202cd0 0x55662f6094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.258+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f202cd0 0x55662f6094d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee657d885896e9a1 server_cookie=10c04d5cbf2d46b8 in_seq=0 out_seq=0 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.258+0000 7fa0d5ffb640 10 client.26206 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/3157966420 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55662f563db0 con 0x55662f202cd0 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0d5ffb640 10 client.26206 handle_command_reply: tid=0 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 client.26206 shutdown 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 msgr2=0x55662f5f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5f7570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40058c0 tx=0x7fa0e4003820 comp rx=0 tx=0).stop 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 shutdown_connections 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f202cd0 0x55662f6094d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f602f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5f7570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 --2- 172.21.15.31:0/3157966420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.259+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 >> 172.21.15.31:0/3157966420 conn(0x55662f238740 msgr2=0x55662f5ee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:00.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.260+0000 7fa0d5ffb640 0 client.26206 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:00.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.260+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 shutdown_connections 2024-09-21T06:16:00.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.260+0000 7fa0fb383740 1 -- 172.21.15.31:0/3157966420 wait complete. 2024-09-21T06:16:00.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.260+0000 7fa0fb383740 20 client.26206 trim_cache size 0 max 16384 2024-09-21T06:16:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.290+0000 7fa0fb383740 1 Processor -- start 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 -- start start 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f602e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 0x55662f5ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 0x55662f5eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01b3cf0 con 0x55662f244350 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01b6570 con 0x55662f223670 2024-09-21T06:16:00.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.291+0000 7fa0fb383740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01b4360 con 0x55662f21b490 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 0x55662f5ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0eed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 0x55662f5eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0eed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 0x55662f5eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42876/0 (socket says 172.21.15.31:42876) 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 0x55662f5ee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46500/0 (socket says 172.21.15.31:46500) 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0eed9d640 1 -- 172.21.15.31:0/521353925 learned_addr learned my addr 172.21.15.31:0/521353925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f602e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 -- 172.21.15.31:0/521353925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 msgr2=0x55662f602e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f602e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 -- 172.21.15.31:0/521353925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 msgr2=0x55662f5eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 0x55662f5eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 -- 172.21.15.31:0/521353925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662f5f80d0 con 0x55662f223670 2024-09-21T06:16:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 0x55662f5eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.292+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 0x55662f5ee530 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011440 tx=0x7fa0e4005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb572469bed69897 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.293+0000 7fa0d7fff640 1 -- 172.21.15.31:0/521353925 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40040e0 con 0x55662f223670 2024-09-21T06:16:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.293+0000 7fa0d7fff640 1 -- 172.21.15.31:0/521353925 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e4035020 con 0x55662f223670 2024-09-21T06:16:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.293+0000 7fa0d7fff640 1 -- 172.21.15.31:0/521353925 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4005300 con 0x55662f223670 2024-09-21T06:16:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 -- 172.21.15.31:0/521353925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 msgr2=0x55662f5ee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 0x55662f5ee530 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011440 tx=0x7fa0e4005b90 comp rx=0 tx=0).stop 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 -- 172.21.15.31:0/521353925 shutdown_connections 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f244350 0x55662f5eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f223670 0x55662f5ee530 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 --2- 172.21.15.31:0/521353925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f21b490 0x55662f602e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 -- 172.21.15.31:0/521353925 >> 172.21.15.31:0/521353925 conn(0x55662f238740 msgr2=0x55662f5f5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 -- 172.21.15.31:0/521353925 shutdown_connections 2024-09-21T06:16:00.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.294+0000 7fa0fb383740 1 -- 172.21.15.31:0/521353925 wait complete. 2024-09-21T06:16:00.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.295+0000 7fa0fb383740 1 -- start start 2024-09-21T06:16:00.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.295+0000 7fa0fb383740 1 Processor -- start 2024-09-21T06:16:00.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0fb383740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0fb383740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01b4a30 con 0x55662f21b490 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0fb383740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01b5ea0 con 0x55662f244350 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42886/0 (socket says 172.21.15.31:42886) 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0ee59c640 1 -- 172.21.15.31:0/4077632683 learned_addr learned my addr 172.21.15.31:0/4077632683 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.296+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f01b5100 con 0x55662f223670 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.297+0000 7fa0fb383740 10 client.0 _ll_register_callbacks cb 0x7fa0e4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:00.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.297+0000 7fa0edd9b640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0eed9d640 1 -- 172.21.15.31:0/4077632683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 msgr2=0x55662f5ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0eed9d640 1 -- 172.21.15.31:0/4077632683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 msgr2=0x55662f5ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0eed9d640 1 -- 172.21.15.31:0/4077632683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662f5e8a30 con 0x55662f244350 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:00.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0eed9d640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5e82a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800c760 tx=0x7fa0e80b7410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=202c50c67a254f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:00.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.298+0000 7fa0d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:00.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/4077632683 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80d51d0 con 0x55662f244350 2024-09-21T06:16:00.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e4035020 con 0x55662f244350 2024-09-21T06:16:00.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/4077632683 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa0e80a9750 con 0x55662f244350 2024-09-21T06:16:00.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/4077632683 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa0e80b26e0 con 0x55662f244350 2024-09-21T06:16:00.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0fb383740 10 client.26174 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:00.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55662f5fe980 con 0x55662f244350 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/4077632683 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e4035020 con 0x55662f244350 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.299+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/4077632683 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa0e80aca60 con 0x55662f244350 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0fb383740 10 client.26174 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0fb383740 10 client.26174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0fb383740 20 client.26174 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0fb383740 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f1d8080 0x55662f6760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0fb383740 4 client.26174 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55662f55f860 con 0x55662f1d8080 2024-09-21T06:16:00.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.300+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f1d8080 0x55662f6760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:00.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.301+0000 7fa0ee59c640 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f1d8080 0x55662f6760c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26fcc0129b82f19 server_cookie=917b29d09402288a in_seq=0 out_seq=0 2024-09-21T06:16:00.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.301+0000 7fa0d5ffb640 10 client.26174 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:00.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0d5ffb640 1 -- 172.21.15.31:0/4077632683 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55662f55f860 con 0x55662f1d8080 2024-09-21T06:16:00.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0d5ffb640 10 client.26174 handle_command_reply: tid=0 2024-09-21T06:16:00.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 client.26174 shutdown 2024-09-21T06:16:00.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 msgr2=0x55662f5e82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5e82a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800c760 tx=0x7fa0e80b7410 comp rx=0 tx=0).stop 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 shutdown_connections 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55662f1d8080 0x55662f6760c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55662f244350 0x55662f5e82a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55662f223670 0x55662f5ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 --2- 172.21.15.31:0/4077632683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55662f21b490 0x55662f5ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 >> 172.21.15.31:0/4077632683 conn(0x55662f238740 msgr2=0x55662f5ee500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0d5ffb640 0 client.26174 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.303+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 shutdown_connections 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.304+0000 7fa0fb383740 1 -- 172.21.15.31:0/4077632683 wait complete. 2024-09-21T06:16:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.304+0000 7fa0fb383740 20 client.26174 trim_cache size 0 max 16384 2024-09-21T06:16:00.307 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:16:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "e2bbfb80-25a6-433d-9181-39df795d15f2", 2024-09-21T06:16:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:16:00.307 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.305+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0d80b6800 msgr2=0x7fa0d80b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.305+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0d80b6800 0x7fa0d80b8d10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005f70 tx=0x7fa0e0005a20 comp rx=0 tx=0).stop 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.305+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 msgr2=0x7fa0f01a1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.305+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 0x7fa0f01a1f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fd0 tx=0x7fa0e800ef10 comp rx=0 tx=0).stop 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 shutdown_connections 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0f010c040 0x7fa0f01a1f60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0d80b6800 0x7fa0d80b8d10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0f010b6f0 0x7fa0f019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 --2- 172.21.15.31:0/2159930602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0f0105db0 0x7fa0f019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 >> 172.21.15.31:0/2159930602 conn(0x7fa0f01017a0 msgr2=0x7fa0f0104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 shutdown_connections 2024-09-21T06:16:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:00.306+0000 7fa0f4e0f640 1 -- 172.21.15.31:0/2159930602 wait complete. 2024-09-21T06:16:01.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:00 smithi031 ceph-mon[31237]: pgmap v452: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.5 MiB/s wr, 719 op/s 2024-09-21T06:16:01.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:00 smithi110 ceph-mon[34550]: pgmap v452: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.5 MiB/s wr, 719 op/s 2024-09-21T06:16:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:00 smithi107 ceph-mon[34380]: pgmap v452: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.5 MiB/s wr, 719 op/s 2024-09-21T06:16:01.429 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.436+0000 7fa536c4d640 1 -- 172.21.15.31:0/1382614306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5301029b0 msgr2=0x7fa530102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.436+0000 7fa536c4d640 1 --2- 172.21.15.31:0/1382614306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5301029b0 0x7fa530102dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa524009860 tx=0x7fa52402a340 comp rx=0 tx=0).stop 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 -- 172.21.15.31:0/1382614306 shutdown_connections 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 --2- 172.21.15.31:0/1382614306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa530108c30 0x7fa53010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 --2- 172.21.15.31:0/1382614306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301082e0 0x7fa5301086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 --2- 172.21.15.31:0/1382614306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5301029b0 0x7fa530102dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 -- 172.21.15.31:0/1382614306 >> 172.21.15.31:0/1382614306 conn(0x7fa53007a740 msgr2=0x7fa53007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 -- 172.21.15.31:0/1382614306 shutdown_connections 2024-09-21T06:16:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.437+0000 7fa536c4d640 1 -- 172.21.15.31:0/1382614306 wait complete. 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 Processor -- start 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 -- start start 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 0x7fa530199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5301082e0 0x7fa530199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa530108c30 0x7fa53019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301128f0 con 0x7fa5301082e0 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa530112770 con 0x7fa530108c30 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa536c4d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa530112a70 con 0x7fa5301029b0 2024-09-21T06:16:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa5349e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 0x7fa530199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5301082e0 0x7fa530199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa5349e9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 0x7fa530199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58206/0 (socket says 172.21.15.31:58206) 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa5349e9640 1 -- 172.21.15.31:0/812963554 learned_addr learned my addr 172.21.15.31:0/812963554 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.438+0000 7fa5351ea640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa530108c30 0x7fa53019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5349e9640 1 -- 172.21.15.31:0/812963554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa530108c30 msgr2=0x7fa53019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5349e9640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa530108c30 0x7fa53019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5349e9640 1 -- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5301082e0 msgr2=0x7fa530199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5349e9640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5301082e0 0x7fa530199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5349e9640 1 -- 172.21.15.31:0/812963554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53019f2a0 con 0x7fa5301029b0 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5351ea640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa530108c30 0x7fa53019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa5349e9640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 0x7fa530199370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa524035040 tx=0x7fa524032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa52dffb640 1 -- 172.21.15.31:0/812963554 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524037040 con 0x7fa5301029b0 2024-09-21T06:16:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53019f530 con 0x7fa5301029b0 2024-09-21T06:16:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa52dffb640 1 -- 172.21.15.31:0/812963554 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa524041070 con 0x7fa5301029b0 2024-09-21T06:16:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.439+0000 7fa52dffb640 1 -- 172.21.15.31:0/812963554 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524003850 con 0x7fa5301029b0 2024-09-21T06:16:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.440+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301a5d20 con 0x7fa5301029b0 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.442+0000 7fa53dbc4740 1 Processor -- start 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.442+0000 7fa53dbc4740 1 -- start start 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.442+0000 7fa52dffb640 1 -- 172.21.15.31:0/812963554 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa5240039f0 con 0x7fa5301029b0 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f21561b0 0x5618f24fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f212d2f0 0x5618f24fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 0x5618f24fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618f24706f0 con 0x5618f212d2f0 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618f245fb20 con 0x5618f21561b0 2024-09-21T06:16:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618f2463930 con 0x5618f21354d0 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 0x5618f24fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 0x5618f24fc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58212/0 (socket says 172.21.15.31:58212) 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 -- 172.21.15.31:0/743357244 learned_addr learned my addr 172.21.15.31:0/743357244 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa5351ea640 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f21561b0 0x5618f24fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa5349e9640 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f212d2f0 0x5618f24fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 -- 172.21.15.31:0/743357244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f21561b0 msgr2=0x5618f24fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f21561b0 0x5618f24fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 -- 172.21.15.31:0/743357244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f212d2f0 msgr2=0x5618f24fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f212d2f0 0x5618f24fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa52ffff640 1 -- 172.21.15.31:0/743357244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f2425530 con 0x5618f21354d0 2024-09-21T06:16:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.443+0000 7fa5349e9640 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f212d2f0 0x5618f24fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.444+0000 7fa52ffff640 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 0x5618f24fc430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009c20 tx=0x7fa51c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9da97365490ab91b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.444+0000 7fa51affd640 1 -- 172.21.15.31:0/743357244 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c004030 con 0x5618f21354d0 2024-09-21T06:16:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.444+0000 7fa51affd640 1 -- 172.21.15.31:0/743357244 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c005020 con 0x5618f21354d0 2024-09-21T06:16:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.444+0000 7fa51affd640 1 -- 172.21.15.31:0/743357244 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c004ca0 con 0x5618f21354d0 2024-09-21T06:16:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.444+0000 7fa52dffb640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5200b6680 0x7fa5200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.444+0000 7fa5351ea640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5200b6680 0x7fa5200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa52dffb640 1 -- 172.21.15.31:0/812963554 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa524107300 con 0x7fa5301029b0 2024-09-21T06:16:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 -- 172.21.15.31:0/743357244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 msgr2=0x5618f24fc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 0x5618f24fc430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009c20 tx=0x7fa51c011810 comp rx=0 tx=0).stop 2024-09-21T06:16:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa5351ea640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5200b6680 0x7fa5200b8b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa528005fd0 tx=0x7fa5280095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 -- 172.21.15.31:0/743357244 shutdown_connections 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21354d0 0x5618f24fc430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f212d2f0 0x5618f24fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/743357244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f21561b0 0x5618f24fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 -- 172.21.15.31:0/743357244 >> 172.21.15.31:0/743357244 conn(0x5618f214a300 msgr2=0x5618f24f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 -- 172.21.15.31:0/743357244 shutdown_connections 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 -- 172.21.15.31:0/743357244 wait complete. 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.445+0000 7fa53dbc4740 1 -- start start 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 Processor -- start 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2503e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21561b0 0x5618f2508ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618f2397ab0 con 0x5618f21354d0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618f2405730 con 0x5618f212d2f0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618f250a170 con 0x5618f21561b0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa5349e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2503e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25043e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42924/0 (socket says 172.21.15.31:42924) 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa52ffff640 1 -- 172.21.15.31:0/1174616409 learned_addr learned my addr 172.21.15.31:0/1174616409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa53dbc4740 10 client.0 _ll_register_callbacks cb 0x5618f24596b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.446+0000 7fa5351ea640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21561b0 0x5618f2508ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa5349e9640 1 -- 172.21.15.31:0/1174616409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21561b0 msgr2=0x5618f2508ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa5349e9640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21561b0 0x5618f2508ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa5349e9640 1 -- 172.21.15.31:0/1174616409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 msgr2=0x5618f25043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa5349e9640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa5349e9640 1 -- 172.21.15.31:0/1174616409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f21c1790 con 0x5618f212d2f0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa52ffff640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa5349e9640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2503e80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa524104a90 tx=0x7fa5240303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03251a82e57547c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa50ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa50ffff640 1 -- 172.21.15.31:0/1174616409 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240c97e0 con 0x5618f212d2f0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa50ffff640 1 -- 172.21.15.31:0/1174616409 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa5240cc5f0 con 0x5618f212d2f0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51c005020 con 0x5618f212d2f0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa50ffff640 1 -- 172.21.15.31:0/1174616409 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa5240cc900 con 0x5618f212d2f0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa53dbc4740 10 client.26186 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5618f223a620 con 0x5618f212d2f0 2024-09-21T06:16:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.447+0000 7fa50ffff640 1 -- 172.21.15.31:0/1174616409 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c005020 con 0x5618f212d2f0 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa50ffff640 1 -- 172.21.15.31:0/1174616409 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa5240c8ae0 con 0x5618f212d2f0 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa53dbc4740 10 client.26186 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa53dbc4740 10 client.26186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa53dbc4740 20 client.26186 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f2114d80 0x5618f2521a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa53dbc4740 4 client.26186 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618f2472640 con 0x5618f2114d80 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.448+0000 7fa52ffff640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f2114d80 0x5618f2521a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.449+0000 7fa52ffff640 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f2114d80 0x5618f2521a20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba516b2d8567657 server_cookie=e5fa06966ba8a90a in_seq=0 out_seq=0 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.449+0000 7fa50ffff640 10 client.26186 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.449+0000 7fa50ffff640 1 -- 172.21.15.31:0/1174616409 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618f2472640 con 0x5618f2114d80 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.449+0000 7fa50ffff640 10 client.26186 handle_command_reply: tid=0 2024-09-21T06:16:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.449+0000 7fa53dbc4740 1 client.26186 shutdown 2024-09-21T06:16:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 msgr2=0x5618f2503e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2503e80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa524104a90 tx=0x7fa5240303b0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 shutdown_connections 2024-09-21T06:16:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f2114d80 0x5618f2521a20 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa50ffff640 0 client.26186 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f21561b0 0x5618f2508ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/1174616409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2503e80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 >> 172.21.15.31:0/1174616409 conn(0x5618f214a300 msgr2=0x5618f24fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 shutdown_connections 2024-09-21T06:16:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 1 -- 172.21.15.31:0/1174616409 wait complete. 2024-09-21T06:16:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.450+0000 7fa53dbc4740 20 client.26186 trim_cache size 0 max 16384 2024-09-21T06:16:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 Processor -- start 2024-09-21T06:16:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 -- start start 2024-09-21T06:16:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f2114d80 0x5618f24f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2508d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301ab370 con 0x5618f21354d0 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301adbf0 con 0x5618f212d2f0 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301ab9e0 con 0x5618f2114d80 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa5351ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2508d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa5351ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42936/0 (socket says 172.21.15.31:42936) 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa5351ea640 1 -- 172.21.15.31:0/3248202165 learned_addr learned my addr 172.21.15.31:0/3248202165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa5349e9640 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f2114d80 0x5618f24f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa52ffff640 1 -- 172.21.15.31:0/3248202165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f2114d80 msgr2=0x5618f24f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa52ffff640 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f2114d80 0x5618f24f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa52ffff640 1 -- 172.21.15.31:0/3248202165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 msgr2=0x5618f25092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa52ffff640 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa52ffff640 1 -- 172.21.15.31:0/3248202165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f24fe810 con 0x5618f212d2f0 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.473+0000 7fa5351ea640 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.474+0000 7fa52ffff640 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2508d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5618f24f3ba0 tx=0x7fa51c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1217197d0e8475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.474+0000 7fa51a7fc640 1 -- 172.21.15.31:0/3248202165 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c01e040 con 0x5618f212d2f0 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.474+0000 7fa51a7fc640 1 -- 172.21.15.31:0/3248202165 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c0044f0 con 0x5618f212d2f0 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.474+0000 7fa51a7fc640 1 -- 172.21.15.31:0/3248202165 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c004f40 con 0x5618f212d2f0 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 -- 172.21.15.31:0/3248202165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 msgr2=0x5618f2508d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2508d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5618f24f3ba0 tx=0x7fa51c005d30 comp rx=0 tx=0).stop 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 -- 172.21.15.31:0/3248202165 shutdown_connections 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f25092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f212d2f0 0x5618f2508d60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/3248202165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f2114d80 0x5618f24f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 -- 172.21.15.31:0/3248202165 >> 172.21.15.31:0/3248202165 conn(0x5618f214a300 msgr2=0x5618f250c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 -- 172.21.15.31:0/3248202165 shutdown_connections 2024-09-21T06:16:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 -- 172.21.15.31:0/3248202165 wait complete. 2024-09-21T06:16:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 -- start start 2024-09-21T06:16:01.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.475+0000 7fa53dbc4740 1 Processor -- start 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 0x5618f251c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f212d2f0 0x5618f2519780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f2519cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301ac0b0 con 0x5618f21354d0 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301ad520 con 0x5618f2114d80 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5301ac780 con 0x5618f212d2f0 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa53dbc4740 10 client.0 _ll_register_callbacks cb 0x7fa524005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa5349e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 0x5618f251c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa5351ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f2519cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa5349e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 0x5618f251c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46552/0 (socket says 172.21.15.31:46552) 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa5351ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f2519cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42944/0 (socket says 172.21.15.31:42944) 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa5349e9640 1 -- 172.21.15.31:0/370684712 learned_addr learned my addr 172.21.15.31:0/370684712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.476+0000 7fa52ffff640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f212d2f0 0x5618f2519780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5349e9640 1 -- 172.21.15.31:0/370684712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f212d2f0 msgr2=0x5618f2519780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5349e9640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f212d2f0 0x5618f2519780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5349e9640 1 -- 172.21.15.31:0/370684712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 msgr2=0x5618f2519cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5349e9640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f2519cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5349e9640 1 -- 172.21.15.31:0/370684712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f251a330 con 0x5618f2114d80 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5351ea640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f2519cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa5349e9640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 0x5618f251c2c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5240c8e80 tx=0x7fa5240cc620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01e837566199040 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa50ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa50ffff640 1 -- 172.21.15.31:0/370684712 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240ccae0 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa50ffff640 1 -- 172.21.15.31:0/370684712 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa5240b81b0 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51c0044f0 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa50ffff640 1 -- 172.21.15.31:0/370684712 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa5240ccf60 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa53dbc4740 10 client.26192 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5618f251a330 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.477+0000 7fa50ffff640 1 -- 172.21.15.31:0/370684712 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c0044f0 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa50ffff640 1 -- 172.21.15.31:0/370684712 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa5240d2e00 con 0x5618f2114d80 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa53dbc4740 10 client.26192 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa53dbc4740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa53dbc4740 20 client.26192 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f21561b0 0x5618f258e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa53dbc4740 4 client.26192 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618f246e0f0 con 0x5618f21561b0 2024-09-21T06:16:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa52ffff640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f21561b0 0x5618f258e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa50ffff640 10 client.26192 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.478+0000 7fa52ffff640 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f21561b0 0x5618f258e6a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e44f822ab0d362 server_cookie=14a9f134ef8e2f2b in_seq=0 out_seq=0 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa50ffff640 1 -- 172.21.15.31:0/370684712 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618f246e0f0 con 0x5618f21561b0 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa50ffff640 10 client.26192 handle_command_reply: tid=0 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 client.26192 shutdown 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 msgr2=0x5618f251c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 0x5618f251c2c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5240c8e80 tx=0x7fa5240cc620 comp rx=0 tx=0).stop 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 shutdown_connections 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618f21561b0 0x5618f258e6a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618f21354d0 0x5618f2519cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa50ffff640 0 client.26192 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618f212d2f0 0x5618f2519780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 --2- 172.21.15.31:0/370684712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618f2114d80 0x5618f251c2c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 >> 172.21.15.31:0/370684712 conn(0x5618f214a300 msgr2=0x5618f250c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 shutdown_connections 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 1 -- 172.21.15.31:0/370684712 wait complete. 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.502+0000 7fa53dbc4740 20 client.26192 trim_cache size 0 max 16384 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:16:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:16:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:16:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5200b6680 msgr2=0x7fa5200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5200b6680 0x7fa5200b8b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa528005fd0 tx=0x7fa5280095c0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 msgr2=0x7fa530199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 0x7fa530199370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa524035040 tx=0x7fa524032d50 comp rx=0 tx=0).stop 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 shutdown_connections 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa530108c30 0x7fa53019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa5200b6680 0x7fa5200b8b90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5301082e0 0x7fa530199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 --2- 172.21.15.31:0/812963554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5301029b0 0x7fa530199370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 >> 172.21.15.31:0/812963554 conn(0x7fa53007a740 msgr2=0x7fa530106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 shutdown_connections 2024-09-21T06:16:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:01.503+0000 7fa536c4d640 1 -- 172.21.15.31:0/812963554 wait complete. 2024-09-21T06:16:02.073 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:01 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:16:02.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:01 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:02.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:01 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:16:02.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:01 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 -- 172.21.15.31:0/549428867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 msgr2=0x7fb214102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 --2- 172.21.15.31:0/549428867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb214102eb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb210009900 tx=0x7fb21002a410 comp rx=0 tx=0).stop 2024-09-21T06:16:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 -- 172.21.15.31:0/549428867 shutdown_connections 2024-09-21T06:16:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 --2- 172.21.15.31:0/549428867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb214108d30 0x7fb21410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 --2- 172.21.15.31:0/549428867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2141083e0 0x7fb2141087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 --2- 172.21.15.31:0/549428867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb214102eb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 -- 172.21.15.31:0/549428867 >> 172.21.15.31:0/549428867 conn(0x7fb214079750 msgr2=0x7fb214079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 -- 172.21.15.31:0/549428867 shutdown_connections 2024-09-21T06:16:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.094+0000 7fb21b8be640 1 -- 172.21.15.31:0/549428867 wait complete. 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.095+0000 7fb21b8be640 1 Processor -- start 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.095+0000 7fb21b8be640 1 -- start start 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21b8be640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb2141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21b8be640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2141083e0 0x7fb214198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21b8be640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb214108d30 0x7fb21419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21b8be640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141118f0 con 0x7fb214102aa0 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21b8be640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb214111770 con 0x7fb214108d30 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21b8be640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb214111a70 con 0x7fb2141083e0 2024-09-21T06:16:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb2141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb2141983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42956/0 (socket says 172.21.15.31:42956) 2024-09-21T06:16:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb218e59640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2141083e0 0x7fb214198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb21965a640 1 -- 172.21.15.31:0/3209011768 learned_addr learned my addr 172.21.15.31:0/3209011768 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb219e5b640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb214108d30 0x7fb21419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb218e59640 1 -- 172.21.15.31:0/3209011768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb214108d30 msgr2=0x7fb21419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb218e59640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb214108d30 0x7fb21419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb218e59640 1 -- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 msgr2=0x7fb2141983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb218e59640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb2141983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.096+0000 7fb218e59640 1 -- 172.21.15.31:0/3209011768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21419e310 con 0x7fb2141083e0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.097+0000 7fb218e59640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2141083e0 0x7fb214198940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb21406ab20 tx=0x7fb204010f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.097+0000 7fb20a7fc640 1 -- 172.21.15.31:0/3209011768 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20401a020 con 0x7fb2141083e0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.097+0000 7fb20a7fc640 1 -- 172.21.15.31:0/3209011768 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb204016480 con 0x7fb2141083e0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.097+0000 7fb20a7fc640 1 -- 172.21.15.31:0/3209011768 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204015640 con 0x7fb2141083e0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.097+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21419e600 con 0x7fb2141083e0 2024-09-21T06:16:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.097+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2141a4d60 con 0x7fb2141083e0 2024-09-21T06:16:02.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.100+0000 7fb221e32740 1 Processor -- start 2024-09-21T06:16:02.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.100+0000 7fb221e32740 1 -- start start 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb221e32740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b172650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb221e32740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b172b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb221e32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b1730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb221e32740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560f5b0e8e50 con 0x560f5ada21b0 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb221e32740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f5b0d8280 con 0x560f5adcb070 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb221e32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560f5b0dc090 con 0x560f5adaa390 2024-09-21T06:16:02.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb20a7fc640 1 -- 172.21.15.31:0/3209011768 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb20401d020 con 0x7fb2141083e0 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb21965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b172b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb219e5b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b172650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb219e5b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b172650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46564/0 (socket says 172.21.15.31:46564) 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb21965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b172b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42964/0 (socket says 172.21.15.31:42964) 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb219e5b640 1 -- 172.21.15.31:0/1158111282 learned_addr learned my addr 172.21.15.31:0/1158111282 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.101+0000 7fb218e59640 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b1730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb219e5b640 1 -- 172.21.15.31:0/1158111282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 msgr2=0x560f5b1730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b1730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb219e5b640 1 -- 172.21.15.31:0/1158111282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 msgr2=0x560f5b172b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b172b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb219e5b640 1 -- 172.21.15.31:0/1158111282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5ae36650 con 0x560f5adcb070 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b172650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb20c017fd0 tx=0x7fb20c00e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d54768b531c4bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb1ff7fe640 1 -- 172.21.15.31:0/1158111282 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c014070 con 0x560f5adcb070 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb1ff7fe640 1 -- 172.21.15.31:0/1158111282 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c00ee20 con 0x560f5adcb070 2024-09-21T06:16:02.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.102+0000 7fb20a7fc640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb2000b6700 0x7fb2000b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.103+0000 7fb21965a640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb2000b6700 0x7fb2000b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.103+0000 7fb20a7fc640 1 -- 172.21.15.31:0/3209011768 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb2040e7840 con 0x7fb2141083e0 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.103+0000 7fb1ff7fe640 1 -- 172.21.15.31:0/1158111282 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c004290 con 0x560f5adcb070 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 -- 172.21.15.31:0/1158111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 msgr2=0x560f5b172650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb21965a640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb2000b6700 0x7fb2000b8c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb210031440 tx=0x7fb2100058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b172650 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb20c017fd0 tx=0x7fb20c00e3b0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 -- 172.21.15.31:0/1158111282 shutdown_connections 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b1730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b172b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 --2- 172.21.15.31:0/1158111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b172650 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 -- 172.21.15.31:0/1158111282 >> 172.21.15.31:0/1158111282 conn(0x560f5adbf460 msgr2=0x560f5b170540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:02.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 -- 172.21.15.31:0/1158111282 shutdown_connections 2024-09-21T06:16:02.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.104+0000 7fb221e32740 1 -- 172.21.15.31:0/1158111282 wait complete. 2024-09-21T06:16:02.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.105+0000 7fb221e32740 1 -- start start 2024-09-21T06:16:02.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.105+0000 7fb221e32740 1 Processor -- start 2024-09-21T06:16:02.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb221e32740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b179e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb221e32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b17a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb221e32740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b17f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb21965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b179e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb221e32740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560f5af39700 con 0x560f5ada21b0 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb221e32740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f5b073110 con 0x560f5adcb070 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb21965a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b179e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42970/0 (socket says 172.21.15.31:42970) 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb21965a640 1 -- 172.21.15.31:0/1803201340 learned_addr learned my addr 172.21.15.31:0/1803201340 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b17f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.106+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560f5b180d10 con 0x560f5adaa390 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb21965a640 1 -- 172.21.15.31:0/1803201340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 msgr2=0x560f5b17a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb21965a640 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b17a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb21965a640 1 -- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 msgr2=0x560f5b17f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb21965a640 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b17f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb21965a640 1 -- 172.21.15.31:0/1803201340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5b0a0c40 con 0x560f5ada21b0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb221e32740 10 client.0 _ll_register_callbacks cb 0x560f5b08c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb20c00ee20 con 0x560f5ada21b0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb21965a640 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b179e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb210009a20 tx=0x7fb210009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df310a121c57d647 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb1fcff9640 10 client.16482 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb1fcff9640 1 -- 172.21.15.31:0/1803201340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210035040 con 0x560f5ada21b0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.107+0000 7fb1fcff9640 1 -- 172.21.15.31:0/1803201340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb210004300 con 0x560f5ada21b0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb1fcff9640 1 -- 172.21.15.31:0/1803201340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb2100035f0 con 0x560f5ada21b0 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb221e32740 10 client.16482 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:02.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({fsmap=19}) -- 0x560f5b18cb10 con 0x560f5ada21b0 2024-09-21T06:16:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb1fcff9640 1 -- 172.21.15.31:0/1803201340 <== mon.0 v2:172.21.15.31:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb210054070 con 0x560f5ada21b0 2024-09-21T06:16:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb221e32740 10 client.16482 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:02.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb221e32740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.108+0000 7fb221e32740 20 client.16482 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.109+0000 7fb221e32740 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad899f0 0x560f5b18ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.109+0000 7fb221e32740 4 client.16482 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.109+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f5b0eada0 con 0x560f5ad899f0 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.109+0000 7fb218e59640 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad899f0 0x560f5b18ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.110+0000 7fb1fcff9640 10 client.16482 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:02.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.110+0000 7fb218e59640 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad899f0 0x560f5b18ffa0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b6b6456416fe10 server_cookie=fef9041821179042 in_seq=0 out_seq=0 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb1fcff9640 1 -- 172.21.15.31:0/1803201340 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f5b0eada0 con 0x560f5ad899f0 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb1fcff9640 10 client.16482 handle_command_reply: tid=0 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 client.16482 shutdown 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 msgr2=0x560f5b179e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b179e00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb210009a20 tx=0x7fb210009260 comp rx=0 tx=0).stop 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 shutdown_connections 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad899f0 0x560f5b18ffa0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb1fcff9640 0 client.16482 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adcb070 0x560f5b17f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5adaa390 0x560f5b17a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 --2- 172.21.15.31:0/1803201340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5ada21b0 0x560f5b179e00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.111+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 >> 172.21.15.31:0/1803201340 conn(0x560f5adbf460 msgr2=0x560f5b175830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:02.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.112+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 shutdown_connections 2024-09-21T06:16:02.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.112+0000 7fb221e32740 1 -- 172.21.15.31:0/1803201340 wait complete. 2024-09-21T06:16:02.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.112+0000 7fb221e32740 20 client.16482 trim_cache size 0 max 16384 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 Processor -- start 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 -- start start 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b17bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b175e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b17f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141ab4d0 con 0x560f5adcb070 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141add00 con 0x560f5adaa390 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb221e32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141abb40 con 0x560f5ada21b0 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb219e5b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b17f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb218e59640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b175e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb219e5b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b17f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42980/0 (socket says 172.21.15.31:42980) 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.133+0000 7fb219e5b640 1 -- 172.21.15.31:0/1439558089 learned_addr learned my addr 172.21.15.31:0/1439558089 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb21965a640 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b17bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb219e5b640 1 -- 172.21.15.31:0/1439558089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 msgr2=0x560f5b17bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b17bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb219e5b640 1 -- 172.21.15.31:0/1439558089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 msgr2=0x560f5b175e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b175e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb219e5b640 1 -- 172.21.15.31:0/1439558089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5b170e30 con 0x560f5adcb070 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb218e59640 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b175e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:02.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb219e5b640 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b17f3e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00f010 tx=0x7fb20c0179c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8edebd154757c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:02.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb1feffd640 1 -- 172.21.15.31:0/1439558089 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c026020 con 0x560f5adcb070 2024-09-21T06:16:02.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb1feffd640 1 -- 172.21.15.31:0/1439558089 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20c003970 con 0x560f5adcb070 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.134+0000 7fb1feffd640 1 -- 172.21.15.31:0/1439558089 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c014040 con 0x560f5adcb070 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 -- 172.21.15.31:0/1439558089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 msgr2=0x560f5b17f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b17f3e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00f010 tx=0x7fb20c0179c0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 -- 172.21.15.31:0/1439558089 shutdown_connections 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b17f3e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b175e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 --2- 172.21.15.31:0/1439558089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b17bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 -- 172.21.15.31:0/1439558089 >> 172.21.15.31:0/1439558089 conn(0x560f5adbf460 msgr2=0x560f5b175750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 -- 172.21.15.31:0/1439558089 shutdown_connections 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.135+0000 7fb221e32740 1 -- 172.21.15.31:0/1439558089 wait complete. 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.136+0000 7fb221e32740 1 -- start start 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.136+0000 7fb221e32740 1 Processor -- start 2024-09-21T06:16:02.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.136+0000 7fb221e32740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b187010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb221e32740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b187550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb221e32740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b187a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb221e32740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141ac1c0 con 0x560f5adcb070 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb221e32740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141ad630 con 0x560f5adaa390 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb221e32740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141ac890 con 0x560f5ada21b0 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb221e32740 10 client.0 _ll_register_callbacks cb 0x7fb210009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb219e5b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b187a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb21965a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b187010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb219e5b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b187a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42988/0 (socket says 172.21.15.31:42988) 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb219e5b640 1 -- 172.21.15.31:0/742283693 learned_addr learned my addr 172.21.15.31:0/742283693 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b187550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb21965a640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b187010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58302/0 (socket says 172.21.15.31:58302) 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 -- 172.21.15.31:0/742283693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 msgr2=0x560f5b187010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b187010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 -- 172.21.15.31:0/742283693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 msgr2=0x560f5b187a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b187a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 -- 172.21.15.31:0/742283693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f5b1f9b30 con 0x560f5adaa390 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb219e5b640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b187a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb218e59640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b187550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2040e4fd0 tx=0x7fb2040af7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbaa8365c0a4c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.137+0000 7fb1fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb1fcff9640 1 -- 172.21.15.31:0/742283693 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2040affd0 con 0x560f5adaa390 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb1fcff9640 1 -- 172.21.15.31:0/742283693 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fb2040abb40 con 0x560f5adaa390 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb20c003970 con 0x560f5adaa390 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb1fcff9640 1 -- 172.21.15.31:0/742283693 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb2040b43b0 con 0x560f5adaa390 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 10 client.26210 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:02.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560f5b1fadc0 con 0x560f5adaa390 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb1fcff9640 1 -- 172.21.15.31:0/742283693 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2040b4020 con 0x560f5adaa390 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb1fcff9640 1 -- 172.21.15.31:0/742283693 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb2040ab340 con 0x560f5adaa390 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 10 client.26210 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 20 client.26210 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad5eda0 0x560f5b1fd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 4 client.26210 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.138+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb20c003970 con 0x560f5ad5eda0 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb21965a640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad5eda0 0x560f5b1fd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb1fcff9640 10 client.26210 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:02.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb21965a640 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad5eda0 0x560f5b1fd550 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96391e4e2707122e server_cookie=a1f348d3ce99faf2 in_seq=0 out_seq=0 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb1fcff9640 1 -- 172.21.15.31:0/742283693 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x7fb20c003970 con 0x560f5ad5eda0 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb1fcff9640 10 client.26210 handle_command_reply: tid=0 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb221e32740 1 client.26210 shutdown 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 msgr2=0x560f5b187550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.139+0000 7fb221e32740 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b187550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2040e4fd0 tx=0x7fb2040af7d0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 shutdown_connections 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f5ad5eda0 0x560f5b1fd550 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f5adcb070 0x560f5b187a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f5adaa390 0x560f5b187550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb1fcff9640 0 client.26210 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 --2- 172.21.15.31:0/742283693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f5ada21b0 0x560f5b187010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 >> 172.21.15.31:0/742283693 conn(0x560f5adbf460 msgr2=0x560f5b176110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 shutdown_connections 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 1 -- 172.21.15.31:0/742283693 wait complete. 2024-09-21T06:16:02.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.140+0000 7fb221e32740 20 client.26210 trim_cache size 0 max 16384 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb2000b6700 msgr2=0x7fb2000b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb2000b6700 0x7fb2000b8c10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb210031440 tx=0x7fb2100058d0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2141083e0 msgr2=0x7fb214198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:02.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2141083e0 0x7fb214198940 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb21406ab20 tx=0x7fb204010f30 comp rx=0 tx=0).stop 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 shutdown_connections 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb214108d30 0x7fb21419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2141083e0 0x7fb214198940 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb2000b6700 0x7fb2000b8c10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 --2- 172.21.15.31:0/3209011768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb214102aa0 0x7fb2141983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.141+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 >> 172.21.15.31:0/3209011768 conn(0x7fb214079750 msgr2=0x7fb214106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.142+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 shutdown_connections 2024-09-21T06:16:02.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:02.142+0000 7fb21b8be640 1 -- 172.21.15.31:0/3209011768 wait complete. 2024-09-21T06:16:02.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:01 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:16:02.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:01 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:02.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:01 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:16:02.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:01 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:01 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:16:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:01 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:01 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:16:02.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:01 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:02.986 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:16:02.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:16:03.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:02 smithi031 ceph-mon[31237]: pgmap v453: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 9.0 MiB/s wr, 979 op/s 2024-09-21T06:16:03.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:02 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:16:03.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:02 smithi110 ceph-mon[34550]: pgmap v453: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 9.0 MiB/s wr, 979 op/s 2024-09-21T06:16:03.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:02 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:16:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:02 smithi107 ceph-mon[34380]: pgmap v453: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 9.0 MiB/s wr, 979 op/s 2024-09-21T06:16:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:02 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:16:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:04.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.091+0000 7fabdf6d6640 1 -- 172.21.15.31:0/3622495336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 msgr2=0x7fabd810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/3622495336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd810d8b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009900 tx=0x7fabc802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 -- 172.21.15.31:0/3622495336 shutdown_connections 2024-09-21T06:16:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/3622495336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd810ddf0 0x7fabd8110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/3622495336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd810d8b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/3622495336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd8104840 0x7fabd8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 -- 172.21.15.31:0/3622495336 >> 172.21.15.31:0/3622495336 conn(0x7fabd8079730 msgr2=0x7fabd8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 -- 172.21.15.31:0/3622495336 shutdown_connections 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.092+0000 7fabdf6d6640 1 -- 172.21.15.31:0/3622495336 wait complete. 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 Processor -- start 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 -- start start 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd8104840 0x7fabd8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd810ddf0 0x7fabd819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabd8113a70 con 0x7fabd8105270 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabd81138f0 con 0x7fabd810ddf0 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.093+0000 7fabdf6d6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabd8113bf0 con 0x7fabd8104840 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdcc71640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdcc71640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43012/0 (socket says 172.21.15.31:43012) 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdcc71640 1 -- 172.21.15.31:0/2617885416 learned_addr learned my addr 172.21.15.31:0/2617885416 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd8104840 0x7fabd8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabddc73640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd810ddf0 0x7fabd819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 -- 172.21.15.31:0/2617885416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd810ddf0 msgr2=0x7fabd819ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd810ddf0 0x7fabd819ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 -- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 msgr2=0x7fabd8199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd8199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 -- 172.21.15.31:0/2617885416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd81a04c0 con 0x7fabd8104840 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabdd472640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd8104840 0x7fabd8198f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fabd400e9c0 tx=0x7fabd400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.094+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd400ccc0 con 0x7fabd8104840 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.095+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd40035f0 con 0x7fabd8104840 2024-09-21T06:16:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.095+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd81a07b0 con 0x7fabd8104840 2024-09-21T06:16:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.095+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd400ce60 con 0x7fabd8104840 2024-09-21T06:16:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.095+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd81a0cc0 con 0x7fabd8104840 2024-09-21T06:16:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.097+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc0052c0 con 0x7fabd8104840 2024-09-21T06:16:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.097+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fabd4003fe0 con 0x7fabd8104840 2024-09-21T06:16:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.098+0000 7fabce7fc640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabc40b6680 0x7fabc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:04.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.099+0000 7fabdcc71640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabc40b6680 0x7fabc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:04.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.099+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fabd40e7910 con 0x7fabd8104840 2024-09-21T06:16:04.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.099+0000 7fabdcc71640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabc40b6680 0x7fabc40b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000c00 tx=0x7fabc8009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.104+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fabd40adea0 con 0x7fabd8104840 2024-09-21T06:16:04.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.363+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabbc005880 con 0x7fabd8104840 2024-09-21T06:16:04.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.364+0000 7fabce7fc640 1 -- 172.21.15.31:0/2617885416 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fabd40b2280 con 0x7fabd8104840 2024-09-21T06:16:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:16:04.366 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:16:04.366 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:16:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabc40b6680 msgr2=0x7fabc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabc40b6680 0x7fabc40b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabc8000c00 tx=0x7fabc8009560 comp rx=0 tx=0).stop 2024-09-21T06:16:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd8104840 msgr2=0x7fabd8198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd8104840 0x7fabd8198f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fabd400e9c0 tx=0x7fabd400ee80 comp rx=0 tx=0).stop 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 shutdown_connections 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabd810ddf0 0x7fabd819ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabc40b6680 0x7fabc40b8b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd8105270 0x7fabd8199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 --2- 172.21.15.31:0/2617885416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabd8104840 0x7fabd8198f20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 >> 172.21.15.31:0/2617885416 conn(0x7fabd8079730 msgr2=0x7fabd8103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 shutdown_connections 2024-09-21T06:16:04.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:04.366+0000 7fabdf6d6640 1 -- 172.21.15.31:0/2617885416 wait complete. 2024-09-21T06:16:05.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:04 smithi031 ceph-mon[31237]: pgmap v454: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 734 op/s 2024-09-21T06:16:05.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:04 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2617885416' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:05.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:16:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 265.0 s, 450.0 tps, lat 209.010 ms stddev 244.622 2024-09-21T06:16:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:04 smithi110 ceph-mon[34550]: pgmap v454: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 734 op/s 2024-09-21T06:16:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:04 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2617885416' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:04 smithi107 ceph-mon[34380]: pgmap v454: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 734 op/s 2024-09-21T06:16:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:04 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2617885416' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:05.471 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 -- 172.21.15.31:0/1814106802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd04108cf0 msgr2=0x7efd0410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 --2- 172.21.15.31:0/1814106802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd04108cf0 0x7efd0410f520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009ca0 tx=0x7efcfc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 -- 172.21.15.31:0/1814106802 shutdown_connections 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 --2- 172.21.15.31:0/1814106802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd04108cf0 0x7efd0410f520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 --2- 172.21.15.31:0/1814106802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd041083a0 0x7efd041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 --2- 172.21.15.31:0/1814106802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd04102a60 0x7efd04102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 -- 172.21.15.31:0/1814106802 >> 172.21.15.31:0/1814106802 conn(0x7efd04079730 msgr2=0x7efd04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 -- 172.21.15.31:0/1814106802 shutdown_connections 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.993+0000 7efd0946a640 1 -- 172.21.15.31:0/1814106802 wait complete. 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 Processor -- start 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 -- start start 2024-09-21T06:16:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 0x7efd0419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd041083a0 0x7efd0419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd04108cf0 0x7efd0419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd04112960 con 0x7efd041083a0 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd041127e0 con 0x7efd04102a60 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd0946a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd04112ae0 con 0x7efd04108cf0 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd027fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd041083a0 0x7efd0419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd02ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 0x7efd0419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd02ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 0x7efd0419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46610/0 (socket says 172.21.15.31:46610) 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd02ffd640 1 -- 172.21.15.31:0/665189091 learned_addr learned my addr 172.21.15.31:0/665189091 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.994+0000 7efd037fe640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd04108cf0 0x7efd0419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd02ffd640 1 -- 172.21.15.31:0/665189091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd04108cf0 msgr2=0x7efd0419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd02ffd640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd04108cf0 0x7efd0419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd02ffd640 1 -- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd041083a0 msgr2=0x7efd0419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd02ffd640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd041083a0 0x7efd0419b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd02ffd640 1 -- 172.21.15.31:0/665189091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd041a65a0 con 0x7efd04102a60 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd02ffd640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 0x7efd0419b310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcf400d950 tx=0x7efcf400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efcebfff640 1 -- 172.21.15.31:0/665189091 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4017030 con 0x7efd04102a60 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd041a6890 con 0x7efd04102a60 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efcebfff640 1 -- 172.21.15.31:0/665189091 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf4004020 con 0x7efd04102a60 2024-09-21T06:16:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efcebfff640 1 -- 172.21.15.31:0/665189091 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf40047a0 con 0x7efd04102a60 2024-09-21T06:16:06.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.995+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd041a6e20 con 0x7efd04102a60 2024-09-21T06:16:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 Processor -- start 2024-09-21T06:16:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 -- start start 2024-09-21T06:16:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56541c130ea0 con 0x56541bdf2390 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56541c1202d0 con 0x56541bdea1b0 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd0f9de740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56541c1240e0 con 0x56541be13070 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd027fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd027fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43046/0 (socket says 172.21.15.31:43046) 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1ba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58354/0 (socket says 172.21.15.31:58354) 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.998+0000 7efd027fc640 1 -- 172.21.15.31:0/1471519034 learned_addr learned my addr 172.21.15.31:0/1471519034 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efd037fe640 1 -- 172.21.15.31:0/1471519034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 msgr2=0x56541c1ba800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efd037fe640 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1ba800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efd037fe640 1 -- 172.21.15.31:0/1471519034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 msgr2=0x56541c1bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efd037fe640 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efd037fe640 1 -- 172.21.15.31:0/1471519034 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541be7e650 con 0x56541be13070 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efcebfff640 1 -- 172.21.15.31:0/665189091 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efcf4009a30 con 0x7efd04102a60 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efd037fe640 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1ba2c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005e30 tx=0x7efcfc02f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d1c94854a8d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efce97fa640 1 -- 172.21.15.31:0/1471519034 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc031cf0 con 0x56541be13070 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efce97fa640 1 -- 172.21.15.31:0/1471519034 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcfc036030 con 0x56541be13070 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efce97fa640 1 -- 172.21.15.31:0/1471519034 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc003e60 con 0x56541be13070 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:05.999+0000 7efcebfff640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcec0b66b0 0x7efcec0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd02ffd640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcec0b66b0 0x7efcec0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efcebfff640 1 -- 172.21.15.31:0/665189091 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efcf40e7070 con 0x7efd04102a60 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 -- 172.21.15.31:0/1471519034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 msgr2=0x56541c1ba2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1ba2c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005e30 tx=0x7efcfc02f4c0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 -- 172.21.15.31:0/1471519034 shutdown_connections 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1471519034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1ba2c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 -- 172.21.15.31:0/1471519034 >> 172.21.15.31:0/1471519034 conn(0x56541be07460 msgr2=0x56541c1b81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd02ffd640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcec0b66b0 0x7efcec0b8bc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x56541c1b8f40 tx=0x7efcf4021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 -- 172.21.15.31:0/1471519034 shutdown_connections 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.000+0000 7efd0f9de740 1 -- 172.21.15.31:0/1471519034 wait complete. 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 -- start start 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 Processor -- start 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56541bf81700 con 0x56541bdf2390 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56541c0bb160 con 0x56541bdea1b0 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56541c1c88c0 con 0x56541be13070 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd0f9de740 10 client.0 _ll_register_callbacks cb 0x56541c0d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd027fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd027fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43048/0 (socket says 172.21.15.31:43048) 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58358/0 (socket says 172.21.15.31:58358) 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd027fc640 1 -- 172.21.15.31:0/623808374 learned_addr learned my addr 172.21.15.31:0/623808374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.001+0000 7efd037fe640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd027fc640 1 -- 172.21.15.31:0/623808374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 msgr2=0x56541c1c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd027fc640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd027fc640 1 -- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 msgr2=0x56541c1c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd027fc640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd027fc640 1 -- 172.21.15.31:0/623808374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541c0e8c90 con 0x56541bdf2390 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd037fe640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd027fc640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c5e00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efcf8012740 tx=0x7efcf8012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd610cd707a31e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efce6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efce6ff5640 1 -- 172.21.15.31:0/623808374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800f920 con 0x56541bdf2390 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcfc036030 con 0x56541bdf2390 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efce6ff5640 1 -- 172.21.15.31:0/623808374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efcf800fac0 con 0x56541bdf2390 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efce6ff5640 1 -- 172.21.15.31:0/623808374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efcf800fcc0 con 0x56541bdf2390 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd0f9de740 10 client.16506 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.002+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56541c1d45c0 con 0x56541bdf2390 2024-09-21T06:16:06.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efce6ff5640 1 -- 172.21.15.31:0/623808374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcfc036030 con 0x56541bdf2390 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efce6ff5640 1 -- 172.21.15.31:0/623808374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efcf801c420 con 0x56541bdf2390 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd0f9de740 10 client.16506 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd0f9de740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd0f9de740 20 client.16506 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd0f9de740 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541bdd19f0 0x56541c1d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd0f9de740 4 client.16506 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56541c132df0 con 0x56541bdd19f0 2024-09-21T06:16:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.003+0000 7efd037fe640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541bdd19f0 0x56541c1d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.004+0000 7efce6ff5640 10 client.16506 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.004+0000 7efd037fe640 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541bdd19f0 0x56541c1d7ac0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35ed1485e84643b server_cookie=1894e3d0cb9e2848 in_seq=0 out_seq=0 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.004+0000 7efce6ff5640 1 -- 172.21.15.31:0/623808374 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56541c132df0 con 0x56541bdd19f0 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.004+0000 7efce6ff5640 10 client.16506 handle_command_reply: tid=0 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.004+0000 7efd0f9de740 1 client.16506 shutdown 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 msgr2=0x56541c1c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c5e00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efcf8012740 tx=0x7efcf8012c00 comp rx=0 tx=0).stop 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 shutdown_connections 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541bdd19f0 0x56541c1d7ac0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541be13070 0x56541c1c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efce6ff5640 0 client.16506 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c5e00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 --2- 172.21.15.31:0/623808374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 >> 172.21.15.31:0/623808374 conn(0x56541be07460 msgr2=0x56541c1bd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 shutdown_connections 2024-09-21T06:16:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 1 -- 172.21.15.31:0/623808374 wait complete. 2024-09-21T06:16:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.005+0000 7efd0f9de740 20 client.16506 trim_cache size 0 max 16384 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 Processor -- start 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 -- start start 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdd19f0 0x56541c1bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd041ad5c0 con 0x56541bdf2390 2024-09-21T06:16:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd041afe40 con 0x56541bdea1b0 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.019+0000 7efd0f9de740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd041adc30 con 0x56541bdd19f0 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd037fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1bba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46628/0 (socket says 172.21.15.31:46628) 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd037fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43056/0 (socket says 172.21.15.31:43056) 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 -- 172.21.15.31:0/1960693684 learned_addr learned my addr 172.21.15.31:0/1960693684 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd037fe640 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdd19f0 0x56541c1bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 -- 172.21.15.31:0/1960693684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdd19f0 msgr2=0x56541c1bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdd19f0 0x56541c1bdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 -- 172.21.15.31:0/1960693684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 msgr2=0x56541c1c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 -- 172.21.15.31:0/1960693684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541c1c6b80 con 0x56541bdea1b0 2024-09-21T06:16:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efd027fc640 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1bba00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcf8005bc0 tx=0x7efcf800f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f7329364578646 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.020+0000 7efce8ff9640 1 -- 172.21.15.31:0/1960693684 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8029030 con 0x56541bdea1b0 2024-09-21T06:16:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.021+0000 7efce8ff9640 1 -- 172.21.15.31:0/1960693684 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf8012e50 con 0x56541bdea1b0 2024-09-21T06:16:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.021+0000 7efce8ff9640 1 -- 172.21.15.31:0/1960693684 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf801cca0 con 0x56541bdea1b0 2024-09-21T06:16:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 -- 172.21.15.31:0/1960693684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 msgr2=0x56541c1bba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1bba00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcf8005bc0 tx=0x7efcf800f810 comp rx=0 tx=0).stop 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 -- 172.21.15.31:0/1960693684 shutdown_connections 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdea1b0 0x56541c1bba00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1960693684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdd19f0 0x56541c1bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 -- 172.21.15.31:0/1960693684 >> 172.21.15.31:0/1960693684 conn(0x56541be07460 msgr2=0x56541c1bb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 -- 172.21.15.31:0/1960693684 shutdown_connections 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 -- 172.21.15.31:0/1960693684 wait complete. 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.022+0000 7efd0f9de740 1 -- start start 2024-09-21T06:16:06.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 Processor -- start 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdd19f0 0x56541c1ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 0x56541c1c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd041ae300 con 0x56541bdf2390 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd041af770 con 0x56541bdd19f0 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd041ae9d0 con 0x56541bdea1b0 2024-09-21T06:16:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd0f9de740 10 client.0 _ll_register_callbacks cb 0x7efcf8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd037fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd027fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 0x56541c1c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd027fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 0x56541c1c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58370/0 (socket says 172.21.15.31:58370) 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.023+0000 7efd027fc640 1 -- 172.21.15.31:0/1799305648 learned_addr learned my addr 172.21.15.31:0/1799305648 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd037fe640 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdd19f0 0x56541c1ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd027fc640 1 -- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdd19f0 msgr2=0x56541c1ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd027fc640 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdd19f0 0x56541c1ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd027fc640 1 -- 172.21.15.31:0/1799305648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 msgr2=0x56541c1c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd027fc640 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd027fc640 1 -- 172.21.15.31:0/1799305648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541c1ca290 con 0x56541bdea1b0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd027fc640 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 0x56541c1c96b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efcf801cd30 tx=0x7efcf801cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7997414d0c250c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efce6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efce6ff5640 1 -- 172.21.15.31:0/1799305648 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8032020 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efce6ff5640 1 -- 172.21.15.31:0/1799305648 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efcf801c420 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.024+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf8012e50 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efce6ff5640 1 -- 172.21.15.31:0/1799305648 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efcf803d050 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 10 client.26269 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56541c2466e0 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efce6ff5640 1 -- 172.21.15.31:0/1799305648 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf801c660 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efce6ff5640 1 -- 172.21.15.31:0/1799305648 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efcf8003c10 con 0x56541bdea1b0 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 10 client.26269 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 20 client.26269 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541be13070 0x56541c248da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 4 client.26269 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.025+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efcf8012e50 con 0x56541be13070 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.026+0000 7efd037fe640 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541be13070 0x56541c248da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.026+0000 7efce6ff5640 10 client.26269 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.026+0000 7efd037fe640 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541be13070 0x56541c248da0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bae161b55b56c1 server_cookie=dc9b4f7b3e949efa in_seq=0 out_seq=0 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.026+0000 7efce6ff5640 1 -- 172.21.15.31:0/1799305648 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efcf8012e50 con 0x56541be13070 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.026+0000 7efce6ff5640 10 client.26269 handle_command_reply: tid=0 2024-09-21T06:16:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.026+0000 7efd0f9de740 1 client.26269 shutdown 2024-09-21T06:16:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 msgr2=0x56541c1c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 0x56541c1c96b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efcf801cd30 tx=0x7efcf801cd60 comp rx=0 tx=0).stop 2024-09-21T06:16:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 shutdown_connections 2024-09-21T06:16:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56541be13070 0x56541c248da0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56541bdf2390 0x56541c1c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56541bdea1b0 0x56541c1c96b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 --2- 172.21.15.31:0/1799305648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56541bdd19f0 0x56541c1ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 >> 172.21.15.31:0/1799305648 conn(0x56541be07460 msgr2=0x56541c1bdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efce6ff5640 0 client.26269 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 shutdown_connections 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 1 -- 172.21.15.31:0/1799305648 wait complete. 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.027+0000 7efd0f9de740 20 client.26269 trim_cache size 0 max 16384 2024-09-21T06:16:06.033 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:16:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcec0b66b0 msgr2=0x7efcec0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcec0b66b0 0x7efcec0b8bc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x56541c1b8f40 tx=0x7efcf4021000 comp rx=0 tx=0).stop 2024-09-21T06:16:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 msgr2=0x7efd0419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 0x7efd0419b310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcf400d950 tx=0x7efcf400de10 comp rx=0 tx=0).stop 2024-09-21T06:16:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 shutdown_connections 2024-09-21T06:16:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd04108cf0 0x7efd0419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcec0b66b0 0x7efcec0b8bc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd041083a0 0x7efd0419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 --2- 172.21.15.31:0/665189091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd04102a60 0x7efd0419b310 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 >> 172.21.15.31:0/665189091 conn(0x7efd04079730 msgr2=0x7efd04104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 shutdown_connections 2024-09-21T06:16:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:06.028+0000 7efd0946a640 1 -- 172.21.15.31:0/665189091 wait complete. 2024-09-21T06:16:07.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:06 smithi110 ceph-mon[34550]: pgmap v455: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 7.3 MiB/s wr, 932 op/s 2024-09-21T06:16:07.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:06 smithi107 ceph-mon[34380]: pgmap v455: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 7.3 MiB/s wr, 932 op/s 2024-09-21T06:16:07.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:06 smithi031 ceph-mon[31237]: pgmap v455: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 7.3 MiB/s wr, 932 op/s 2024-09-21T06:16:07.975 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:16:07.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:16:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:08.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.817+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/2727991074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a481048a0 msgr2=0x7f4a48104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.817+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/2727991074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a481048a0 0x7f4a48104cb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009900 tx=0x7f4a3802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/2727991074 shutdown_connections 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/2727991074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a4810de50 0x7f4a48110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/2727991074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 0x7f4a4810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/2727991074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a481048a0 0x7f4a48104cb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/2727991074 >> 172.21.15.31:0/2727991074 conn(0x7f4a48079790 msgr2=0x7f4a48079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/2727991074 shutdown_connections 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.818+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/2727991074 wait complete. 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 Processor -- start 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 -- start start 2024-09-21T06:16:08.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a481048a0 0x7f4a4819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 0x7f4a4819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 0x7f4a4819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48113ab0 con 0x7f4a4810de50 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48113930 con 0x7f4a48105280 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4d1f3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48113c30 con 0x7f4a481048a0 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 0x7f4a4819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 0x7f4a4819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43084/0 (socket says 172.21.15.31:43084) 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 0x7f4a4819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a4759e640 1 -- 172.21.15.31:0/269954758 learned_addr learned my addr 172.21.15.31:0/269954758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.819+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a481048a0 0x7f4a4819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4659c640 1 -- 172.21.15.31:0/269954758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a481048a0 msgr2=0x7f4a4819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4659c640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a481048a0 0x7f4a4819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4659c640 1 -- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 msgr2=0x7f4a4819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4659c640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 0x7f4a4819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4659c640 1 -- 172.21.15.31:0/269954758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a481a0410 con 0x7f4a48105280 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a481048a0 0x7f4a4819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4759e640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 0x7f4a4819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4659c640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 0x7f4a4819aa40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a4806ab20 tx=0x7f4a3c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a2ffff640 1 -- 172.21.15.31:0/269954758 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00cc40 con 0x7f4a48105280 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a2ffff640 1 -- 172.21.15.31:0/269954758 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a3c00cde0 con 0x7f4a48105280 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a481a0700 con 0x7f4a48105280 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.820+0000 7f4a2ffff640 1 -- 172.21.15.31:0/269954758 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c018780 con 0x7f4a48105280 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.821+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a481a6e60 con 0x7f4a48105280 2024-09-21T06:16:08.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.823+0000 7f4a53767740 1 Processor -- start 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.823+0000 7f4a53767740 1 -- start start 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a53767740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e824f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a53767740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e824f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a53767740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e824feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a53767740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0e81c5e60 con 0x55e0e7ea8350 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a53767740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0e81b5290 con 0x55e0e7e7f490 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a53767740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e0e81b90a0 con 0x55e0e7e87670 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e824f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e824f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43100/0 (socket says 172.21.15.31:43100) 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 -- 172.21.15.31:0/1920956939 learned_addr learned my addr 172.21.15.31:0/1920956939 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e824f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a2ffff640 1 -- 172.21.15.31:0/269954758 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4a3c010430 con 0x7f4a48105280 2024-09-21T06:16:08.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 -- 172.21.15.31:0/1920956939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 msgr2=0x55e0e824feb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e824feb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 -- 172.21.15.31:0/1920956939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 msgr2=0x55e0e824f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e824f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a4759e640 1 -- 172.21.15.31:0/1920956939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e7f13930 con 0x55e0e7ea8350 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.824+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e824f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.825+0000 7f4a4759e640 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e824f430 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4a400118b0 tx=0x7f4a40011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0acfb0af6b1cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.825+0000 7f4a2d7fa640 1 -- 172.21.15.31:0/1920956939 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4001e070 con 0x55e0e7ea8350 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.825+0000 7f4a2d7fa640 1 -- 172.21.15.31:0/1920956939 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4000fe70 con 0x55e0e7ea8350 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.825+0000 7f4a2d7fa640 1 -- 172.21.15.31:0/1920956939 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40019d00 con 0x55e0e7ea8350 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.825+0000 7f4a2ffff640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4a300b6680 0x7f4a300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.825+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4a300b6680 0x7f4a300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a2ffff640 1 -- 172.21.15.31:0/269954758 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4a3c0e7980 con 0x7f4a48105280 2024-09-21T06:16:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 -- 172.21.15.31:0/1920956939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 msgr2=0x55e0e824f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e824f430 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4a400118b0 tx=0x7f4a40011d70 comp rx=0 tx=0).stop 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 -- 172.21.15.31:0/1920956939 shutdown_connections 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e824feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e824f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 --2- 172.21.15.31:0/1920956939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e824f430 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 -- 172.21.15.31:0/1920956939 >> 172.21.15.31:0/1920956939 conn(0x55e0e7e9c740 msgr2=0x55e0e824d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 -- 172.21.15.31:0/1920956939 shutdown_connections 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a53767740 1 -- 172.21.15.31:0/1920956939 wait complete. 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.826+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4a300b6680 0x7f4a300b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a38031440 tx=0x7f4a38032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 -- start start 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 Processor -- start 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e825afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7e87670 0x55e0e825b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7ea8350 0x55e0e8266f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e0e8016890 con 0x55e0e7e87670 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0e814fd40 con 0x55e0e7e7f490 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e0e825dc10 con 0x55e0e7ea8350 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a53767740 10 client.0 _ll_register_callbacks cb 0x55e0e81b3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7e87670 0x55e0e825b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e825afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7e87670 0x55e0e825b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43106/0 (socket says 172.21.15.31:43106) 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a4659c640 1 -- 172.21.15.31:0/509823966 learned_addr learned my addr 172.21.15.31:0/509823966 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.827+0000 7f4a4759e640 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7ea8350 0x55e0e8266f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a46d9d640 1 -- 172.21.15.31:0/509823966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7ea8350 msgr2=0x55e0e8266f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7ea8350 0x55e0e8266f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a46d9d640 1 -- 172.21.15.31:0/509823966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7e87670 msgr2=0x55e0e825b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7e87670 0x55e0e825b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a46d9d640 1 -- 172.21.15.31:0/509823966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e82674c0 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e825afe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009230 tx=0x7f4a38009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f34497ca6b94b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a2aff5640 1 -- 172.21.15.31:0/509823966 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3804d070 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a2aff5640 1 -- 172.21.15.31:0/509823966 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4a38004250 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4000fe70 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a2aff5640 1 -- 172.21.15.31:0/509823966 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4a38059050 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a53767740 10 client.26243 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.828+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e0e8269df0 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a2aff5640 1 -- 172.21.15.31:0/509823966 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4000fe70 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a2aff5640 1 -- 172.21.15.31:0/509823966 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4a380037b0 con 0x55e0e7e7f490 2024-09-21T06:16:08.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a53767740 10 client.26243 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a53767740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a53767740 20 client.26243 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a53767740 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e66cd0 0x55e0e826d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a53767740 4 client.26243 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e81c7db0 con 0x55e0e7e66cd0 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a4659c640 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e66cd0 0x55e0e826d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.829+0000 7f4a2aff5640 10 client.26243 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.830+0000 7f4a4659c640 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e66cd0 0x55e0e826d2c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a03b8318454bde7c server_cookie=dbd3ecb1fd64f36e in_seq=0 out_seq=0 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.830+0000 7f4a2aff5640 1 -- 172.21.15.31:0/509823966 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0e81c7db0 con 0x55e0e7e66cd0 2024-09-21T06:16:08.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.830+0000 7f4a2aff5640 10 client.26243 handle_command_reply: tid=0 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.830+0000 7f4a53767740 1 client.26243 shutdown 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.830+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 msgr2=0x55e0e825afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.830+0000 7f4a53767740 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e825afe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009230 tx=0x7f4a38009260 comp rx=0 tx=0).stop 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 shutdown_connections 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e66cd0 0x55e0e826d2c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7ea8350 0x55e0e8266f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7e87670 0x55e0e825b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 --2- 172.21.15.31:0/509823966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e825afe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a2aff5640 0 client.26243 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 >> 172.21.15.31:0/509823966 conn(0x55e0e7e9c740 msgr2=0x55e0e8252610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 shutdown_connections 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 1 -- 172.21.15.31:0/509823966 wait complete. 2024-09-21T06:16:08.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.831+0000 7f4a53767740 20 client.26243 trim_cache size 0 max 16384 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 Processor -- start 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 -- start start 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e7f490 0x55e0e825a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e87670 0x55e0e8250b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e8257fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481ad670 con 0x55e0e7ea8350 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481afef0 con 0x55e0e7e87670 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a53767740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481adce0 con 0x55e0e7e7f490 2024-09-21T06:16:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e8257fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e8257fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43114/0 (socket says 172.21.15.31:43114) 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e7f490 0x55e0e825a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a4759e640 1 -- 172.21.15.31:0/2725823494 learned_addr learned my addr 172.21.15.31:0/2725823494 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.849+0000 7f4a4659c640 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e87670 0x55e0e8250b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a4759e640 1 -- 172.21.15.31:0/2725823494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e7f490 msgr2=0x55e0e825a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a4759e640 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e7f490 0x55e0e825a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a4759e640 1 -- 172.21.15.31:0/2725823494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e87670 msgr2=0x55e0e8250b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a4759e640 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e87670 0x55e0e8250b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a4759e640 1 -- 172.21.15.31:0/2725823494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e8251720 con 0x55e0e7ea8350 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e7f490 0x55e0e825a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a4759e640 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e8257fd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4a40011850 tx=0x7f4a400195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b4b11b04dfc4aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a2cff9640 1 -- 172.21.15.31:0/2725823494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40028020 con 0x55e0e7ea8350 2024-09-21T06:16:08.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a2cff9640 1 -- 172.21.15.31:0/2725823494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4001f040 con 0x55e0e7ea8350 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.850+0000 7f4a2cff9640 1 -- 172.21.15.31:0/2725823494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4001e070 con 0x55e0e7ea8350 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 -- 172.21.15.31:0/2725823494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 msgr2=0x55e0e8257fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e8257fd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4a40011850 tx=0x7f4a400195e0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 -- 172.21.15.31:0/2725823494 shutdown_connections 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e8257fd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e87670 0x55e0e8250b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 --2- 172.21.15.31:0/2725823494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e7f490 0x55e0e825a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 -- 172.21.15.31:0/2725823494 >> 172.21.15.31:0/2725823494 conn(0x55e0e7e9c740 msgr2=0x55e0e8250460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 -- 172.21.15.31:0/2725823494 shutdown_connections 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.851+0000 7f4a53767740 1 -- 172.21.15.31:0/2725823494 wait complete. 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 -- start start 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 Processor -- start 2024-09-21T06:16:08.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e82d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e82d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e825f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481ae3b0 con 0x55e0e7ea8350 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481af820 con 0x55e0e7e7f490 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a481aea80 con 0x55e0e7e87670 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a53767740 10 client.0 _ll_register_callbacks cb 0x7f4a38005e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e82d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e82d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a46d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e82d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46688/0 (socket says 172.21.15.31:46688) 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a4659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e82d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58412/0 (socket says 172.21.15.31:58412) 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a46d9d640 1 -- 172.21.15.31:0/1974505084 learned_addr learned my addr 172.21.15.31:0/1974505084 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.852+0000 7f4a4759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e825f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a46d9d640 1 -- 172.21.15.31:0/1974505084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 msgr2=0x55e0e82d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e82d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a46d9d640 1 -- 172.21.15.31:0/1974505084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 msgr2=0x55e0e825f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e825f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a46d9d640 1 -- 172.21.15.31:0/1974505084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e825fd50 con 0x55e0e7e7f490 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a46d9d640 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e82d0f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009230 tx=0x7f4a38032fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c0e917a70aa181e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a2aff5640 1 -- 172.21.15.31:0/1974505084 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38062070 con 0x55e0e7e7f490 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a2aff5640 1 -- 172.21.15.31:0/1974505084 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4a38004b80 con 0x55e0e7e7f490 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4001f040 con 0x55e0e7e7f490 2024-09-21T06:16:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a2aff5640 1 -- 172.21.15.31:0/1974505084 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4a3806e050 con 0x55e0e7e7f490 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a53767740 10 client.26249 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.853+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e0e82d8790 con 0x55e0e7e7f490 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a2aff5640 1 -- 172.21.15.31:0/1974505084 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a4001f040 con 0x55e0e7e7f490 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a2aff5640 1 -- 172.21.15.31:0/1974505084 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4a3803d610 con 0x55e0e7e7f490 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a53767740 10 client.26249 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a53767740 10 client.26249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a53767740 20 client.26249 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a53767740 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e3c080 0x55e0e82daf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a53767740 4 client.26249 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e0e81c3860 con 0x55e0e7e3c080 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.854+0000 7f4a4659c640 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e3c080 0x55e0e82daf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.855+0000 7f4a4659c640 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e3c080 0x55e0e82daf50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e511dc895b0797b4 server_cookie=b699fe7a15bec5c2 in_seq=0 out_seq=0 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.855+0000 7f4a2aff5640 10 client.26249 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a2aff5640 1 -- 172.21.15.31:0/1974505084 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e0e81c3860 con 0x55e0e7e3c080 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a2aff5640 10 client.26249 handle_command_reply: tid=0 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 client.26249 shutdown 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 msgr2=0x55e0e82d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e82d0f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009230 tx=0x7f4a38032fc0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 shutdown_connections 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e0e7e3c080 0x55e0e82daf50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e0e7ea8350 0x55e0e825f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e0e7e87670 0x55e0e82d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 --2- 172.21.15.31:0/1974505084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0e7e7f490 0x55e0e82d0f70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 >> 172.21.15.31:0/1974505084 conn(0x55e0e7e9c740 msgr2=0x55e0e824d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a2aff5640 0 client.26249 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 shutdown_connections 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.856+0000 7f4a53767740 1 -- 172.21.15.31:0/1974505084 wait complete. 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a53767740 20 client.26249 trim_cache size 0 max 16384 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "88fd931c-cb53-4956-bb22-4366d123d517", 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:16:08.872 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4a300b6680 msgr2=0x7f4a300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4a300b6680 0x7f4a300b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a38031440 tx=0x7f4a38032ed0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 msgr2=0x7f4a4819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 0x7f4a4819aa40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a4806ab20 tx=0x7f4a3c00ee50 comp rx=0 tx=0).stop 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 shutdown_connections 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4a300b6680 0x7f4a300b8b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a4810de50 0x7f4a4819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a48105280 0x7f4a4819aa40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 --2- 172.21.15.31:0/269954758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a481048a0 0x7f4a4819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:08.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 >> 172.21.15.31:0/269954758 conn(0x7f4a48079790 msgr2=0x7f4a48100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:08.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 shutdown_connections 2024-09-21T06:16:08.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:08.857+0000 7f4a4d1f3640 1 -- 172.21.15.31:0/269954758 wait complete. 2024-09-21T06:16:09.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:08 smithi031 ceph-mon[31237]: pgmap v456: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.6 MiB/s wr, 659 op/s 2024-09-21T06:16:09.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:08 smithi110 ceph-mon[34550]: pgmap v456: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.6 MiB/s wr, 659 op/s 2024-09-21T06:16:09.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:08 smithi107 ceph-mon[34380]: pgmap v456: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 4.6 MiB/s wr, 659 op/s 2024-09-21T06:16:10.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:09 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:16:10.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:09 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:10.091 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:09 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:16:10.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 270.0 s, 249.2 tps, lat 412.945 ms stddev 253.615 2024-09-21T06:16:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:09 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:16:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:09 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:10.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:09 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:16:10.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:09 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:16:10.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:09 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:10.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:09 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:16:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 -- 172.21.15.31:0/1898170357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c103b80 msgr2=0x7f483c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 --2- 172.21.15.31:0/1898170357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c103b80 0x7f483c103f90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f482c009900 tx=0x7f482c02a410 comp rx=0 tx=0).stop 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 -- 172.21.15.31:0/1898170357 shutdown_connections 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 --2- 172.21.15.31:0/1898170357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c109e10 0x7f483c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 --2- 172.21.15.31:0/1898170357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f483c1094c0 0x7f483c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 --2- 172.21.15.31:0/1898170357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c103b80 0x7f483c103f90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 -- 172.21.15.31:0/1898170357 >> 172.21.15.31:0/1898170357 conn(0x7f483c0ff610 msgr2=0x7f483c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 -- 172.21.15.31:0/1898170357 shutdown_connections 2024-09-21T06:16:10.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 -- 172.21.15.31:0/1898170357 wait complete. 2024-09-21T06:16:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.967+0000 7f4841c94640 1 Processor -- start 2024-09-21T06:16:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 -- start start 2024-09-21T06:16:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f483c103b80 0x7f483c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 0x7f483c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 0x7f483c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f483c113ad0 con 0x7f483c109e10 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f483c113950 con 0x7f483c1094c0 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f4841c94640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f483c113c50 con 0x7f483c103b80 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f483bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 0x7f483c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f483affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 0x7f483c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f483bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 0x7f483c076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35872/0 (socket says 172.21.15.31:35872) 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f483affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 0x7f483c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58008/0 (socket says 172.21.15.31:58008) 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f483bfff640 1 -- 172.21.15.31:0/1392459145 learned_addr learned my addr 172.21.15.31:0/1392459145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.968+0000 7f483b7fe640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f483c103b80 0x7f483c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483affd640 1 -- 172.21.15.31:0/1392459145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f483c103b80 msgr2=0x7f483c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483affd640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f483c103b80 0x7f483c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483affd640 1 -- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 msgr2=0x7f483c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483affd640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 0x7f483c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483affd640 1 -- 172.21.15.31:0/1392459145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483c1a8740 con 0x7f483c1094c0 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483bfff640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 0x7f483c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f483affd640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 0x7f483c075c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f483c06cc00 tx=0x7f483000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f4838ff9640 1 -- 172.21.15.31:0/1392459145 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483000cc40 con 0x7f483c1094c0 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483c1a8a30 con 0x7f483c1094c0 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f4838ff9640 1 -- 172.21.15.31:0/1392459145 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483000cde0 con 0x7f483c1094c0 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f4838ff9640 1 -- 172.21.15.31:0/1392459145 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830018760 con 0x7f483c1094c0 2024-09-21T06:16:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.969+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483c1a8fc0 con 0x7f483c1094c0 2024-09-21T06:16:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 Processor -- start 2024-09-21T06:16:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 -- start start 2024-09-21T06:16:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4838ff9640 1 -- 172.21.15.31:0/1392459145 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4830010a90 con 0x7f483c1094c0 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e8ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c86e844e50 con 0x55c86e506390 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c86e834280 con 0x55c86e527070 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.972+0000 7f4848208740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c86e838090 con 0x55c86e4fe1b0 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e8ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:54232/0 (socket says 172.21.15.31:54232) 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 -- 172.21.15.31:0/2980573276 learned_addr learned my addr 172.21.15.31:0/2980573276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483affd640 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 -- 172.21.15.31:0/2980573276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 msgr2=0x55c86e8ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e8ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 -- 172.21.15.31:0/2980573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 msgr2=0x55c86e8cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483b7fe640 1 -- 172.21.15.31:0/2980573276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86e592650 con 0x55c86e4fe1b0 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483bfff640 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e8ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:10.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f483affd640 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.973+0000 7f4838ff9640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48240b6680 0x7f48240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.974+0000 7f483b7fe640 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8ce7b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f482c009900 tx=0x7f482c032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89edd0201040e246 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.974+0000 7f483bfff640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48240b6680 0x7f48240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.974+0000 7f4838ff9640 1 -- 172.21.15.31:0/1392459145 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f48300e7960 con 0x7f483c1094c0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.974+0000 7f4821ffb640 1 -- 172.21.15.31:0/2980573276 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c040070 con 0x55c86e4fe1b0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.974+0000 7f4821ffb640 1 -- 172.21.15.31:0/2980573276 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482c004030 con 0x55c86e4fe1b0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.974+0000 7f4821ffb640 1 -- 172.21.15.31:0/2980573276 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c035040 con 0x55c86e4fe1b0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 -- 172.21.15.31:0/2980573276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 msgr2=0x55c86e8ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8ce7b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f482c009900 tx=0x7f482c032b20 comp rx=0 tx=0).stop 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f483bfff640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48240b6680 0x7f48240b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48340049d0 tx=0x7f4834009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 -- 172.21.15.31:0/2980573276 shutdown_connections 2024-09-21T06:16:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8ce7b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 --2- 172.21.15.31:0/2980573276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e8ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 -- 172.21.15.31:0/2980573276 >> 172.21.15.31:0/2980573276 conn(0x55c86e51b460 msgr2=0x55c86e8cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 -- 172.21.15.31:0/2980573276 shutdown_connections 2024-09-21T06:16:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.975+0000 7f4848208740 1 -- 172.21.15.31:0/2980573276 wait complete. 2024-09-21T06:16:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 -- start start 2024-09-21T06:16:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 Processor -- start 2024-09-21T06:16:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c86e695700 con 0x55c86e506390 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c86e7cf110 con 0x55c86e4fe1b0 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.976+0000 7f4848208740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c86e8dc9f0 con 0x55c86e527070 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8e5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:54240/0 (socket says 172.21.15.31:54240) 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 -- 172.21.15.31:0/3158136343 learned_addr learned my addr 172.21.15.31:0/3158136343 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f4848208740 10 client.0 _ll_register_callbacks cb 0x55c86e7e8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483affd640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 -- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e4fe1b0 msgr2=0x55c86e8d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 -- 172.21.15.31:0/3158136343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 msgr2=0x55c86e8d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8d9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 -- 172.21.15.31:0/3158136343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86e7fcc40 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483affd640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f483bfff640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8e5ce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4834011470 tx=0x7f4834004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9c1219d66349d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f481affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.977+0000 7f481affd640 1 -- 172.21.15.31:0/3158136343 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4834035060 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f481affd640 1 -- 172.21.15.31:0/3158136343 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f4834018030 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c004030 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f481affd640 1 -- 172.21.15.31:0/3158136343 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4834011dd0 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 10 client.26296 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c86e8e8730 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f481affd640 1 -- 172.21.15.31:0/3158136343 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482c004030 con 0x55c86e527070 2024-09-21T06:16:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f481affd640 1 -- 172.21.15.31:0/3158136343 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f483401d460 con 0x55c86e527070 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 10 client.26296 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 20 client.26296 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4e59f0 0x55c86e8ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 4 client.26296 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.978+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86e846da0 con 0x55c86e4e59f0 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.979+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4e59f0 0x55c86e8ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.979+0000 7f481affd640 10 client.26296 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.979+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4e59f0 0x55c86e8ebc50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66113f27544ee736 server_cookie=f225c401523b3774 in_seq=0 out_seq=0 2024-09-21T06:16:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f481affd640 1 -- 172.21.15.31:0/3158136343 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c86e846da0 con 0x55c86e4e59f0 2024-09-21T06:16:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f481affd640 10 client.26296 handle_command_reply: tid=0 2024-09-21T06:16:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 client.26296 shutdown 2024-09-21T06:16:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 msgr2=0x55c86e8e5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8e5ce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4834011470 tx=0x7f4834004ab0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 shutdown_connections 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4e59f0 0x55c86e8ebc50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8e5ce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 --2- 172.21.15.31:0/3158136343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 >> 172.21.15.31:0/3158136343 conn(0x55c86e51b460 msgr2=0x55c86e8d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f481affd640 0 client.26296 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 shutdown_connections 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.980+0000 7f4848208740 1 -- 172.21.15.31:0/3158136343 wait complete. 2024-09-21T06:16:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.981+0000 7f4848208740 20 client.26296 trim_cache size 0 max 16384 2024-09-21T06:16:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.998+0000 7f4848208740 1 Processor -- start 2024-09-21T06:16:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 -- start start 2024-09-21T06:16:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e506390 0x55c86e8cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1ab4c0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1add40 con 0x55c86e506390 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f4848208740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1abb30 con 0x55c86e527070 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35902/0 (socket says 172.21.15.31:35902) 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 -- 172.21.15.31:0/1453203099 learned_addr learned my addr 172.21.15.31:0/1453203099 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483affd640 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e506390 0x55c86e8cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483bfff640 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 -- 172.21.15.31:0/1453203099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 msgr2=0x55c86e8cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 -- 172.21.15.31:0/1453203099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e506390 msgr2=0x55c86e8cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e506390 0x55c86e8cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:10.999+0000 7f483b7fe640 1 -- 172.21.15.31:0/1453203099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86e8d8820 con 0x55c86e4fe1b0 2024-09-21T06:16:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.000+0000 7f483b7fe640 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9df0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55c86e8ccef0 tx=0x7f482c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bed9e93cb0e6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.000+0000 7f48217fa640 1 -- 172.21.15.31:0/1453203099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c052030 con 0x55c86e4fe1b0 2024-09-21T06:16:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.000+0000 7f48217fa640 1 -- 172.21.15.31:0/1453203099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482c0318e0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.000+0000 7f48217fa640 1 -- 172.21.15.31:0/1453203099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c035040 con 0x55c86e4fe1b0 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 -- 172.21.15.31:0/1453203099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 msgr2=0x55c86e8d9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9df0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55c86e8ccef0 tx=0x7f482c02a410 comp rx=0 tx=0).stop 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 -- 172.21.15.31:0/1453203099 shutdown_connections 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e527070 0x55c86e8cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e506390 0x55c86e8cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 --2- 172.21.15.31:0/1453203099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e4fe1b0 0x55c86e8d9df0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 -- 172.21.15.31:0/1453203099 >> 172.21.15.31:0/1453203099 conn(0x55c86e51b460 msgr2=0x55c86e8d96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 -- 172.21.15.31:0/1453203099 shutdown_connections 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 -- 172.21.15.31:0/1453203099 wait complete. 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.001+0000 7f4848208740 1 -- start start 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 Processor -- start 2024-09-21T06:16:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e955610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1ac200 con 0x55c86e506390 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1ad670 con 0x55c86e527070 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f483c1ac8d0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f4848208740 10 client.0 _ll_register_callbacks cb 0x7f4834012960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f483affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8de360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:54260/0 (socket says 172.21.15.31:54260) 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f483b7fe640 1 -- 172.21.15.31:0/3983005223 learned_addr learned my addr 172.21.15.31:0/3983005223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.002+0000 7f483bfff640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e955610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483b7fe640 1 -- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 msgr2=0x55c86e955610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e955610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483b7fe640 1 -- 172.21.15.31:0/3983005223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 msgr2=0x55c86e8de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483b7fe640 1 -- 172.21.15.31:0/3983005223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86e955b50 con 0x55c86e4fe1b0 2024-09-21T06:16:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483bfff640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e955610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483affd640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f483b7fe640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8de360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f482c009850 tx=0x7f482c032d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea2699be6fd8aee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f481affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f481affd640 1 -- 172.21.15.31:0/3983005223 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c035070 con 0x55c86e4fe1b0 2024-09-21T06:16:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f481affd640 1 -- 172.21.15.31:0/3983005223 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f482c0316e0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c0318e0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f481affd640 1 -- 172.21.15.31:0/3983005223 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f482c052040 con 0x55c86e4fe1b0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.003+0000 7f4848208740 10 client.26302 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c86e956de0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f481affd640 1 -- 172.21.15.31:0/3983005223 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482c0318e0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f481affd640 1 -- 172.21.15.31:0/3983005223 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f482c0049f0 con 0x55c86e4fe1b0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 10 client.26302 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 10 client.26302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 20 client.26302 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4bada0 0x55c86e9590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 4 client.26302 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c86e842850 con 0x55c86e4bada0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.004+0000 7f483affd640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4bada0 0x55c86e9590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f481affd640 10 client.26302 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f483affd640 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4bada0 0x55c86e9590c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=415a9da542de4a54 server_cookie=d9b0a9ea54078d12 in_seq=0 out_seq=0 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f481affd640 1 -- 172.21.15.31:0/3983005223 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55c86e842850 con 0x55c86e4bada0 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f481affd640 10 client.26302 handle_command_reply: tid=0 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 client.26302 shutdown 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 msgr2=0x55c86e8de360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8de360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f482c009850 tx=0x7f482c032d10 comp rx=0 tx=0).stop 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 shutdown_connections 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c86e4bada0 0x55c86e9590c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c86e527070 0x55c86e955610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c86e506390 0x55c86e8de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 --2- 172.21.15.31:0/3983005223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c86e4fe1b0 0x55c86e8de360 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f481affd640 0 client.26302 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 >> 172.21.15.31:0/3983005223 conn(0x55c86e51b460 msgr2=0x55c86e8cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 shutdown_connections 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.005+0000 7f4848208740 1 -- 172.21.15.31:0/3983005223 wait complete. 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4848208740 20 client.26302 trim_cache size 0 max 16384 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "88fd931c-cb53-4956-bb22-4366d123d517", 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.014 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:16:11.016 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.019 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.021 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:16:11.022 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48240b6680 msgr2=0x7f48240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48240b6680 0x7f48240b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f48340049d0 tx=0x7f4834009500 comp rx=0 tx=0).stop 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 msgr2=0x7f483c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 0x7f483c075c50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f483c06cc00 tx=0x7f483000ee50 comp rx=0 tx=0).stop 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 shutdown_connections 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48240b6680 0x7f48240b8b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c109e10 0x7f483c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f483c1094c0 0x7f483c075c50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 --2- 172.21.15.31:0/1392459145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f483c103b80 0x7f483c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 >> 172.21.15.31:0/1392459145 conn(0x7f483c0ff610 msgr2=0x7f483c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 shutdown_connections 2024-09-21T06:16:11.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:11.006+0000 7f4841c94640 1 -- 172.21.15.31:0/1392459145 wait complete. 2024-09-21T06:16:11.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:10 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:11.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:10 smithi031 ceph-mon[31237]: pgmap v457: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.5 MiB/s wr, 663 op/s 2024-09-21T06:16:11.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:10 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:11.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:10 smithi110 ceph-mon[34550]: pgmap v457: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.5 MiB/s wr, 663 op/s 2024-09-21T06:16:11.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:10 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:11.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:10 smithi107 ceph-mon[34380]: pgmap v457: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.5 MiB/s wr, 663 op/s 2024-09-21T06:16:11.961 INFO:tasks.cephfs.filesystem:scrub status for tag:88fd931c-cb53-4956-bb22-4366d123d517 - {'path': '/', 'tag': '88fd931c-cb53-4956-bb22-4366d123d517', 'options': 'recursive,force'} 2024-09-21T06:16:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:12 smithi110 ceph-mon[34550]: pgmap v458: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.4 MiB/s wr, 879 op/s 2024-09-21T06:16:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:12 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:16:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:12 smithi107 ceph-mon[34380]: pgmap v458: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.4 MiB/s wr, 879 op/s 2024-09-21T06:16:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:12 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:16:13.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:12 smithi031 ceph-mon[31237]: pgmap v458: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 6.4 MiB/s wr, 879 op/s 2024-09-21T06:16:13.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:12 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 -- 172.21.15.31:0/282724825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c1107b0 msgr2=0x7efd8c116fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 --2- 172.21.15.31:0/282724825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c1107b0 0x7efd8c116fe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efd84009ca0 tx=0x7efd8402c790 comp rx=0 tx=0).stop 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 -- 172.21.15.31:0/282724825 shutdown_connections 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 --2- 172.21.15.31:0/282724825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c1107b0 0x7efd8c116fe0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 --2- 172.21.15.31:0/282724825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd8c10fe60 0x7efd8c110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 --2- 172.21.15.31:0/282724825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10a660 0x7efd8c10aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 -- 172.21.15.31:0/282724825 >> 172.21.15.31:0/282724825 conn(0x7efd8c071220 msgr2=0x7efd8c073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 -- 172.21.15.31:0/282724825 shutdown_connections 2024-09-21T06:16:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.654+0000 7efd9428b640 1 -- 172.21.15.31:0/282724825 wait complete. 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.655+0000 7efd9428b640 1 Processor -- start 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.655+0000 7efd9428b640 1 -- start start 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.655+0000 7efd9428b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 0x7efd8c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.655+0000 7efd9428b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10fe60 0x7efd8c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.655+0000 7efd9428b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd8c1107b0 0x7efd8c1a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd9428b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c11a060 con 0x7efd8c10a660 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd9428b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c119ee0 con 0x7efd8c10fe60 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd9428b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c11a1e0 con 0x7efd8c1107b0 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd92027640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 0x7efd8c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd92027640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 0x7efd8c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35934/0 (socket says 172.21.15.31:35934) 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd92027640 1 -- 172.21.15.31:0/4233936745 learned_addr learned my addr 172.21.15.31:0/4233936745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.656+0000 7efd92828640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd8c1107b0 0x7efd8c1a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd91826640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10fe60 0x7efd8c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd92027640 1 -- 172.21.15.31:0/4233936745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd8c1107b0 msgr2=0x7efd8c1a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd92027640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd8c1107b0 0x7efd8c1a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd92027640 1 -- 172.21.15.31:0/4233936745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10fe60 msgr2=0x7efd8c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd92027640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10fe60 0x7efd8c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd92027640 1 -- 172.21.15.31:0/4233936745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c1a5740 con 0x7efd8c10a660 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd91826640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10fe60 0x7efd8c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.657+0000 7efd92027640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 0x7efd8c19f830 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efd88030ab0 tx=0x7efd88030f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.658+0000 7efd837fe640 1 -- 172.21.15.31:0/4233936745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88039070 con 0x7efd8c10a660 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.658+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1a5a30 con 0x7efd8c10a660 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.658+0000 7efd837fe640 1 -- 172.21.15.31:0/4233936745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd8802ee30 con 0x7efd8c10a660 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.658+0000 7efd837fe640 1 -- 172.21.15.31:0/4233936745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88003d70 con 0x7efd8c10a660 2024-09-21T06:16:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.658+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a61d0 con 0x7efd8c10a660 2024-09-21T06:16:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.663+0000 7efd9b1f8740 1 Processor -- start 2024-09-21T06:16:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.663+0000 7efd837fe640 1 -- 172.21.15.31:0/4233936745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efd8803d400 con 0x7efd8c10a660 2024-09-21T06:16:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.666+0000 7efd9b1f8740 1 -- start start 2024-09-21T06:16:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.666+0000 7efd837fe640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd780b67d0 0x7efd780b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.666+0000 7efd92828640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd780b67d0 0x7efd780b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.666+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.666+0000 7efd837fe640 1 -- 172.21.15.31:0/4233936745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efd88108dd0 con 0x7efd8c10a660 2024-09-21T06:16:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.666+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 0x5637052d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd92027640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd91826640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 0x5637052d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd92027640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35948/0 (socket says 172.21.15.31:35948) 2024-09-21T06:16:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd91826640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 0x5637052d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:54294/0 (socket says 172.21.15.31:54294) 2024-09-21T06:16:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd92828640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd780b67d0 0x7efd780b8ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efd84006010 tx=0x7efd84031f10 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd92027640 1 -- 172.21.15.31:0/597727260 learned_addr learned my addr 172.21.15.31:0/597727260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637052476f0 con 0x563704f2d1b0 2024-09-21T06:16:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563705236b20 con 0x563704f0c4d0 2024-09-21T06:16:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd92828640 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.667+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56370523a930 con 0x563704f042f0 2024-09-21T06:16:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd91826640 1 -- 172.21.15.31:0/597727260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 msgr2=0x5637052d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd91826640 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd91826640 1 -- 172.21.15.31:0/597727260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 msgr2=0x5637052d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd91826640 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd91826640 1 -- 172.21.15.31:0/597727260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637051fc530 con 0x563704f042f0 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd91826640 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 0x5637052d2ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd8c06c850 tx=0x7efd7c003b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9333ff51a8b0e498 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd80bfe640 1 -- 172.21.15.31:0/597727260 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00ab30 con 0x563704f042f0 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd80bfe640 1 -- 172.21.15.31:0/597727260 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c00acd0 con 0x563704f042f0 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.668+0000 7efd80bfe640 1 -- 172.21.15.31:0/597727260 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00d6d0 con 0x563704f042f0 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 msgr2=0x5637052d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 0x5637052d2ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd8c06c850 tx=0x7efd7c003b30 comp rx=0 tx=0).stop 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 shutdown_connections 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f042f0 0x5637052d2ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/597727260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 >> 172.21.15.31:0/597727260 conn(0x563704f21300 msgr2=0x5637052d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.669+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 shutdown_connections 2024-09-21T06:16:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.670+0000 7efd9b1f8740 1 -- 172.21.15.31:0/597727260 wait complete. 2024-09-21T06:16:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.670+0000 7efd9b1f8740 1 -- start start 2024-09-21T06:16:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 Processor -- start 2024-09-21T06:16:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56370516eab0 con 0x563704f2d1b0 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5637051dc730 con 0x563704f042f0 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637052e1130 con 0x563704f0c4d0 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.672+0000 7efd9b1f8740 10 client.0 _ll_register_callbacks cb 0x5637052306b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052ea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35964/0 (socket says 172.21.15.31:35964) 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 -- 172.21.15.31:0/212660556 learned_addr learned my addr 172.21.15.31:0/212660556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd91826640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92027640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 -- 172.21.15.31:0/212660556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 msgr2=0x5637052de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 -- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 msgr2=0x5637052de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.673+0000 7efd92828640 1 -- 172.21.15.31:0/212660556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563704f98790 con 0x563704f2d1b0 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.674+0000 7efd92828640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052ea420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efd84009ca0 tx=0x7efd8402c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e404c3505957697 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.674+0000 7efd75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:16:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.675+0000 7efd75ffb640 1 -- 172.21.15.31:0/212660556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84031be0 con 0x563704f2d1b0 2024-09-21T06:16:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.675+0000 7efd75ffb640 1 -- 172.21.15.31:0/212660556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efd84003970 con 0x563704f2d1b0 2024-09-21T06:16:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.675+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd7c00acd0 con 0x563704f2d1b0 2024-09-21T06:16:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.675+0000 7efd75ffb640 1 -- 172.21.15.31:0/212660556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efd84003b70 con 0x563704f2d1b0 2024-09-21T06:16:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.675+0000 7efd9b1f8740 10 client.16560 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.675+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563705011620 con 0x563704f2d1b0 2024-09-21T06:16:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd75ffb640 1 -- 172.21.15.31:0/212660556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c00acd0 con 0x563704f2d1b0 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd75ffb640 1 -- 172.21.15.31:0/212660556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efd8404c430 con 0x563704f2d1b0 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd9b1f8740 10 client.16560 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd9b1f8740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd9b1f8740 20 client.16560 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704eebd80 0x5637052f0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd9b1f8740 4 client.16560 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563705249640 con 0x563704eebd80 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.676+0000 7efd92027640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704eebd80 0x5637052f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.677+0000 7efd92027640 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704eebd80 0x5637052f0190 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740eb70393e970cd server_cookie=679e55fe79c7c999 in_seq=0 out_seq=0 2024-09-21T06:16:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.678+0000 7efd75ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.678+0000 7efd75ffb640 1 -- 172.21.15.31:0/212660556 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563705249640 con 0x563704eebd80 2024-09-21T06:16:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.678+0000 7efd75ffb640 10 client.16560 handle_command_reply: tid=0 2024-09-21T06:16:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.678+0000 7efd9b1f8740 1 client.16560 shutdown 2024-09-21T06:16:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 msgr2=0x5637052ea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052ea420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efd84009ca0 tx=0x7efd8402c790 comp rx=0 tx=0).stop 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 shutdown_connections 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704eebd80 0x5637052f0190 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052ea420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/212660556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.679+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 >> 172.21.15.31:0/212660556 conn(0x563704f21300 msgr2=0x5637052d39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.680+0000 7efd75ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.680+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 shutdown_connections 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.680+0000 7efd9b1f8740 1 -- 172.21.15.31:0/212660556 wait complete. 2024-09-21T06:16:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.680+0000 7efd9b1f8740 20 client.16560 trim_cache size 0 max 16384 2024-09-21T06:16:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.706+0000 7efd9b1f8740 1 Processor -- start 2024-09-21T06:16:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.706+0000 7efd9b1f8740 1 -- start start 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.706+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f042f0 0x5637052d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 0x5637052db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd9b1f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1adcd0 con 0x563704f042f0 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd9b1f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1b0550 con 0x563704f0c4d0 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd9b1f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ae340 con 0x563704f2d1b0 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd92828640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 0x5637052db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd92828640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 0x5637052db260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:54308/0 (socket says 172.21.15.31:54308) 2024-09-21T06:16:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.707+0000 7efd92828640 1 -- 172.21.15.31:0/3008459191 learned_addr learned my addr 172.21.15.31:0/3008459191 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd92828640 1 -- 172.21.15.31:0/3008459191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 msgr2=0x5637052cf5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd92828640 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052cf5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd92828640 1 -- 172.21.15.31:0/3008459191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f042f0 msgr2=0x5637052d43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd92828640 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f042f0 0x5637052d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd92828640 1 -- 172.21.15.31:0/3008459191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637052d11a0 con 0x563704f2d1b0 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd92828640 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 0x5637052db260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd8402cd80 tx=0x7efd84009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=980ecd6b87684e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd77fff640 1 -- 172.21.15.31:0/3008459191 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8404c430 con 0x563704f2d1b0 2024-09-21T06:16:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd77fff640 1 -- 172.21.15.31:0/3008459191 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd84050070 con 0x563704f2d1b0 2024-09-21T06:16:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.708+0000 7efd77fff640 1 -- 172.21.15.31:0/3008459191 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84054680 con 0x563704f2d1b0 2024-09-21T06:16:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 -- 172.21.15.31:0/3008459191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 msgr2=0x5637052db260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 0x5637052db260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd8402cd80 tx=0x7efd84009670 comp rx=0 tx=0).stop 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 -- 172.21.15.31:0/3008459191 shutdown_connections 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f2d1b0 0x5637052db260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f0c4d0 0x5637052cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/3008459191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f042f0 0x5637052d43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 -- 172.21.15.31:0/3008459191 >> 172.21.15.31:0/3008459191 conn(0x563704f21300 msgr2=0x5637052ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 -- 172.21.15.31:0/3008459191 shutdown_connections 2024-09-21T06:16:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.710+0000 7efd9b1f8740 1 -- 172.21.15.31:0/3008459191 wait complete. 2024-09-21T06:16:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.711+0000 7efd9b1f8740 1 -- start start 2024-09-21T06:16:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.711+0000 7efd9b1f8740 1 Processor -- start 2024-09-21T06:16:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.711+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd9b1f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd92027640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd92027640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58102/0 (socket says 172.21.15.31:58102) 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd91826640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd91826640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052e3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:54320/0 (socket says 172.21.15.31:54320) 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd92828640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd92828640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052e4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35968/0 (socket says 172.21.15.31:35968) 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd92027640 1 -- 172.21.15.31:0/4277968203 learned_addr learned my addr 172.21.15.31:0/4277968203 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1aea10 con 0x563704f2d1b0 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1afe80 con 0x563704f042f0 2024-09-21T06:16:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.712+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1af0e0 con 0x563704f0c4d0 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd9b1f8740 10 client.0 _ll_register_callbacks cb 0x7efd84009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd92027640 1 -- 172.21.15.31:0/4277968203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 msgr2=0x5637052e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd92027640 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052e3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd92027640 1 -- 172.21.15.31:0/4277968203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 msgr2=0x5637052e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd92027640 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052e4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd92027640 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637052e4840 con 0x563704f042f0 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd91826640 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.713+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd84050070 con 0x563704f042f0 2024-09-21T06:16:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.715+0000 7efd92027640 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052e68b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd880ce900 tx=0x7efd88005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee787c4a968d85d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.715+0000 7efd75ffb640 10 client.26282 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.715+0000 7efd75ffb640 1 -- 172.21.15.31:0/4277968203 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88042040 con 0x563704f042f0 2024-09-21T06:16:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.715+0000 7efd75ffb640 1 -- 172.21.15.31:0/4277968203 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efd880cae70 con 0x563704f042f0 2024-09-21T06:16:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.715+0000 7efd75ffb640 1 -- 172.21.15.31:0/4277968203 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efd88009d00 con 0x563704f042f0 2024-09-21T06:16:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 10 client.26282 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x56370535a720 con 0x563704f042f0 2024-09-21T06:16:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd75ffb640 1 -- 172.21.15.31:0/4277968203 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efd88045060 con 0x563704f042f0 2024-09-21T06:16:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 10 client.26282 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 10 client.26282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 20 client.26282 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704ec11c0 0x56370535cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 4 client.26282 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.716+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637052450f0 con 0x563704ec11c0 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.717+0000 7efd91826640 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704ec11c0 0x56370535cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.717+0000 7efd91826640 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704ec11c0 0x56370535cd50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330f92583a644ec2 server_cookie=e002039ec872df3a in_seq=0 out_seq=0 2024-09-21T06:16:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.717+0000 7efd75ffb640 10 client.26282 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.734+0000 7efd75ffb640 1 -- 172.21.15.31:0/4277968203 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637052450f0 con 0x563704ec11c0 2024-09-21T06:16:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.734+0000 7efd75ffb640 10 client.26282 handle_command_reply: tid=0 2024-09-21T06:16:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.734+0000 7efd9b1f8740 1 client.26282 shutdown 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 msgr2=0x5637052e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052e68b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efd880ce900 tx=0x7efd88005f10 comp rx=0 tx=0).stop 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 shutdown_connections 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563704ec11c0 0x56370535cd50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd75ffb640 0 client.26282 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563704f2d1b0 0x5637052e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563704f0c4d0 0x5637052e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 --2- 172.21.15.31:0/4277968203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563704f042f0 0x5637052e68b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 >> 172.21.15.31:0/4277968203 conn(0x563704f21300 msgr2=0x5637052d43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 shutdown_connections 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 1 -- 172.21.15.31:0/4277968203 wait complete. 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.735+0000 7efd9b1f8740 20 client.26282 trim_cache size 0 max 16384 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:16:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd780b67d0 msgr2=0x7efd780b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd780b67d0 0x7efd780b8ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efd84006010 tx=0x7efd84031f10 comp rx=0 tx=0).stop 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 msgr2=0x7efd8c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 0x7efd8c19f830 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efd88030ab0 tx=0x7efd88030f70 comp rx=0 tx=0).stop 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 shutdown_connections 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd8c1107b0 0x7efd8c1a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd8c10fe60 0x7efd8c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd780b67d0 0x7efd780b8ce0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 --2- 172.21.15.31:0/4233936745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd8c10a660 0x7efd8c19f830 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 >> 172.21.15.31:0/4233936745 conn(0x7efd8c071220 msgr2=0x7efd8c10f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 shutdown_connections 2024-09-21T06:16:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:14.736+0000 7efd9428b640 1 -- 172.21.15.31:0/4233936745 wait complete. 2024-09-21T06:16:15.187 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:14 smithi031 ceph-mon[31237]: pgmap v459: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 6.8 MiB/s wr, 687 op/s 2024-09-21T06:16:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 275.0 s, 709.8 tps, lat 145.410 ms stddev 410.757 2024-09-21T06:16:15.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:14 smithi110 ceph-mon[34550]: pgmap v459: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 6.8 MiB/s wr, 687 op/s 2024-09-21T06:16:15.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:14 smithi107 ceph-mon[34380]: pgmap v459: 97 pgs: 97 active+clean; 3.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 6.8 MiB/s wr, 687 op/s 2024-09-21T06:16:17.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:16 smithi031 ceph-mon[31237]: pgmap v460: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 31 MiB/s wr, 1.51k op/s 2024-09-21T06:16:17.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:16 smithi110 ceph-mon[34550]: pgmap v460: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 31 MiB/s wr, 1.51k op/s 2024-09-21T06:16:17.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:16 smithi107 ceph-mon[34380]: pgmap v460: 97 pgs: 97 active+clean; 3.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 31 MiB/s wr, 1.51k op/s 2024-09-21T06:16:19.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:18 smithi110 ceph-mon[34550]: pgmap v461: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 35 MiB/s wr, 1.48k op/s 2024-09-21T06:16:19.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:18 smithi107 ceph-mon[34380]: pgmap v461: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 35 MiB/s wr, 1.48k op/s 2024-09-21T06:16:19.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:18 smithi031 ceph-mon[31237]: pgmap v461: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 35 MiB/s wr, 1.48k op/s 2024-09-21T06:16:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 280.0 s, 2524.0 tps, lat 39.715 ms stddev 42.007 2024-09-21T06:16:21.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:20 smithi110 ceph-mon[34550]: pgmap v462: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2024-09-21T06:16:21.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:20 smithi107 ceph-mon[34380]: pgmap v462: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2024-09-21T06:16:21.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:20 smithi031 ceph-mon[31237]: pgmap v462: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2024-09-21T06:16:23.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:22 smithi110 ceph-mon[34550]: pgmap v463: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 38 MiB/s wr, 1.70k op/s 2024-09-21T06:16:23.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:22 smithi107 ceph-mon[34380]: pgmap v463: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 38 MiB/s wr, 1.70k op/s 2024-09-21T06:16:23.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:22 smithi031 ceph-mon[31237]: pgmap v463: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 38 MiB/s wr, 1.70k op/s 2024-09-21T06:16:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 285.0 s, 3327.8 tps, lat 29.995 ms stddev 33.652 2024-09-21T06:16:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:24 smithi110 ceph-mon[34550]: pgmap v464: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-09-21T06:16:25.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:24 smithi107 ceph-mon[34380]: pgmap v464: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-09-21T06:16:25.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:24 smithi031 ceph-mon[31237]: pgmap v464: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-09-21T06:16:27.276 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:26 smithi031 ceph-mon[31237]: pgmap v465: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.55k op/s 2024-09-21T06:16:27.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:26 smithi110 ceph-mon[34550]: pgmap v465: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.55k op/s 2024-09-21T06:16:27.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:26 smithi107 ceph-mon[34380]: pgmap v465: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 MiB/s wr, 1.55k op/s 2024-09-21T06:16:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.894+0000 7ff07dbc6640 1 -- 172.21.15.31:0/796786641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 msgr2=0x7ff078108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.894+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/796786641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff078108710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff068009900 tx=0x7ff06802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 -- 172.21.15.31:0/796786641 shutdown_connections 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/796786641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 0x7ff07810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/796786641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff078108710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/796786641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff078102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 -- 172.21.15.31:0/796786641 >> 172.21.15.31:0/796786641 conn(0x7ff0780fe410 msgr2=0x7ff078100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 -- 172.21.15.31:0/796786641 shutdown_connections 2024-09-21T06:16:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.895+0000 7ff07dbc6640 1 -- 172.21.15.31:0/796786641 wait complete. 2024-09-21T06:16:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 Processor -- start 2024-09-21T06:16:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 -- start start 2024-09-21T06:16:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff07819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff07819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 0x7ff07819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff078113a00 con 0x7ff078108c50 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff078113880 con 0x7ff0781029d0 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.897+0000 7ff07dbc6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff078113b80 con 0x7ff078108300 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff07819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff07819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff07819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42024/0 (socket says 172.21.15.31:42024) 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff07819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38054/0 (socket says 172.21.15.31:38054) 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 -- 172.21.15.31:0/32274033 learned_addr learned my addr 172.21.15.31:0/32274033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff077fff640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 0x7ff07819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 -- 172.21.15.31:0/32274033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 msgr2=0x7ff07819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff07819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 -- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 msgr2=0x7ff07819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 0x7ff07819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 -- 172.21.15.31:0/32274033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0781a03b0 con 0x7ff078108300 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff077fff640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 0x7ff07819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff0777fe640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff07819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.898+0000 7ff076ffd640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff07819aa20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff068006010 tx=0x7ff068004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.899+0000 7ff074ff9640 1 -- 172.21.15.31:0/32274033 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06803f030 con 0x7ff078108300 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.899+0000 7ff074ff9640 1 -- 172.21.15.31:0/32274033 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff068003aa0 con 0x7ff078108300 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.899+0000 7ff074ff9640 1 -- 172.21.15.31:0/32274033 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068035070 con 0x7ff078108300 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.899+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0781a0640 con 0x7ff078108300 2024-09-21T06:16:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.899+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0781a0a20 con 0x7ff078108300 2024-09-21T06:16:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.903+0000 7ff074ff9640 1 -- 172.21.15.31:0/32274033 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff068042030 con 0x7ff078108300 2024-09-21T06:16:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.904+0000 7ff074ff9640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff0600b67d0 0x7ff0600b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.904+0000 7ff074ff9640 1 -- 172.21.15.31:0/32274033 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff068107b80 con 0x7ff078108300 2024-09-21T06:16:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.906+0000 7ff084c19740 1 Processor -- start 2024-09-21T06:16:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.906+0000 7ff0777fe640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff0600b67d0 0x7ff0600b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.906+0000 7ff084c19740 1 -- start start 2024-09-21T06:16:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff084c19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff084c19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dd32f0 0x5639a91a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff084c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8ddb4d0 0x5639a91a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42036/0 (socket says 172.21.15.31:42036) 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5639a91166f0 con 0x5639a8dd32f0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639a9105b20 con 0x5639a8ddb4d0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5639a9109930 con 0x5639a8dfc1b0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.907+0000 7ff0777fe640 1 -- 172.21.15.31:0/713514566 learned_addr learned my addr 172.21.15.31:0/713514566 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff076ffd640 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dd32f0 0x5639a91a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff0600b67d0 0x7ff0600b8ce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff0700099a0 tx=0x7ff070009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff077fff640 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8ddb4d0 0x5639a91a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 -- 172.21.15.31:0/713514566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8ddb4d0 msgr2=0x5639a91a2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8ddb4d0 0x5639a91a2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 -- 172.21.15.31:0/713514566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dd32f0 msgr2=0x5639a91a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dd32f0 0x5639a91a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 -- 172.21.15.31:0/713514566 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a90cb530 con 0x5639a8dfc1b0 2024-09-21T06:16:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff077fff640 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8ddb4d0 0x5639a91a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:16:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.908+0000 7ff0777fe640 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91a19b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff07000fcf0 tx=0x7ff070014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f083f3a10f29cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.909+0000 7ff05dffb640 1 -- 172.21.15.31:0/713514566 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700273f0 con 0x5639a8dfc1b0 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.909+0000 7ff05dffb640 1 -- 172.21.15.31:0/713514566 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff070027a40 con 0x5639a8dfc1b0 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.909+0000 7ff05dffb640 1 -- 172.21.15.31:0/713514566 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07000e8a0 con 0x5639a8dfc1b0 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 msgr2=0x5639a91a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91a19b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff07000fcf0 tx=0x7ff070014000 comp rx=0 tx=0).stop 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 shutdown_connections 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8ddb4d0 0x5639a91a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dd32f0 0x5639a91a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 --2- 172.21.15.31:0/713514566 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91a19b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 >> 172.21.15.31:0/713514566 conn(0x5639a8df0300 msgr2=0x5639a919f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 shutdown_connections 2024-09-21T06:16:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.910+0000 7ff084c19740 1 -- 172.21.15.31:0/713514566 wait complete. 2024-09-21T06:16:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.911+0000 7ff084c19740 1 -- start start 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.911+0000 7ff084c19740 1 Processor -- start 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91ad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38078/0 (socket says 172.21.15.31:38078) 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff0777fe640 1 -- 172.21.15.31:0/688747989 learned_addr learned my addr 172.21.15.31:0/688747989 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5639a903dab0 con 0x5639a8ddb4d0 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639a90ab730 con 0x5639a8dd32f0 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5639a91b00d0 con 0x5639a8dfc1b0 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.912+0000 7ff084c19740 10 client.0 _ll_register_callbacks cb 0x5639a90ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff077fff640 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff0777fe640 1 -- 172.21.15.31:0/688747989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 msgr2=0x5639a91c1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff0777fe640 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91c1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff0777fe640 1 -- 172.21.15.31:0/688747989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 msgr2=0x5639a91ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff0777fe640 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff0777fe640 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a8e67790 con 0x5639a8dd32f0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff0777fe640 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91ad0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff070005c40 tx=0x7ff070027e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20bbbb3e07157ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.913+0000 7ff056ffd640 1 -- 172.21.15.31:0/688747989 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070004a40 con 0x5639a8dd32f0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.914+0000 7ff056ffd640 1 -- 172.21.15.31:0/688747989 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff070004be0 con 0x5639a8dd32f0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.914+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff070027a40 con 0x5639a8dd32f0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.914+0000 7ff056ffd640 1 -- 172.21.15.31:0/688747989 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff070004de0 con 0x5639a8dd32f0 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.914+0000 7ff084c19740 10 client.26294 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.914+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5639a8ee0620 con 0x5639a8dd32f0 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff056ffd640 1 -- 172.21.15.31:0/688747989 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff070027a40 con 0x5639a8dd32f0 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff056ffd640 1 -- 172.21.15.31:0/688747989 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff070026070 con 0x5639a8dd32f0 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff084c19740 10 client.26294 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff084c19740 10 client.26294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff084c19740 20 client.26294 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8dbad80 0x5639a91c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff084c19740 4 client.26294 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a9118640 con 0x5639a8dbad80 2024-09-21T06:16:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.915+0000 7ff076ffd640 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8dbad80 0x5639a91c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.916+0000 7ff076ffd640 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8dbad80 0x5639a91c78a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579fa088034b5fca server_cookie=d475dc9539f5ae23 in_seq=0 out_seq=0 2024-09-21T06:16:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.916+0000 7ff056ffd640 10 client.26294 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.917+0000 7ff056ffd640 1 -- 172.21.15.31:0/688747989 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5639a9118640 con 0x5639a8dbad80 2024-09-21T06:16:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.917+0000 7ff056ffd640 10 client.26294 handle_command_reply: tid=0 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.917+0000 7ff084c19740 1 client.26294 shutdown 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 msgr2=0x5639a91ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91ad0d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff070005c40 tx=0x7ff070027e20 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 shutdown_connections 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8dbad80 0x5639a91c78a0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dfc1b0 0x5639a91c1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 --2- 172.21.15.31:0/688747989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91ad0d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 >> 172.21.15.31:0/688747989 conn(0x5639a8df0300 msgr2=0x5639a91a29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff056ffd640 0 client.26294 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.918+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 shutdown_connections 2024-09-21T06:16:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.919+0000 7ff084c19740 1 -- 172.21.15.31:0/688747989 wait complete. 2024-09-21T06:16:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.919+0000 7ff084c19740 20 client.26294 trim_cache size 0 max 16384 2024-09-21T06:16:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.944+0000 7ff084c19740 1 Processor -- start 2024-09-21T06:16:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 -- start start 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8ddb4d0 0x5639a91a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dfc1b0 0x5639a91aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ab410 con 0x5639a8dfc1b0 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781adc90 con 0x5639a8dd32f0 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff084c19740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781aba80 con 0x5639a8ddb4d0 2024-09-21T06:16:27.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:38088/0 (socket says 172.21.15.31:38088) 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff077fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dfc1b0 0x5639a91aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.945+0000 7ff0777fe640 1 -- 172.21.15.31:0/776314460 learned_addr learned my addr 172.21.15.31:0/776314460 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.946+0000 7ff0777fe640 1 -- 172.21.15.31:0/776314460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8ddb4d0 msgr2=0x5639a91a87f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.946+0000 7ff0777fe640 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8ddb4d0 0x5639a91a87f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.946+0000 7ff0777fe640 1 -- 172.21.15.31:0/776314460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dfc1b0 msgr2=0x5639a91aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.946+0000 7ff0777fe640 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dfc1b0 0x5639a91aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.946+0000 7ff0777fe640 1 -- 172.21.15.31:0/776314460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a91a47c0 con 0x5639a8dd32f0 2024-09-21T06:16:27.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.946+0000 7ff0777fe640 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91a70e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff07000e430 tx=0x7ff07000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936cb3b6bd64faaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.947+0000 7ff05d7fa640 1 -- 172.21.15.31:0/776314460 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070004bd0 con 0x5639a8dd32f0 2024-09-21T06:16:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.947+0000 7ff05d7fa640 1 -- 172.21.15.31:0/776314460 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07001e4f0 con 0x5639a8dd32f0 2024-09-21T06:16:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.947+0000 7ff05d7fa640 1 -- 172.21.15.31:0/776314460 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070005960 con 0x5639a8dd32f0 2024-09-21T06:16:27.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 -- 172.21.15.31:0/776314460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 msgr2=0x5639a91a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91a70e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff07000e430 tx=0x7ff07000ef40 comp rx=0 tx=0).stop 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 -- 172.21.15.31:0/776314460 shutdown_connections 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8dfc1b0 0x5639a91aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8ddb4d0 0x5639a91a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 --2- 172.21.15.31:0/776314460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dd32f0 0x5639a91a70e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.948+0000 7ff084c19740 1 -- 172.21.15.31:0/776314460 >> 172.21.15.31:0/776314460 conn(0x5639a8df0300 msgr2=0x5639a91ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.949+0000 7ff084c19740 1 -- 172.21.15.31:0/776314460 shutdown_connections 2024-09-21T06:16:27.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.949+0000 7ff084c19740 1 -- 172.21.15.31:0/776314460 wait complete. 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.949+0000 7ff084c19740 1 -- start start 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.949+0000 7ff084c19740 1 Processor -- start 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff084c19740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dd32f0 0x5639a91c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff084c19740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dd32f0 0x5639a91c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff076ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff0777fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dd32f0 0x5639a91c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42072/0 (socket says 172.21.15.31:42072) 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff076ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43730/0 (socket says 172.21.15.31:43730) 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff0777fe640 1 -- 172.21.15.31:0/347361047 learned_addr learned my addr 172.21.15.31:0/347361047 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.950+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dfc1b0 0x5639a9230010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ac150 con 0x5639a8ddb4d0 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ad5c0 con 0x5639a8dfc1b0 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781ac820 con 0x5639a8dd32f0 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff077fff640 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dfc1b0 0x5639a9230010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff084c19740 10 client.0 _ll_register_callbacks cb 0x7ff07000fb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff077fff640 1 -- 172.21.15.31:0/347361047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dd32f0 msgr2=0x5639a91c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff077fff640 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dd32f0 0x5639a91c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff077fff640 1 -- 172.21.15.31:0/347361047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 msgr2=0x5639a91c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff077fff640 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.951+0000 7ff077fff640 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a91c8a40 con 0x5639a8dfc1b0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.952+0000 7ff077fff640 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dfc1b0 0x5639a9230010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5639a91afd00 tx=0x7ff06c013940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d818afdb1944278 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.952+0000 7ff056ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.953+0000 7ff056ffd640 1 -- 172.21.15.31:0/347361047 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c021060 con 0x5639a8dfc1b0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.953+0000 7ff056ffd640 1 -- 172.21.15.31:0/347361047 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff06c011710 con 0x5639a8dfc1b0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.953+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07001e4f0 con 0x5639a8dfc1b0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.953+0000 7ff056ffd640 1 -- 172.21.15.31:0/347361047 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff06c011910 con 0x5639a8dfc1b0 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.953+0000 7ff084c19740 10 client.26300 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.953+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5639a9231650 con 0x5639a8dfc1b0 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff056ffd640 1 -- 172.21.15.31:0/347361047 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07001e4f0 con 0x5639a8dfc1b0 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff056ffd640 1 -- 172.21.15.31:0/347361047 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff06c026c50 con 0x5639a8dfc1b0 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff084c19740 10 client.26300 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff084c19740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff084c19740 20 client.26300 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8d901c0 0x5639a9233e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff084c19740 4 client.26300 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639a91140f0 con 0x5639a8d901c0 2024-09-21T06:16:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.954+0000 7ff0777fe640 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8d901c0 0x5639a9233e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.955+0000 7ff0777fe640 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8d901c0 0x5639a9233e20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999c3a5b59081a0f server_cookie=8b1a91cf4e4d6d10 in_seq=0 out_seq=0 2024-09-21T06:16:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.955+0000 7ff056ffd640 10 client.26300 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff056ffd640 1 -- 172.21.15.31:0/347361047 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639a91140f0 con 0x5639a8d901c0 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff056ffd640 10 client.26300 handle_command_reply: tid=0 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 client.26300 shutdown 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dfc1b0 msgr2=0x5639a9230010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dfc1b0 0x5639a9230010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5639a91afd00 tx=0x7ff06c013940 comp rx=0 tx=0).stop 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 shutdown_connections 2024-09-21T06:16:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5639a8d901c0 0x5639a9233e20 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a8dfc1b0 0x5639a9230010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5639a8ddb4d0 0x5639a91c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 --2- 172.21.15.31:0/347361047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5639a8dd32f0 0x5639a91c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 >> 172.21.15.31:0/347361047 conn(0x5639a8df0300 msgr2=0x5639a91a29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff056ffd640 0 client.26300 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 shutdown_connections 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.968+0000 7ff084c19740 1 -- 172.21.15.31:0/347361047 wait complete. 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:16:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:16:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:16:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.969+0000 7ff084c19740 20 client.26300 trim_cache size 0 max 16384 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.970+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff0600b67d0 msgr2=0x7ff0600b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.970+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff0600b67d0 0x7ff0600b8ce0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff0700099a0 tx=0x7ff070009500 comp rx=0 tx=0).stop 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.970+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 msgr2=0x7ff07819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.970+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff07819aa20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff068006010 tx=0x7ff068004060 comp rx=0 tx=0).stop 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 shutdown_connections 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff0600b67d0 0x7ff0600b8ce0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078108c50 0x7ff07819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff078108300 0x7ff07819aa20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 --2- 172.21.15.31:0/32274033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0781029d0 0x7ff07819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 >> 172.21.15.31:0/32274033 conn(0x7ff0780fe410 msgr2=0x7ff078100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 shutdown_connections 2024-09-21T06:16:27.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:27.971+0000 7ff07dbc6640 1 -- 172.21.15.31:0/32274033 wait complete. 2024-09-21T06:16:29.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:28 smithi110 ceph-mon[34550]: pgmap v466: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-09-21T06:16:29.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:28 smithi107 ceph-mon[34380]: pgmap v466: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-09-21T06:16:29.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:28 smithi031 ceph-mon[31237]: pgmap v466: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 761 op/s 2024-09-21T06:16:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 290.0 s, 3533.8 tps, lat 28.231 ms stddev 33.665 2024-09-21T06:16:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:30 smithi110 ceph-mon[34550]: pgmap v467: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-09-21T06:16:31.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:30 smithi107 ceph-mon[34380]: pgmap v467: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-09-21T06:16:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:30 smithi031 ceph-mon[31237]: pgmap v467: 97 pgs: 97 active+clean; 3.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.0 MiB/s wr, 630 op/s 2024-09-21T06:16:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:32 smithi110 ceph-mon[34550]: pgmap v468: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 851 op/s 2024-09-21T06:16:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:32 smithi107 ceph-mon[34380]: pgmap v468: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 851 op/s 2024-09-21T06:16:33.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:32 smithi031 ceph-mon[31237]: pgmap v468: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 851 op/s 2024-09-21T06:16:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 295.0 s, 4299.8 tps, lat 23.364 ms stddev 29.156 2024-09-21T06:16:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:34 smithi110 ceph-mon[34550]: pgmap v469: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-21T06:16:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:34 smithi107 ceph-mon[34380]: pgmap v469: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-21T06:16:35.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:34 smithi031 ceph-mon[31237]: pgmap v469: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 670 op/s 2024-09-21T06:16:37.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:36 smithi110 ceph-mon[34550]: pgmap v470: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.6 MiB/s wr, 785 op/s 2024-09-21T06:16:37.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:36 smithi107 ceph-mon[34380]: pgmap v470: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.6 MiB/s wr, 785 op/s 2024-09-21T06:16:37.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:36 smithi031 ceph-mon[31237]: pgmap v470: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.6 MiB/s wr, 785 op/s 2024-09-21T06:16:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:38 smithi110 ceph-mon[34550]: pgmap v471: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.7 MiB/s wr, 676 op/s 2024-09-21T06:16:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:38 smithi107 ceph-mon[34380]: pgmap v471: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.7 MiB/s wr, 676 op/s 2024-09-21T06:16:39.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:38 smithi031 ceph-mon[31237]: pgmap v471: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.7 MiB/s wr, 676 op/s 2024-09-21T06:16:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 300.0 s, 3967.0 tps, lat 25.086 ms stddev 28.158 2024-09-21T06:16:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 -- 172.21.15.31:0/2218478731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 msgr2=0x7f8d6410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 --2- 172.21.15.31:0/2218478731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d6410a4f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009900 tx=0x7f8d5402a440 comp rx=0 tx=0).stop 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 -- 172.21.15.31:0/2218478731 shutdown_connections 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 --2- 172.21.15.31:0/2218478731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 0x7f8d64116b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 --2- 172.21.15.31:0/2218478731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 0x7f8d6410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 --2- 172.21.15.31:0/2218478731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d6410a4f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.134+0000 7f8d69713640 1 -- 172.21.15.31:0/2218478731 >> 172.21.15.31:0/2218478731 conn(0x7f8d64079630 msgr2=0x7f8d641081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.135+0000 7f8d69713640 1 -- 172.21.15.31:0/2218478731 shutdown_connections 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.135+0000 7f8d69713640 1 -- 172.21.15.31:0/2218478731 wait complete. 2024-09-21T06:16:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.136+0000 7f8d69713640 1 Processor -- start 2024-09-21T06:16:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.136+0000 7f8d69713640 1 -- start start 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d69713640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d641a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d69713640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 0x7f8d641a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d69713640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 0x7f8d641a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 0x7f8d641a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d641a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 0x7f8d641a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37450/0 (socket says 172.21.15.31:37450) 2024-09-21T06:16:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d641a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47864/0 (socket says 172.21.15.31:47864) 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d627fc640 1 -- 172.21.15.31:0/1677219814 learned_addr learned my addr 172.21.15.31:0/1677219814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64119fd0 con 0x7f8d6410fa10 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64119e50 con 0x7f8d64110360 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.137+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6411a150 con 0x7f8d6410a0e0 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d637fe640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 0x7f8d641a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d62ffd640 1 -- 172.21.15.31:0/1677219814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 msgr2=0x7f8d641a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 0x7f8d641a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d62ffd640 1 -- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 msgr2=0x7f8d641a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 0x7f8d641a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d62ffd640 1 -- 172.21.15.31:0/1677219814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d641a68f0 con 0x7f8d6410a0e0 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d637fe640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 0x7f8d641a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d641a0a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d54000c00 tx=0x7f8d54003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.138+0000 7f8d4bfff640 1 -- 172.21.15.31:0/1677219814 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54040030 con 0x7f8d6410a0e0 2024-09-21T06:16:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.139+0000 7f8d4bfff640 1 -- 172.21.15.31:0/1677219814 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5403c8f0 con 0x7f8d6410a0e0 2024-09-21T06:16:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.139+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d641a6b80 con 0x7f8d6410a0e0 2024-09-21T06:16:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.139+0000 7f8d4bfff640 1 -- 172.21.15.31:0/1677219814 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54031970 con 0x7f8d6410a0e0 2024-09-21T06:16:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.139+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d641a7070 con 0x7f8d6410a0e0 2024-09-21T06:16:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.143+0000 7f8d4bfff640 1 -- 172.21.15.31:0/1677219814 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8d54043030 con 0x7f8d6410a0e0 2024-09-21T06:16:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.144+0000 7f8d4bfff640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8d4c0b6700 0x7f8d4c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.144+0000 7f8d4bfff640 1 -- 172.21.15.31:0/1677219814 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8d54107e40 con 0x7f8d6410a0e0 2024-09-21T06:16:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.144+0000 7f8d627fc640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8d4c0b6700 0x7f8d4c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.144+0000 7f8d70680740 1 Processor -- start 2024-09-21T06:16:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.144+0000 7f8d70680740 1 -- start start 2024-09-21T06:16:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d70680740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3cb31b0 0x5557f40589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d70680740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c8a2f0 0x5557f4058ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3cb31b0 0x5557f40589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3cb31b0 0x5557f40589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54158/0 (socket says 172.21.15.31:54158) 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c8a2f0 0x5557f4058ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d627fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c8a2f0 0x5557f4058ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37452/0 (socket says 172.21.15.31:37452) 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d62ffd640 1 -- 172.21.15.31:0/249467746 learned_addr learned my addr 172.21.15.31:0/249467746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d70680740 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f4059430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5557f3fcd6f0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5557f3fbcb20 con 0x5557f3cb31b0 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5557f3fc0930 con 0x5557f3c924d0 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d627fc640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8d4c0b6700 0x7f8d4c0b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d64079ca0 tx=0x7f8d58005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.145+0000 7f8d637fe640 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f4059430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.146+0000 7f8d637fe640 1 -- 172.21.15.31:0/249467746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3cb31b0 msgr2=0x5557f40589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.146+0000 7f8d637fe640 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3cb31b0 0x5557f40589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.146+0000 7f8d637fe640 1 -- 172.21.15.31:0/249467746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c8a2f0 msgr2=0x5557f4058ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.146+0000 7f8d637fe640 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c8a2f0 0x5557f4058ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.146+0000 7f8d637fe640 1 -- 172.21.15.31:0/249467746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f3f82530 con 0x5557f3c924d0 2024-09-21T06:16:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.146+0000 7f8d637fe640 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f4059430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c007e30 tx=0x7f8d5c00d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e316eb74628c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.147+0000 7f8d497fa640 1 -- 172.21.15.31:0/249467746 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c004d20 con 0x5557f3c924d0 2024-09-21T06:16:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.148+0000 7f8d497fa640 1 -- 172.21.15.31:0/249467746 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5c00fae0 con 0x5557f3c924d0 2024-09-21T06:16:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.148+0000 7f8d497fa640 1 -- 172.21.15.31:0/249467746 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c005580 con 0x5557f3c924d0 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 msgr2=0x5557f4059430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f4059430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c007e30 tx=0x7f8d5c00d710 comp rx=0 tx=0).stop 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 shutdown_connections 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f4059430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c8a2f0 0x5557f4058ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 --2- 172.21.15.31:0/249467746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3cb31b0 0x5557f40589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 >> 172.21.15.31:0/249467746 conn(0x5557f3ca7300 msgr2=0x5557f40568a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 shutdown_connections 2024-09-21T06:16:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.149+0000 7f8d70680740 1 -- 172.21.15.31:0/249467746 wait complete. 2024-09-21T06:16:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.151+0000 7f8d70680740 1 -- start start 2024-09-21T06:16:41.153 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.151+0000 7f8d70680740 1 Processor -- start 2024-09-21T06:16:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d70680740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f4064070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d70680740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f40645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f4064070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f4064070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54160/0 (socket says 172.21.15.31:54160) 2024-09-21T06:16:41.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d62ffd640 1 -- 172.21.15.31:0/2228111987 learned_addr learned my addr 172.21.15.31:0/2228111987 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d627fc640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f40645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3cb31b0 0x5557f4070420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5557f3ef4ab0 con 0x5557f3cb31b0 2024-09-21T06:16:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5557f3f62730 con 0x5557f3c8a2f0 2024-09-21T06:16:41.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.152+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5557f4067130 con 0x5557f3c924d0 2024-09-21T06:16:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d637fe640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3cb31b0 0x5557f4070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d62ffd640 1 -- 172.21.15.31:0/2228111987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 msgr2=0x5557f40645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f40645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d62ffd640 1 -- 172.21.15.31:0/2228111987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3cb31b0 msgr2=0x5557f4070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3cb31b0 0x5557f4070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d62ffd640 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f3d1e790 con 0x5557f3c8a2f0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d70680740 10 client.0 _ll_register_callbacks cb 0x5557f3fb66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d5c00fae0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f4064070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d54105680 tx=0x7f8d5403cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87715d8c29e2cba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.153+0000 7f8d46ff5640 10 client.26306 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d46ff5640 1 -- 172.21.15.31:0/2228111987 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5404a040 con 0x5557f3c8a2f0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d46ff5640 1 -- 172.21.15.31:0/2228111987 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8d540c9f40 con 0x5557f3c8a2f0 2024-09-21T06:16:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d46ff5640 1 -- 172.21.15.31:0/2228111987 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8d540ce3d0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 10 client.26306 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5557f3d97620 con 0x5557f3c8a2f0 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d46ff5640 1 -- 172.21.15.31:0/2228111987 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8d540efce0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 10 client.26306 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 10 client.26306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 20 client.26306 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3c71d80 0x5557f4075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 4 client.26306 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.154+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f3fcf640 con 0x5557f3c71d80 2024-09-21T06:16:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.155+0000 7f8d627fc640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3c71d80 0x5557f4075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.155+0000 7f8d46ff5640 10 client.26306 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.155+0000 7f8d627fc640 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3c71d80 0x5557f4075fc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c101f2dcc9da5fb4 server_cookie=5a14129e4f55e050 in_seq=0 out_seq=0 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.156+0000 7f8d46ff5640 1 -- 172.21.15.31:0/2228111987 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5557f3fcf640 con 0x5557f3c71d80 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.156+0000 7f8d46ff5640 10 client.26306 handle_command_reply: tid=0 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.157+0000 7f8d70680740 1 client.26306 shutdown 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.157+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 msgr2=0x5557f4064070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.157+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f4064070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d54105680 tx=0x7f8d5403cf90 comp rx=0 tx=0).stop 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.157+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 shutdown_connections 2024-09-21T06:16:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.157+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3c71d80 0x5557f4075fc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3cb31b0 0x5557f4070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d46ff5640 0 client.26306 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f40645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 1 --2- 172.21.15.31:0/2228111987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f4064070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 >> 172.21.15.31:0/2228111987 conn(0x5557f3ca7300 msgr2=0x5557f40599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 shutdown_connections 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 1 -- 172.21.15.31:0/2228111987 wait complete. 2024-09-21T06:16:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.158+0000 7f8d70680740 20 client.26306 trim_cache size 0 max 16384 2024-09-21T06:16:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.186+0000 7f8d70680740 1 Processor -- start 2024-09-21T06:16:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.186+0000 7f8d70680740 1 -- start start 2024-09-21T06:16:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.186+0000 7f8d70680740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c71d80 0x5557f4055860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.186+0000 7f8d70680740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c8a2f0 0x5557f4065ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c71d80 0x5557f4055860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c71d80 0x5557f4055860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54172/0 (socket says 172.21.15.31:54172) 2024-09-21T06:16:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d62ffd640 1 -- 172.21.15.31:0/3199043806 learned_addr learned my addr 172.21.15.31:0/3199043806 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d70680740 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c924d0 0x5557f4061240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64073530 con 0x5557f3c924d0 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641adc20 con 0x5557f3c71d80 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64073ba0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c8a2f0 0x5557f4065ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 -- 172.21.15.31:0/3199043806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c71d80 msgr2=0x5557f4055860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c71d80 0x5557f4055860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 -- 172.21.15.31:0/3199043806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c924d0 msgr2=0x5557f4061240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c924d0 0x5557f4061240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 -- 172.21.15.31:0/3199043806 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f4062550 con 0x5557f3c8a2f0 2024-09-21T06:16:41.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.187+0000 7f8d627fc640 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c8a2f0 0x5557f4065ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5557f4076730 tx=0x7f8d58014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e58a429e1dcf23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.188+0000 7f8d48ff9640 1 -- 172.21.15.31:0/3199043806 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5803a660 con 0x5557f3c8a2f0 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.188+0000 7f8d48ff9640 1 -- 172.21.15.31:0/3199043806 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d580044d0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.188+0000 7f8d48ff9640 1 -- 172.21.15.31:0/3199043806 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5803b9c0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.189+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c8a2f0 msgr2=0x5557f4065ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.189+0000 7f8d70680740 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c8a2f0 0x5557f4065ca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5557f4076730 tx=0x7f8d58014000 comp rx=0 tx=0).stop 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.189+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 shutdown_connections 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c924d0 0x5557f4061240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c8a2f0 0x5557f4065ca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 --2- 172.21.15.31:0/3199043806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c71d80 0x5557f4055860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 >> 172.21.15.31:0/3199043806 conn(0x5557f3ca7300 msgr2=0x5557f4065590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 shutdown_connections 2024-09-21T06:16:41.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 -- 172.21.15.31:0/3199043806 wait complete. 2024-09-21T06:16:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 -- start start 2024-09-21T06:16:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.190+0000 7f8d70680740 1 Processor -- start 2024-09-21T06:16:41.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.191+0000 7f8d70680740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c71d80 0x5557f406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.191+0000 7f8d70680740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f406a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.191+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c71d80 0x5557f406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.191+0000 7f8d62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c71d80 0x5557f406cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37480/0 (socket says 172.21.15.31:37480) 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d62ffd640 1 -- 172.21.15.31:0/938443542 learned_addr learned my addr 172.21.15.31:0/938443542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d627fc640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f406a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f406a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64074270 con 0x5557f3c71d80 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8d641ad550 con 0x5557f3c8a2f0 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8d64074940 con 0x5557f3c924d0 2024-09-21T06:16:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.192+0000 7f8d637fe640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f406a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d70680740 10 client.0 _ll_register_callbacks cb 0x7f8d540ca410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d627fc640 1 -- 172.21.15.31:0/938443542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 msgr2=0x5557f406a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d627fc640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f406a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d627fc640 1 -- 172.21.15.31:0/938443542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c71d80 msgr2=0x5557f406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d627fc640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c71d80 0x5557f406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d627fc640 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f406adf0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.193+0000 7f8d627fc640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f406a0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d58040000 tx=0x7f8d5803b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770cd4f57ab0f1d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d580044d0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d46ff5640 1 -- 172.21.15.31:0/938443542 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5803b3b0 con 0x5557f3c8a2f0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d46ff5640 1 -- 172.21.15.31:0/938443542 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8d5803bc20 con 0x5557f3c8a2f0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d46ff5640 1 -- 172.21.15.31:0/938443542 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8d5803a460 con 0x5557f3c8a2f0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d70680740 10 client.26312 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.194+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5557f40e0900 con 0x5557f3c8a2f0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d46ff5640 1 -- 172.21.15.31:0/938443542 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8d5803a600 con 0x5557f3c8a2f0 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d70680740 10 client.26312 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d70680740 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d70680740 20 client.26312 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3cb31b0 0x5557f40e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d70680740 4 client.26312 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557f3fcb0f0 con 0x5557f3cb31b0 2024-09-21T06:16:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.195+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3cb31b0 0x5557f40e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.196+0000 7f8d62ffd640 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3cb31b0 0x5557f40e2de0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2998b217297c4b2 server_cookie=fa3857b2292d83e6 in_seq=0 out_seq=0 2024-09-21T06:16:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.196+0000 7f8d46ff5640 10 client.26312 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.217+0000 7f8d46ff5640 1 -- 172.21.15.31:0/938443542 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557f3fcb0f0 con 0x5557f3cb31b0 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.217+0000 7f8d46ff5640 10 client.26312 handle_command_reply: tid=0 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 client.26312 shutdown 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 msgr2=0x5557f406a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f406a0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d58040000 tx=0x7f8d5803b870 comp rx=0 tx=0).stop 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 shutdown_connections 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5557f3cb31b0 0x5557f40e2de0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5557f3c924d0 0x5557f406a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d46ff5640 0 client.26312 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5557f3c8a2f0 0x5557f406a0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 --2- 172.21.15.31:0/938443542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5557f3c71d80 0x5557f406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 >> 172.21.15.31:0/938443542 conn(0x5557f3ca7300 msgr2=0x5557f4055630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 shutdown_connections 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 1 -- 172.21.15.31:0/938443542 wait complete. 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.218+0000 7f8d70680740 20 client.26312 trim_cache size 0 max 16384 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:16:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8d4c0b6700 msgr2=0x7f8d4c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8d4c0b6700 0x7f8d4c0b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d64079ca0 tx=0x7f8d58005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 msgr2=0x7f8d641a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d641a0a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d54000c00 tx=0x7f8d54003ee0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.222 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 shutdown_connections 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d64110360 0x7f8d641a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8d4c0b6700 0x7f8d4c0b8c10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d6410fa10 0x7f8d641a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 --2- 172.21.15.31:0/1677219814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d6410a0e0 0x7f8d641a0a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 >> 172.21.15.31:0/1677219814 conn(0x7f8d64079630 msgr2=0x7f8d6407a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.219+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 shutdown_connections 2024-09-21T06:16:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:41.220+0000 7f8d69713640 1 -- 172.21.15.31:0/1677219814 wait complete. 2024-09-21T06:16:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:40 smithi110 ceph-mon[34550]: pgmap v472: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-09-21T06:16:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:40 smithi107 ceph-mon[34380]: pgmap v472: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-09-21T06:16:41.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:40 smithi031 ceph-mon[31237]: pgmap v472: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.3 MiB/s wr, 722 op/s 2024-09-21T06:16:41.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:42.429 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 -- 172.21.15.31:0/1771469659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b84111f40 msgr2=0x7f1b8410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/1771469659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b84111f40 0x7f1b8410a3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009860 tx=0x7f1b7402a340 comp rx=0 tx=0).stop 2024-09-21T06:16:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 -- 172.21.15.31:0/1771469659 shutdown_connections 2024-09-21T06:16:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/1771469659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8411cc30 0x7f1b8411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/1771469659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b8410a9e0 0x7f1b8410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/1771469659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b84111f40 0x7f1b8410a3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 -- 172.21.15.31:0/1771469659 >> 172.21.15.31:0/1771469659 conn(0x7f1b84070ec0 msgr2=0x7f1b840712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 -- 172.21.15.31:0/1771469659 shutdown_connections 2024-09-21T06:16:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.076+0000 7f1b8ae84640 1 -- 172.21.15.31:0/1771469659 wait complete. 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.077+0000 7f1b8ae84640 1 Processor -- start 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.077+0000 7f1b8ae84640 1 -- start start 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.077+0000 7f1b8ae84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8410a9e0 0x7f1b841a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.077+0000 7f1b8ae84640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b84111f40 0x7f1b841a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b8ae84640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b8411cc30 0x7f1b841aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b88c20640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8410a9e0 0x7f1b841a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b88c20640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8410a9e0 0x7f1b841a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47908/0 (socket says 172.21.15.31:47908) 2024-09-21T06:16:43.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b8ae84640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841228b0 con 0x7f1b84111f40 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b88c20640 1 -- 172.21.15.31:0/2010495664 learned_addr learned my addr 172.21.15.31:0/2010495664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84122730 con 0x7f1b8411cc30 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b84122a30 con 0x7f1b8410a9e0 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b89421640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b8411cc30 0x7f1b841aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b83fff640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b84111f40 0x7f1b841a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b89421640 1 -- 172.21.15.31:0/2010495664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8410a9e0 msgr2=0x7f1b841a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b89421640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8410a9e0 0x7f1b841a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b89421640 1 -- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b84111f40 msgr2=0x7f1b841a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b89421640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b84111f40 0x7f1b841a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.078+0000 7f1b89421640 1 -- 172.21.15.31:0/2010495664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b841af2d0 con 0x7f1b8411cc30 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.079+0000 7f1b89421640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b8411cc30 0x7f1b841aeba0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00efc0 tx=0x7f1b7c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.079+0000 7f1b81ffb640 1 -- 172.21.15.31:0/2010495664 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c003970 con 0x7f1b8411cc30 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.079+0000 7f1b81ffb640 1 -- 172.21.15.31:0/2010495664 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b7c004020 con 0x7f1b8411cc30 2024-09-21T06:16:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.079+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b841af5c0 con 0x7f1b8411cc30 2024-09-21T06:16:43.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.080+0000 7f1b81ffb640 1 -- 172.21.15.31:0/2010495664 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c00b940 con 0x7f1b8411cc30 2024-09-21T06:16:43.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.080+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b841b5d20 con 0x7f1b8411cc30 2024-09-21T06:16:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.083+0000 7f1b81ffb640 1 -- 172.21.15.31:0/2010495664 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1b7c01d020 con 0x7f1b8411cc30 2024-09-21T06:16:43.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 Processor -- start 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 -- start start 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b81ffb640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b6c0b6780 0x7f1b6c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f101b0 0x55d6882e0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d688256e50 con 0x55d687f101b0 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d688246280 con 0x55d687f39070 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b913f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d68824a090 con 0x55d687f18390 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b83fff640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b6c0b6780 0x7f1b6c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.086+0000 7f1b81ffb640 1 -- 172.21.15.31:0/2010495664 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1b7c0e7600 con 0x7f1b8411cc30 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54226/0 (socket says 172.21.15.31:54226) 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 -- 172.21.15.31:0/534064247 learned_addr learned my addr 172.21.15.31:0/534064247 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:43.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b89421640 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b83fff640 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f101b0 0x55d6882e0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 -- 172.21.15.31:0/534064247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 msgr2=0x55d6882e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 -- 172.21.15.31:0/534064247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f101b0 msgr2=0x55d6882e0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f101b0 0x55d6882e0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 -- 172.21.15.31:0/534064247 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d687fa4650 con 0x55d687f39070 2024-09-21T06:16:43.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.087+0000 7f1b88c20640 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882e0650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1b74035040 tx=0x7f1b74009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f62acbce0ead2230 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:43.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.088+0000 7f1b6affd640 1 -- 172.21.15.31:0/534064247 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74041020 con 0x55d687f39070 2024-09-21T06:16:43.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.088+0000 7f1b6affd640 1 -- 172.21.15.31:0/534064247 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b74030450 con 0x55d687f39070 2024-09-21T06:16:43.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.088+0000 7f1b6affd640 1 -- 172.21.15.31:0/534064247 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74037040 con 0x55d687f39070 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.089+0000 7f1b913f8740 1 -- 172.21.15.31:0/534064247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 msgr2=0x55d6882e0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.089+0000 7f1b913f8740 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882e0650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1b74035040 tx=0x7f1b74009490 comp rx=0 tx=0).stop 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.089+0000 7f1b83fff640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b6c0b6780 0x7f1b6c0b8c90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d6882df2d0 tx=0x7f1b780097c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.089+0000 7f1b913f8740 1 -- 172.21.15.31:0/534064247 shutdown_connections 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f101b0 0x55d6882e0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 --2- 172.21.15.31:0/534064247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882e0650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 -- 172.21.15.31:0/534064247 >> 172.21.15.31:0/534064247 conn(0x55d687f2d460 msgr2=0x55d6882de540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 -- 172.21.15.31:0/534064247 shutdown_connections 2024-09-21T06:16:43.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 -- 172.21.15.31:0/534064247 wait complete. 2024-09-21T06:16:43.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.090+0000 7f1b913f8740 1 -- start start 2024-09-21T06:16:43.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.091+0000 7f1b913f8740 1 Processor -- start 2024-09-21T06:16:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882ebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f18390 0x55d6882ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d6880a7700 con 0x55d687f18390 2024-09-21T06:16:43.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d6881e1110 con 0x55d687f39070 2024-09-21T06:16:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d6882eedd0 con 0x55d687f101b0 2024-09-21T06:16:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b913f8740 10 client.0 _ll_register_callbacks cb 0x55d6881fa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f18390 0x55d6882ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b83fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f18390 0x55d6882ec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37514/0 (socket says 172.21.15.31:37514) 2024-09-21T06:16:43.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b83fff640 1 -- 172.21.15.31:0/3589900075 learned_addr learned my addr 172.21.15.31:0/3589900075 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.092+0000 7f1b89421640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b88c20640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882ebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b89421640 1 -- 172.21.15.31:0/3589900075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 msgr2=0x55d6882ebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b89421640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882ebdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b89421640 1 -- 172.21.15.31:0/3589900075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f18390 msgr2=0x55d6882ec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b89421640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f18390 0x55d6882ec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b89421640 1 -- 172.21.15.31:0/3589900075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68820ec40 con 0x55d687f39070 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b89421640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882f8120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0e4d30 tx=0x7f1b7c004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f33421f1fa74b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:43.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.093+0000 7f1b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.094+0000 7f1b63fff640 1 -- 172.21.15.31:0/3589900075 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0e51c0 con 0x55d687f39070 2024-09-21T06:16:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.094+0000 7f1b63fff640 1 -- 172.21.15.31:0/3589900075 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1b7c094d70 con 0x55d687f39070 2024-09-21T06:16:43.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.094+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b74030450 con 0x55d687f39070 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.094+0000 7f1b63fff640 1 -- 172.21.15.31:0/3589900075 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1b7c0b1f10 con 0x55d687f39070 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.094+0000 7f1b913f8740 10 client.26324 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.094+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d6882fabc0 con 0x55d687f39070 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b63fff640 1 -- 172.21.15.31:0/3589900075 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b7c098020 con 0x55d687f39070 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b63fff640 1 -- 172.21.15.31:0/3589900075 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1b7c00b040 con 0x55d687f39070 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b913f8740 10 client.26324 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b913f8740 10 client.26324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b913f8740 20 client.26324 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b913f8740 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687ef79f0 0x55d6882fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b913f8740 4 client.26324 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:43.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b74030450 con 0x55d687ef79f0 2024-09-21T06:16:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.095+0000 7f1b88c20640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687ef79f0 0x55d6882fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.096+0000 7f1b63fff640 10 client.26324 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.096+0000 7f1b88c20640 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687ef79f0 0x55d6882fe1f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e96ab6ba945e02f9 server_cookie=264b4b33993fc1f7 in_seq=0 out_seq=0 2024-09-21T06:16:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b63fff640 1 -- 172.21.15.31:0/3589900075 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1b74030450 con 0x55d687ef79f0 2024-09-21T06:16:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b63fff640 10 client.26324 handle_command_reply: tid=0 2024-09-21T06:16:43.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 client.26324 shutdown 2024-09-21T06:16:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 msgr2=0x55d6882f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882f8120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0e4d30 tx=0x7f1b7c004450 comp rx=0 tx=0).stop 2024-09-21T06:16:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 shutdown_connections 2024-09-21T06:16:43.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687ef79f0 0x55d6882fe1f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f39070 0x55d6882f8120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f18390 0x55d6882ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 --2- 172.21.15.31:0/3589900075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882ebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 >> 172.21.15.31:0/3589900075 conn(0x55d687f2d460 msgr2=0x55d6882e37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.097+0000 7f1b63fff640 0 client.26324 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.098+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 shutdown_connections 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.098+0000 7f1b913f8740 1 -- 172.21.15.31:0/3589900075 wait complete. 2024-09-21T06:16:43.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.099+0000 7f1b913f8740 20 client.26324 trim_cache size 0 max 16384 2024-09-21T06:16:43.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.139+0000 7f1b913f8740 1 Processor -- start 2024-09-21T06:16:43.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:42 smithi031 ceph-mon[31237]: pgmap v473: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 954 op/s 2024-09-21T06:16:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 -- start start 2024-09-21T06:16:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f101b0 0x55d6882edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d6882de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841bc660 con 0x55d687f39070 2024-09-21T06:16:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841beee0 con 0x55d687f101b0 2024-09-21T06:16:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.144+0000 7f1b913f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841bccd0 con 0x55d687f18390 2024-09-21T06:16:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.150+0000 7f1b89421640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d6882de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.150+0000 7f1b89421640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d6882de0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37524/0 (socket says 172.21.15.31:37524) 2024-09-21T06:16:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.150+0000 7f1b89421640 1 -- 172.21.15.31:0/322728664 learned_addr learned my addr 172.21.15.31:0/322728664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.150+0000 7f1b88c20640 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f101b0 0x55d6882edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.151+0000 7f1b83fff640 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.154+0000 7f1b89421640 1 -- 172.21.15.31:0/322728664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 msgr2=0x55d6882ddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.154+0000 7f1b89421640 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882ddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.154+0000 7f1b89421640 1 -- 172.21.15.31:0/322728664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f101b0 msgr2=0x55d6882edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.154+0000 7f1b89421640 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f101b0 0x55d6882edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.155+0000 7f1b89421640 1 -- 172.21.15.31:0/322728664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6882e2b70 con 0x55d687f39070 2024-09-21T06:16:43.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.155+0000 7f1b89421640 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d6882de0b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0ac8b0 tx=0x7f1b7c0a9210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86b826e72d59d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.158+0000 7f1b6a7fc640 1 -- 172.21.15.31:0/322728664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0ac690 con 0x55d687f39070 2024-09-21T06:16:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.158+0000 7f1b6a7fc640 1 -- 172.21.15.31:0/322728664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b7c0b6e50 con 0x55d687f39070 2024-09-21T06:16:43.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.158+0000 7f1b6a7fc640 1 -- 172.21.15.31:0/322728664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c0a8f80 con 0x55d687f39070 2024-09-21T06:16:43.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.161+0000 7f1b913f8740 1 -- 172.21.15.31:0/322728664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 msgr2=0x55d6882de0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.161+0000 7f1b913f8740 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d6882de0b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0ac8b0 tx=0x7f1b7c0a9210 comp rx=0 tx=0).stop 2024-09-21T06:16:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.165+0000 7f1b913f8740 1 -- 172.21.15.31:0/322728664 shutdown_connections 2024-09-21T06:16:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.165+0000 7f1b913f8740 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d6882de0b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.165+0000 7f1b913f8740 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f18390 0x55d6882ddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.165+0000 7f1b913f8740 1 --2- 172.21.15.31:0/322728664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f101b0 0x55d6882edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.165+0000 7f1b913f8740 1 -- 172.21.15.31:0/322728664 >> 172.21.15.31:0/322728664 conn(0x55d687f2d460 msgr2=0x55d6882ec210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.167+0000 7f1b913f8740 1 -- 172.21.15.31:0/322728664 shutdown_connections 2024-09-21T06:16:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.167+0000 7f1b913f8740 1 -- 172.21.15.31:0/322728664 wait complete. 2024-09-21T06:16:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.168+0000 7f1b913f8740 1 -- start start 2024-09-21T06:16:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.168+0000 7f1b913f8740 1 Processor -- start 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882f0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 0x55d6882f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d688367bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841bd3a0 con 0x55d687f39070 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841be810 con 0x55d687f18390 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841bda70 con 0x55d687f101b0 2024-09-21T06:16:43.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.170+0000 7f1b913f8740 10 client.0 _ll_register_callbacks cb 0x7f1b7c005b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 0x55d6882f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 0x55d6882f08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54246/0 (socket says 172.21.15.31:54246) 2024-09-21T06:16:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b83fff640 1 -- 172.21.15.31:0/50744026 learned_addr learned my addr 172.21.15.31:0/50744026 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:43.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b88c20640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b83fff640 1 -- 172.21.15.31:0/50744026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 msgr2=0x55d6882f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b83fff640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b83fff640 1 -- 172.21.15.31:0/50744026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 msgr2=0x55d688367bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.172+0000 7f1b89421640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d688367bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.173+0000 7f1b83fff640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d688367bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.173+0000 7f1b83fff640 1 -- 172.21.15.31:0/50744026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6882f1150 con 0x55d687f18390 2024-09-21T06:16:43.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.173+0000 7f1b89421640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d688367bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:43.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.176+0000 7f1b83fff640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 0x55d6882f08b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1b78020b90 tx=0x7f1b78005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8338edf6903c3ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.177+0000 7f1b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.177+0000 7f1b63fff640 1 -- 172.21.15.31:0/50744026 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78013ca0 con 0x55d687f18390 2024-09-21T06:16:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.177+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b7c0b6e50 con 0x55d687f18390 2024-09-21T06:16:43.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.177+0000 7f1b63fff640 1 -- 172.21.15.31:0/50744026 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1b78004020 con 0x55d687f18390 2024-09-21T06:16:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.179+0000 7f1b63fff640 1 -- 172.21.15.31:0/50744026 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1b78013e40 con 0x55d687f18390 2024-09-21T06:16:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.180+0000 7f1b913f8740 10 client.26330 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.180+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d6883691b0 con 0x55d687f18390 2024-09-21T06:16:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.182+0000 7f1b63fff640 1 -- 172.21.15.31:0/50744026 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b7c0b6e50 con 0x55d687f18390 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.182+0000 7f1b63fff640 1 -- 172.21.15.31:0/50744026 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1b78011430 con 0x55d687f18390 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b913f8740 10 client.26330 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b913f8740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b913f8740 20 client.26330 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b913f8740 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687eccda0 0x55d68836b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b913f8740 4 client.26330 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d688258da0 con 0x55d687eccda0 2024-09-21T06:16:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.183+0000 7f1b88c20640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687eccda0 0x55d68836b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.185+0000 7f1b88c20640 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687eccda0 0x55d68836b930 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61dce392d24f6c79 server_cookie=1bd0f142f5db648d in_seq=0 out_seq=0 2024-09-21T06:16:43.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.185+0000 7f1b63fff640 10 client.26330 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:43.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.185+0000 7f1b63fff640 1 -- 172.21.15.31:0/50744026 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55d688258da0 con 0x55d687eccda0 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.185+0000 7f1b63fff640 10 client.26330 handle_command_reply: tid=0 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.185+0000 7f1b913f8740 1 client.26330 shutdown 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.186+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 msgr2=0x55d6882f08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.186+0000 7f1b913f8740 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 0x55d6882f08b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1b78020b90 tx=0x7f1b78005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 shutdown_connections 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d687eccda0 0x55d68836b930 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d687f39070 0x55d688367bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d687f18390 0x55d6882f08b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b63fff640 0 client.26330 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 --2- 172.21.15.31:0/50744026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d687f101b0 0x55d6882f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 >> 172.21.15.31:0/50744026 conn(0x55d687f2d460 msgr2=0x55d6882ec210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 shutdown_connections 2024-09-21T06:16:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 1 -- 172.21.15.31:0/50744026 wait complete. 2024-09-21T06:16:43.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.187+0000 7f1b913f8740 20 client.26330 trim_cache size 0 max 16384 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:16:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:16:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b6c0b6780 msgr2=0x7f1b6c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b6c0b6780 0x7f1b6c0b8c90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55d6882df2d0 tx=0x7f1b780097c0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b8411cc30 msgr2=0x7f1b841aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b8411cc30 0x7f1b841aeba0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00efc0 tx=0x7f1b7c00c800 comp rx=0 tx=0).stop 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 shutdown_connections 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b8411cc30 0x7f1b841aeba0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b6c0b6780 0x7f1b6c0b8c90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.189+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b84111f40 0x7f1b841a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.190+0000 7f1b8ae84640 1 --2- 172.21.15.31:0/2010495664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b8410a9e0 0x7f1b841a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.190+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 >> 172.21.15.31:0/2010495664 conn(0x7f1b84070ec0 msgr2=0x7f1b8411e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.190+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 shutdown_connections 2024-09-21T06:16:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:43.190+0000 7f1b8ae84640 1 -- 172.21.15.31:0/2010495664 wait complete. 2024-09-21T06:16:43.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:42 smithi110 ceph-mon[34550]: pgmap v473: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 954 op/s 2024-09-21T06:16:43.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:42 smithi107 ceph-mon[34380]: pgmap v473: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 954 op/s 2024-09-21T06:16:44.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:16:44.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:16:44.646 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:45.133 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:44 smithi031 ceph-mon[31237]: pgmap v474: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-09-21T06:16:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 305.0 s, 3825.0 tps, lat 26.120 ms stddev 31.604 2024-09-21T06:16:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 -- 172.21.15.31:0/3821802451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c101b60 msgr2=0x7f586c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 --2- 172.21.15.31:0/3821802451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c101b60 0x7f586c101f70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5868009900 tx=0x7f586802a410 comp rx=0 tx=0).stop 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 -- 172.21.15.31:0/3821802451 shutdown_connections 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 --2- 172.21.15.31:0/3821802451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c109e10 0x7f586c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 --2- 172.21.15.31:0/3821802451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f586c1094c0 0x7f586c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 --2- 172.21.15.31:0/3821802451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c101b60 0x7f586c101f70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 -- 172.21.15.31:0/3821802451 >> 172.21.15.31:0/3821802451 conn(0x7f586c077800 msgr2=0x7f586c077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.268+0000 7f5874702640 1 -- 172.21.15.31:0/3821802451 shutdown_connections 2024-09-21T06:16:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.269+0000 7f5874702640 1 -- 172.21.15.31:0/3821802451 wait complete. 2024-09-21T06:16:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.269+0000 7f5874702640 1 Processor -- start 2024-09-21T06:16:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 -- start start 2024-09-21T06:16:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f586c101b60 0x7f586c1104d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:45.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 0x7f586c10c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c109e10 0x7f586c10ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f586c103990 con 0x7f586c109e10 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f586c103810 con 0x7f586c1094c0 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5874702640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f586c103b10 con 0x7f586c101b60 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.270+0000 7f5871c9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 0x7f586c10c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 0x7f586c10c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54260/0 (socket says 172.21.15.31:54260) 2024-09-21T06:16:45.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 -- 172.21.15.31:0/2237600962 learned_addr learned my addr 172.21.15.31:0/2237600962 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5872c9f640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c109e10 0x7f586c10ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 -- 172.21.15.31:0/2237600962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f586c101b60 msgr2=0x7f586c1104d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f587249e640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f586c101b60 0x7f586c1104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f586c101b60 0x7f586c1104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 -- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c109e10 msgr2=0x7f586c10ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c109e10 0x7f586c10ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 -- 172.21.15.31:0/2237600962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c1acd60 con 0x7f586c1094c0 2024-09-21T06:16:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.271+0000 7f5871c9d640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 0x7f586c10c7b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f586400ea20 tx=0x7f586400eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:45.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.272+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586400cde0 con 0x7f586c1094c0 2024-09-21T06:16:45.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.272+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5864005020 con 0x7f586c1094c0 2024-09-21T06:16:45.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.272+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c1acf30 con 0x7f586c1094c0 2024-09-21T06:16:45.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.272+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58640116d0 con 0x7f586c1094c0 2024-09-21T06:16:45.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.272+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1ad330 con 0x7f586c1094c0 2024-09-21T06:16:45.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.274+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c101f70 con 0x7f586c1094c0 2024-09-21T06:16:45.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.276+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5864004b20 con 0x7f586c1094c0 2024-09-21T06:16:45.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:44 smithi110 ceph-mon[34550]: pgmap v474: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-09-21T06:16:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.277+0000 7f58637fe640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f58580b66b0 0x7f58580b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.277+0000 7f587249e640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f58580b66b0 0x7f58580b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:45.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.277+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f58640e7d90 con 0x7f586c1094c0 2024-09-21T06:16:45.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.279+0000 7f587249e640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f58580b66b0 0x7f58580b8bc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5868009850 tx=0x7f5868005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:45.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.283+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5864016030 con 0x7f586c1094c0 2024-09-21T06:16:45.288 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:44 smithi107 ceph-mon[34380]: pgmap v474: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.0 MiB/s wr, 689 op/s 2024-09-21T06:16:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.668+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f586c10dab0 con 0x7f586c1094c0 2024-09-21T06:16:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.669+0000 7f58637fe640 1 -- 172.21.15.31:0/2237600962 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f58640ae320 con 0x7f586c1094c0 2024-09-21T06:16:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:16:45.672 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:16:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f58580b66b0 msgr2=0x7f58580b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f58580b66b0 0x7f58580b8bc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5868009850 tx=0x7f5868005fd0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 msgr2=0x7f586c10c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 0x7f586c10c7b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f586400ea20 tx=0x7f586400eee0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 shutdown_connections 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f58580b66b0 0x7f58580b8bc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f586c109e10 0x7f586c10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f586c1094c0 0x7f586c10c7b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 --2- 172.21.15.31:0/2237600962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f586c101b60 0x7f586c1104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 >> 172.21.15.31:0/2237600962 conn(0x7f586c077800 msgr2=0x7f586c1083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 shutdown_connections 2024-09-21T06:16:45.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:45.673+0000 7f5874702640 1 -- 172.21.15.31:0/2237600962 wait complete. 2024-09-21T06:16:46.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:45 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2237600962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:46.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:45 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2237600962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:46.330 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:45 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2237600962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:46.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:16:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:47.172 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:46 smithi031 ceph-mon[31237]: pgmap v475: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 959 op/s 2024-09-21T06:16:47.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:46 smithi110 ceph-mon[34550]: pgmap v475: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 959 op/s 2024-09-21T06:16:47.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:46 smithi107 ceph-mon[34380]: pgmap v475: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 10 MiB/s wr, 959 op/s 2024-09-21T06:16:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.474+0000 7f0f47fff640 1 -- 172.21.15.31:0/2314079374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 msgr2=0x7f0f4811efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.474+0000 7f0f47fff640 1 --2- 172.21.15.31:0/2314079374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f4811efc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f4001a910 tx=0x7f0f4003b430 comp rx=0 tx=0).stop 2024-09-21T06:16:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.475+0000 7f0f47fff640 1 -- 172.21.15.31:0/2314079374 shutdown_connections 2024-09-21T06:16:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.475+0000 7f0f47fff640 1 --2- 172.21.15.31:0/2314079374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f4811efc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.475+0000 7f0f47fff640 1 --2- 172.21.15.31:0/2314079374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4810dbf0 0x7f0f4810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.475+0000 7f0f47fff640 1 --2- 172.21.15.31:0/2314079374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f48111cd0 0x7f0f4810d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.475+0000 7f0f47fff640 1 -- 172.21.15.31:0/2314079374 >> 172.21.15.31:0/2314079374 conn(0x7f0f4806e9f0 msgr2=0x7f0f4806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.475+0000 7f0f47fff640 1 -- 172.21.15.31:0/2314079374 shutdown_connections 2024-09-21T06:16:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.477+0000 7f0f47fff640 1 -- 172.21.15.31:0/2314079374 wait complete. 2024-09-21T06:16:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.478+0000 7f0f47fff640 1 Processor -- start 2024-09-21T06:16:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.478+0000 7f0f47fff640 1 -- start start 2024-09-21T06:16:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.478+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4810dbf0 0x7f0f4811c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.478+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f48111cd0 0x7f0f481186b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f48118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f47fff640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48114bf0 con 0x7f0f4810dbf0 2024-09-21T06:16:47.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f47fff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48114a70 con 0x7f0f48111cd0 2024-09-21T06:16:47.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f47fff640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48114d70 con 0x7f0f4811cba0 2024-09-21T06:16:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f48118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f48118bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47988/0 (socket says 172.21.15.31:47988) 2024-09-21T06:16:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.479+0000 7f0f477fe640 1 -- 172.21.15.31:0/3145935028 learned_addr learned my addr 172.21.15.31:0/3145935028 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.480+0000 7f0f477fe640 1 -- 172.21.15.31:0/3145935028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f48111cd0 msgr2=0x7f0f481186b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:16:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.480+0000 7f0f477fe640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f48111cd0 0x7f0f481186b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.480+0000 7f0f477fe640 1 -- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4810dbf0 msgr2=0x7f0f4811c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.480+0000 7f0f477fe640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4810dbf0 0x7f0f4811c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.480+0000 7f0f477fe640 1 -- 172.21.15.31:0/3145935028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f481193f0 con 0x7f0f4811cba0 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.480+0000 7f0f477fe640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f48118bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f4003b9a0 tx=0x7f0f40005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.481+0000 7f0f4c987640 1 -- 172.21.15.31:0/3145935028 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4004f020 con 0x7f0f4811cba0 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.481+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f481acc00 con 0x7f0f4811cba0 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.481+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f481ad0b0 con 0x7f0f4811cba0 2024-09-21T06:16:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.481+0000 7f0f4c987640 1 -- 172.21.15.31:0/3145935028 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f40004b70 con 0x7f0f4811cba0 2024-09-21T06:16:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.482+0000 7f0f4c987640 1 -- 172.21.15.31:0/3145935028 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40046070 con 0x7f0f4811cba0 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.485+0000 7f0f53efd740 1 Processor -- start 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.487+0000 7f0f53efd740 1 -- start start 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.487+0000 7f0f4c987640 1 -- 172.21.15.31:0/3145935028 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0f40052020 con 0x7f0f4811cba0 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 0x5654e7664460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e76649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 0x5654e7664460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e729c390 0x5654e7664ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f53efd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5654e75daea0 con 0x5654e72bd070 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 0x5654e7664460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37578/0 (socket says 172.21.15.31:37578) 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f467fc640 1 -- 172.21.15.31:0/979843428 learned_addr learned my addr 172.21.15.31:0/979843428 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654e75ca2d0 con 0x5654e72941b0 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654e75ce0e0 con 0x5654e729c390 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f467fc640 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e729c390 0x5654e7664ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.488+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e76649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.489+0000 7f0f467fc640 1 -- 172.21.15.31:0/979843428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e729c390 msgr2=0x5654e7664ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.489+0000 7f0f467fc640 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e729c390 0x5654e7664ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.489+0000 7f0f467fc640 1 -- 172.21.15.31:0/979843428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 msgr2=0x5654e76649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.489+0000 7f0f467fc640 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e76649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.489+0000 7f0f467fc640 1 -- 172.21.15.31:0/979843428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e7328650 con 0x5654e72bd070 2024-09-21T06:16:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.490+0000 7f0f467fc640 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 0x5654e7664460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f30014750 tx=0x7f0f3000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99de2a0b29f5540 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.490+0000 7f0f4c987640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0f340b66f0 0x7f0f340b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.490+0000 7f0f3dffb640 1 -- 172.21.15.31:0/979843428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30005220 con 0x5654e72bd070 2024-09-21T06:16:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.490+0000 7f0f3dffb640 1 -- 172.21.15.31:0/979843428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f30004c50 con 0x5654e72bd070 2024-09-21T06:16:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.491+0000 7f0f4c987640 1 -- 172.21.15.31:0/3145935028 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0f40117fa0 con 0x7f0f4811cba0 2024-09-21T06:16:47.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.491+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0f340b66f0 0x7f0f340b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.492+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0f340b66f0 0x7f0f340b8c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5654e76630e0 tx=0x7f0f380098e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:47.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.494+0000 7f0f3dffb640 1 -- 172.21.15.31:0/979843428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30011670 con 0x5654e72bd070 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 msgr2=0x5654e7664460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 0x5654e7664460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f30014750 tx=0x7f0f3000b260 comp rx=0 tx=0).stop 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 shutdown_connections 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e729c390 0x5654e7664ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e76649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 --2- 172.21.15.31:0/979843428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72bd070 0x5654e7664460 secure :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f30014750 tx=0x7f0f3000b260 comp rx=0 tx=0).stop 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 >> 172.21.15.31:0/979843428 conn(0x5654e72b1460 msgr2=0x5654e7662350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 shutdown_connections 2024-09-21T06:16:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.495+0000 7f0f53efd740 1 -- 172.21.15.31:0/979843428 wait complete. 2024-09-21T06:16:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.496+0000 7f0f53efd740 1 -- start start 2024-09-21T06:16:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.496+0000 7f0f53efd740 1 Processor -- start 2024-09-21T06:16:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e72941b0 0x5654e766fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e76701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 0x5654e767bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5654e742b700 con 0x5654e727b9f0 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654e7565160 con 0x5654e729c390 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654e7672c40 con 0x5654e72941b0 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f53efd740 10 client.0 _ll_register_callbacks cb 0x5654e757e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e76701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 0x5654e767bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e76701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54290/0 (socket says 172.21.15.31:54290) 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f467fc640 1 -- 172.21.15.31:0/770990704 learned_addr learned my addr 172.21.15.31:0/770990704 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 0x5654e767bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37592/0 (socket says 172.21.15.31:37592) 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e72941b0 0x5654e766fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 -- 172.21.15.31:0/770990704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e72941b0 msgr2=0x5654e766fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e72941b0 0x5654e766fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 -- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 msgr2=0x5654e76701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e76701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f477fe640 1 -- 172.21.15.31:0/770990704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e7592c90 con 0x5654e727b9f0 2024-09-21T06:16:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.497+0000 7f0f467fc640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e76701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.498+0000 7f0f477fe640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 0x5654e767bfc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0f401156d0 tx=0x7f0f400dd720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0996e4fafcfe787 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.499+0000 7f0f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:16:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.499+0000 7f0f26ffd640 1 -- 172.21.15.31:0/770990704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40040740 con 0x5654e727b9f0 2024-09-21T06:16:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.499+0000 7f0f26ffd640 1 -- 172.21.15.31:0/770990704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0f40115950 con 0x5654e727b9f0 2024-09-21T06:16:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.499+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f30004c50 con 0x5654e727b9f0 2024-09-21T06:16:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.499+0000 7f0f26ffd640 1 -- 172.21.15.31:0/770990704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0f40115b50 con 0x5654e727b9f0 2024-09-21T06:16:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.500+0000 7f0f53efd740 10 client.16620 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.500+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5654e767eab0 con 0x5654e727b9f0 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.501+0000 7f0f26ffd640 1 -- 172.21.15.31:0/770990704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f40115b50 con 0x5654e727b9f0 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.501+0000 7f0f26ffd640 1 -- 172.21.15.31:0/770990704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0f401051a0 con 0x5654e727b9f0 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.501+0000 7f0f53efd740 10 client.16620 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.501+0000 7f0f53efd740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.501+0000 7f0f53efd740 20 client.16620 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.502+0000 7f0f53efd740 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e7250da0 0x5654e7681ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.502+0000 7f0f53efd740 4 client.16620 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.502+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654e75dcdf0 con 0x5654e7250da0 2024-09-21T06:16:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.502+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e7250da0 0x5654e7681ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.503+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e7250da0 0x5654e7681ff0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c1613daaaaf761 server_cookie=d27afb6c46389b41 in_seq=0 out_seq=0 2024-09-21T06:16:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f26ffd640 10 client.16620 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:47.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f26ffd640 1 -- 172.21.15.31:0/770990704 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654e75dcdf0 con 0x5654e7250da0 2024-09-21T06:16:47.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f26ffd640 10 client.16620 handle_command_reply: tid=0 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 client.16620 shutdown 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 msgr2=0x5654e767bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 0x5654e767bfc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0f401156d0 tx=0x7f0f400dd720 comp rx=0 tx=0).stop 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 shutdown_connections 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e7250da0 0x5654e7681ff0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e727b9f0 0x5654e767bfc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e76701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 --2- 172.21.15.31:0/770990704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e72941b0 0x5654e766fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 >> 172.21.15.31:0/770990704 conn(0x5654e72b1460 msgr2=0x5654e76675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.504+0000 7f0f26ffd640 0 client.16620 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:47.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.505+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 shutdown_connections 2024-09-21T06:16:47.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.505+0000 7f0f53efd740 1 -- 172.21.15.31:0/770990704 wait complete. 2024-09-21T06:16:47.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.505+0000 7f0f53efd740 20 client.16620 trim_cache size 0 max 16384 2024-09-21T06:16:47.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.526+0000 7f0f53efd740 1 Processor -- start 2024-09-21T06:16:47.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.526+0000 7f0f53efd740 1 -- start start 2024-09-21T06:16:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.526+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7662380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.526+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e767bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.526+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e729c390 0x5654e767c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f53efd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800c7a0 con 0x5654e729c390 2024-09-21T06:16:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f53efd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800f060 con 0x5654e72941b0 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f53efd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800ce50 con 0x5654e727b9f0 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7662380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7662380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48006/0 (socket says 172.21.15.31:48006) 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 -- 172.21.15.31:0/1084151954 learned_addr learned my addr 172.21.15.31:0/1084151954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f467fc640 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e767bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 -- 172.21.15.31:0/1084151954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 msgr2=0x5654e767bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e767bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 -- 172.21.15.31:0/1084151954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e729c390 msgr2=0x5654e767c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e729c390 0x5654e767c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 -- 172.21.15.31:0/1084151954 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654e7666980 con 0x5654e727b9f0 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7662380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005fd0 tx=0x7f0f38014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fbd1852701daebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f3d7fa640 1 -- 172.21.15.31:0/1084151954 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3801f900 con 0x5654e727b9f0 2024-09-21T06:16:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f3d7fa640 1 -- 172.21.15.31:0/1084151954 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3802c070 con 0x5654e727b9f0 2024-09-21T06:16:47.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.527+0000 7f0f3d7fa640 1 -- 172.21.15.31:0/1084151954 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3801c400 con 0x5654e727b9f0 2024-09-21T06:16:47.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.528+0000 7f0f53efd740 1 -- 172.21.15.31:0/1084151954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 msgr2=0x5654e7662380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.528+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7662380 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005fd0 tx=0x7f0f38014000 comp rx=0 tx=0).stop 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 -- 172.21.15.31:0/1084151954 shutdown_connections 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e729c390 0x5654e767c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e72941b0 0x5654e767bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1084151954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7662380 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 -- 172.21.15.31:0/1084151954 >> 172.21.15.31:0/1084151954 conn(0x5654e72b1460 msgr2=0x5654e7661c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 -- 172.21.15.31:0/1084151954 shutdown_connections 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 -- 172.21.15.31:0/1084151954 wait complete. 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.529+0000 7f0f53efd740 1 -- start start 2024-09-21T06:16:47.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f53efd740 1 Processor -- start 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7674070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72941b0 0x5654e76745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f53efd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e7674af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72941b0 0x5654e76745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7674070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f467fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72941b0 0x5654e76745b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37608/0 (socket says 172.21.15.31:37608) 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.530+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7674070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48016/0 (socket says 172.21.15.31:48016) 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f467fc640 1 -- 172.21.15.31:0/1866772007 learned_addr learned my addr 172.21.15.31:0/1866772007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f477fe640 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e7674af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800d520 con 0x5654e72941b0 2024-09-21T06:16:47.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800e990 con 0x5654e729c390 2024-09-21T06:16:47.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2800dbf0 con 0x5654e727b9f0 2024-09-21T06:16:47.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f53efd740 10 client.0 _ll_register_callbacks cb 0x7f0f40005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:47.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f477fe640 1 -- 172.21.15.31:0/1866772007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 msgr2=0x5654e7674070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f477fe640 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7674070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f477fe640 1 -- 172.21.15.31:0/1866772007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72941b0 msgr2=0x5654e76745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:47.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f477fe640 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72941b0 0x5654e76745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.531+0000 7f0f477fe640 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654e76ebb70 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.532+0000 7f0f477fe640 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e7674af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0f400ddd60 tx=0x7f0f40040a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddf9078318d7eab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.532+0000 7f0f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.532+0000 7f0f26ffd640 1 -- 172.21.15.31:0/1866772007 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400d9c70 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.532+0000 7f0f26ffd640 1 -- 172.21.15.31:0/1866772007 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f0f40042bc0 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.532+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3802c070 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f26ffd640 1 -- 172.21.15.31:0/1866772007 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0f40115b60 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 10 client.26348 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5654e76ebb70 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f26ffd640 1 -- 172.21.15.31:0/1866772007 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f30004c50 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f26ffd640 1 -- 172.21.15.31:0/1866772007 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0f400dd030 con 0x5654e729c390 2024-09-21T06:16:47.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 10 client.26348 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 20 client.26348 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e76ed140 0x5654e76ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 4 client.26348 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.533+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0f3802c070 con 0x5654e76ed140 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.534+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e76ed140 0x5654e76ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.534+0000 7f0f26ffd640 10 client.26348 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.534+0000 7f0f46ffd640 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e76ed140 0x5654e76ef560 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af722a62a2b36920 server_cookie=f85c29d1f40e5d98 in_seq=0 out_seq=0 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.535+0000 7f0f26ffd640 1 -- 172.21.15.31:0/1866772007 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0f3802c070 con 0x5654e76ed140 2024-09-21T06:16:47.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.535+0000 7f0f26ffd640 10 client.26348 handle_command_reply: tid=0 2024-09-21T06:16:47.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 client.26348 shutdown 2024-09-21T06:16:47.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 msgr2=0x5654e7674af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e7674af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0f400ddd60 tx=0x7f0f40040a90 comp rx=0 tx=0).stop 2024-09-21T06:16:47.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 shutdown_connections 2024-09-21T06:16:47.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654e76ed140 0x5654e76ef560 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654e729c390 0x5654e7674af0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f26ffd640 0 client.26348 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654e72941b0 0x5654e76745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 --2- 172.21.15.31:0/1866772007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654e727b9f0 0x5654e7674070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 >> 172.21.15.31:0/1866772007 conn(0x5654e72b1460 msgr2=0x5654e7665490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:47.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 shutdown_connections 2024-09-21T06:16:47.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 1 -- 172.21.15.31:0/1866772007 wait complete. 2024-09-21T06:16:47.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.536+0000 7f0f53efd740 20 client.26348 trim_cache size 0 max 16384 2024-09-21T06:16:47.543 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:16:47.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0f340b66f0 msgr2=0x7f0f340b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0f340b66f0 0x7f0f340b8c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5654e76630e0 tx=0x7f0f380098e0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 msgr2=0x7f0f48118bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f48118bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f4003b9a0 tx=0x7f0f40005400 comp rx=0 tx=0).stop 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 shutdown_connections 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f4811cba0 0x7f0f48118bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f48111cd0 0x7f0f481186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0f340b66f0 0x7f0f340b8c00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 --2- 172.21.15.31:0/3145935028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f4810dbf0 0x7f0f4811c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 >> 172.21.15.31:0/3145935028 conn(0x7f0f4806e9f0 msgr2=0x7f0f4811dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 shutdown_connections 2024-09-21T06:16:47.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:47.538+0000 7f0f47fff640 1 -- 172.21.15.31:0/3145935028 wait complete. 2024-09-21T06:16:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:48 smithi110 ceph-mon[34550]: pgmap v476: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.0 MiB/s wr, 832 op/s 2024-09-21T06:16:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:48 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:16:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:48 smithi107 ceph-mon[34380]: pgmap v476: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.0 MiB/s wr, 832 op/s 2024-09-21T06:16:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:48 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:16:49.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:16:49.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:16:49.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:48 smithi031 ceph-mon[31237]: pgmap v476: 97 pgs: 97 active+clean; 3.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.0 MiB/s wr, 832 op/s 2024-09-21T06:16:49.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:48 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:16:49.778 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 310.0 s, 3197.2 tps, lat 31.376 ms stddev 38.670 2024-09-21T06:16:50.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:16:50.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:16:50.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:16:50.286 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:16:50.286 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:16:50.286 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:16:50.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:16:50.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:16:50.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:16:50.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 -- 172.21.15.31:0/2361884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 msgr2=0x7f2f681098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 --2- 172.21.15.31:0/2361884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f681098c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009ca0 tx=0x7f2f5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 -- 172.21.15.31:0/2361884478 shutdown_connections 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 --2- 172.21.15.31:0/2361884478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f68109e00 0x7f2f68110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 --2- 172.21.15.31:0/2361884478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f681098c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 --2- 172.21.15.31:0/2361884478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 0x7f2f68103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.402+0000 7f2f701bc640 1 -- 172.21.15.31:0/2361884478 >> 172.21.15.31:0/2361884478 conn(0x7f2f6807b910 msgr2=0x7f2f6807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.403+0000 7f2f701bc640 1 -- 172.21.15.31:0/2361884478 shutdown_connections 2024-09-21T06:16:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.403+0000 7f2f701bc640 1 -- 172.21.15.31:0/2361884478 wait complete. 2024-09-21T06:16:50.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.404+0000 7f2f701bc640 1 Processor -- start 2024-09-21T06:16:50.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.404+0000 7f2f701bc640 1 -- start start 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.404+0000 7f2f701bc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 0x7f2f6810e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f701bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f6810a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f701bc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f68109e00 0x7f2f6810ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f701bc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68113ac0 con 0x7f2f681094b0 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f701bc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68113940 con 0x7f2f68109e00 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f701bc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68113c40 con 0x7f2f68103bc0 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6d757640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f6810a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6d757640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f6810a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55878/0 (socket says 172.21.15.31:55878) 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6d757640 1 -- 172.21.15.31:0/3645140069 learned_addr learned my addr 172.21.15.31:0/3645140069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6df58640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 0x7f2f6810e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f68109e00 0x7f2f6810ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6df58640 1 -- 172.21.15.31:0/3645140069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f68109e00 msgr2=0x7f2f6810ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6df58640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f68109e00 0x7f2f6810ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6df58640 1 -- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 msgr2=0x7f2f6810a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6df58640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f6810a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.405+0000 7f2f6df58640 1 -- 172.21.15.31:0/3645140069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f681accc0 con 0x7f2f68103bc0 2024-09-21T06:16:50.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.406+0000 7f2f6d757640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f6810a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:50.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.406+0000 7f2f6df58640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 0x7f2f6810e300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e9f0 tx=0x7f2f5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:50.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.407+0000 7f2f66ffd640 1 -- 172.21.15.31:0/3645140069 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800ccc0 con 0x7f2f68103bc0 2024-09-21T06:16:50.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.407+0000 7f2f66ffd640 1 -- 172.21.15.31:0/3645140069 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f580108e0 con 0x7f2f68103bc0 2024-09-21T06:16:50.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.407+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f681acef0 con 0x7f2f68103bc0 2024-09-21T06:16:50.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.407+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f681ad3a0 con 0x7f2f68103bc0 2024-09-21T06:16:50.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.408+0000 7f2f66ffd640 1 -- 172.21.15.31:0/3645140069 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58018880 con 0x7f2f68103bc0 2024-09-21T06:16:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.411+0000 7f2f76730740 1 Processor -- start 2024-09-21T06:16:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 -- start start 2024-09-21T06:16:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e378430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e378970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e378eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556e7e2eee60 con 0x556e7dfd1350 2024-09-21T06:16:50.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e7e2de290 con 0x556e7dfa8490 2024-09-21T06:16:50.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.412+0000 7f2f76730740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556e7e2e20a0 con 0x556e7dfb0670 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.413+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e378430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.413+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e378430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55882/0 (socket says 172.21.15.31:55882) 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.413+0000 7f2f6e759640 1 -- 172.21.15.31:0/1653117149 learned_addr learned my addr 172.21.15.31:0/1653117149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.414+0000 7f2f6e759640 1 -- 172.21.15.31:0/1653117149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 msgr2=0x556e7e378eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.414+0000 7f2f6e759640 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e378eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.414+0000 7f2f6e759640 1 -- 172.21.15.31:0/1653117149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 msgr2=0x556e7e378970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.414+0000 7f2f6e759640 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e378970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.414+0000 7f2f6e759640 1 -- 172.21.15.31:0/1653117149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e7e03c930 con 0x556e7dfd1350 2024-09-21T06:16:50.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.414+0000 7f2f6e759640 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e378430 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009c20 tx=0x7f2f60011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0dda9c7ee0994c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.421+0000 7f2f66ffd640 1 -- 172.21.15.31:0/3645140069 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2f58010430 con 0x7f2f68103bc0 2024-09-21T06:16:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.423+0000 7f2f66ffd640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f500b6780 0x7f2f500b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.423+0000 7f2f66ffd640 1 -- 172.21.15.31:0/3645140069 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2f580e7890 con 0x7f2f68103bc0 2024-09-21T06:16:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.423+0000 7f2f6df58640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f500b6780 0x7f2f500b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.424+0000 7f2f4ffff640 1 -- 172.21.15.31:0/1653117149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6001e070 con 0x556e7dfd1350 2024-09-21T06:16:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.424+0000 7f2f4ffff640 1 -- 172.21.15.31:0/1653117149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f60019ae0 con 0x556e7dfd1350 2024-09-21T06:16:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.424+0000 7f2f4ffff640 1 -- 172.21.15.31:0/1653117149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6000e990 con 0x556e7dfd1350 2024-09-21T06:16:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.425+0000 7f2f76730740 1 -- 172.21.15.31:0/1653117149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 msgr2=0x556e7e378430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.425+0000 7f2f76730740 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e378430 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009c20 tx=0x7f2f60011810 comp rx=0 tx=0).stop 2024-09-21T06:16:50.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.428+0000 7f2f6df58640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f500b6780 0x7f2f500b8c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f58000c00 tx=0x7f2f5800ce60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.432+0000 7f2f76730740 1 -- 172.21.15.31:0/1653117149 shutdown_connections 2024-09-21T06:16:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.432+0000 7f2f76730740 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e378eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.432+0000 7f2f76730740 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e378970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.432+0000 7f2f76730740 1 --2- 172.21.15.31:0/1653117149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e378430 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.432+0000 7f2f76730740 1 -- 172.21.15.31:0/1653117149 >> 172.21.15.31:0/1653117149 conn(0x556e7dfc5740 msgr2=0x556e7e376320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:50.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.437+0000 7f2f76730740 1 -- 172.21.15.31:0/1653117149 shutdown_connections 2024-09-21T06:16:50.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.439+0000 7f2f76730740 1 -- 172.21.15.31:0/1653117149 wait complete. 2024-09-21T06:16:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.439+0000 7f2f76730740 1 -- start start 2024-09-21T06:16:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.439+0000 7f2f76730740 1 Processor -- start 2024-09-21T06:16:50.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.440+0000 7f2f76730740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e384020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.440+0000 7f2f76730740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e384560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.440+0000 7f2f76730740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e38ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.440+0000 7f2f76730740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556e7e13f890 con 0x556e7dfd1350 2024-09-21T06:16:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.440+0000 7f2f76730740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556e7e278d40 con 0x556e7dfa8490 2024-09-21T06:16:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.440+0000 7f2f76730740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556e7e386c10 con 0x556e7dfb0670 2024-09-21T06:16:50.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.441+0000 7f2f76730740 10 client.0 _ll_register_callbacks cb 0x556e7e2dcc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:50.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.445+0000 7f2f6d757640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e384560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.446+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e384020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.446+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e384020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56186/0 (socket says 172.21.15.31:56186) 2024-09-21T06:16:50.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.446+0000 7f2f6e759640 1 -- 172.21.15.31:0/3569547151 learned_addr learned my addr 172.21.15.31:0/3569547151 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:50.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.446+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e38ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.449+0000 7f2f6d757640 1 -- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 msgr2=0x556e7e384020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.449+0000 7f2f6d757640 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e384020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.449+0000 7f2f6d757640 1 -- 172.21.15.31:0/3569547151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 msgr2=0x556e7e38ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.449+0000 7f2f6d757640 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e38ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.449+0000 7f2f6d757640 1 -- 172.21.15.31:0/3569547151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e7e390490 con 0x556e7dfb0670 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.450+0000 7f2f6d757640 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e384560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005e30 tx=0x7f2f5c031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3746142ed8dc3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.450+0000 7f2f4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.451+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/3569547151 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c003b70 con 0x556e7dfb0670 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.451+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f60019ae0 con 0x556e7dfb0670 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.451+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/3569547151 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2f5c003d10 con 0x556e7dfb0670 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.452+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/3569547151 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c032eb0 con 0x556e7dfb0670 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.452+0000 7f2f76730740 10 client.26398 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:50.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.452+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556e7e392dc0 con 0x556e7dfb0670 2024-09-21T06:16:50.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.454+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/3569547151 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f60019ae0 con 0x556e7dfb0670 2024-09-21T06:16:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/3569547151 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2f5c044070 con 0x556e7dfb0670 2024-09-21T06:16:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f76730740 10 client.26398 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f76730740 10 client.26398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f76730740 20 client.26398 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f76730740 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df8fcd0 0x556e7e3964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f76730740 4 client.26398 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.455+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e7e2f0db0 con 0x556e7df8fcd0 2024-09-21T06:16:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.456+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df8fcd0 0x556e7e3964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.457+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df8fcd0 0x556e7e3964f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ce236013b3c693 server_cookie=26f244323aedf7b3 in_seq=0 out_seq=0 2024-09-21T06:16:50.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.457+0000 7f2f4d7fa640 10 client.26398 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.460+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/3569547151 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e7e2f0db0 con 0x556e7df8fcd0 2024-09-21T06:16:50.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.460+0000 7f2f4d7fa640 10 client.26398 handle_command_reply: tid=0 2024-09-21T06:16:50.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.462+0000 7f2f76730740 1 client.26398 shutdown 2024-09-21T06:16:50.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.463+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 msgr2=0x556e7e384560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.463+0000 7f2f76730740 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e384560 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005e30 tx=0x7f2f5c031a50 comp rx=0 tx=0).stop 2024-09-21T06:16:50.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 shutdown_connections 2024-09-21T06:16:50.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f76730740 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df8fcd0 0x556e7e3964f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f76730740 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfd1350 0x556e7e38ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f76730740 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfb0670 0x556e7e384560 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f76730740 1 --2- 172.21.15.31:0/3569547151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e384020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 >> 172.21.15.31:0/3569547151 conn(0x556e7dfc5740 msgr2=0x556e7e37b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:50.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.466+0000 7f2f4d7fa640 0 client.26398 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:50.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.470+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 shutdown_connections 2024-09-21T06:16:50.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.470+0000 7f2f76730740 1 -- 172.21.15.31:0/3569547151 wait complete. 2024-09-21T06:16:50.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.470+0000 7f2f76730740 20 client.26398 trim_cache size 0 max 16384 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.512+0000 7f2f76730740 1 Processor -- start 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.512+0000 7f2f76730740 1 -- start start 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.513+0000 7f2f76730740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e376420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.513+0000 7f2f76730740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfb0670 0x556e7e38fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.513+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e376420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.513+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e376420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56200/0 (socket says 172.21.15.31:56200) 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.513+0000 7f2f6e759640 1 -- 172.21.15.31:0/3362239352 learned_addr learned my addr 172.21.15.31:0/3362239352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f76730740 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e37d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b3dc0 con 0x556e7dfb0670 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b6560 con 0x556e7dfa8490 2024-09-21T06:16:50.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b4350 con 0x556e7dfd1350 2024-09-21T06:16:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f6d757640 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfb0670 0x556e7e38fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e37d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f6e759640 1 -- 172.21.15.31:0/3362239352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 msgr2=0x556e7e37d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e37d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.514+0000 7f2f6e759640 1 -- 172.21.15.31:0/3362239352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfb0670 msgr2=0x556e7e38fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.515+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfb0670 0x556e7e38fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.515+0000 7f2f6e759640 1 -- 172.21.15.31:0/3362239352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e7e37ab60 con 0x556e7dfa8490 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.515+0000 7f2f6e759640 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e376420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009c20 tx=0x7f2f60005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f18888aeb2af90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.515+0000 7f2f4f7fe640 1 -- 172.21.15.31:0/3362239352 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6002d070 con 0x556e7dfa8490 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.515+0000 7f2f4f7fe640 1 -- 172.21.15.31:0/3362239352 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f60005510 con 0x556e7dfa8490 2024-09-21T06:16:50.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.515+0000 7f2f4f7fe640 1 -- 172.21.15.31:0/3362239352 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6001e040 con 0x556e7dfa8490 2024-09-21T06:16:50.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 msgr2=0x556e7e376420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e376420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009c20 tx=0x7f2f60005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 shutdown_connections 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e37d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfb0670 0x556e7e38fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 --2- 172.21.15.31:0/3362239352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfa8490 0x556e7e376420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 >> 172.21.15.31:0/3362239352 conn(0x556e7dfc5740 msgr2=0x556e7e375d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 shutdown_connections 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.517+0000 7f2f76730740 1 -- 172.21.15.31:0/3362239352 wait complete. 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.518+0000 7f2f76730740 1 -- start start 2024-09-21T06:16:50.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.518+0000 7f2f76730740 1 Processor -- start 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.519+0000 7f2f76730740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfa8490 0x556e7e38c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f76730740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfb0670 0x556e7e38c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f76730740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e386e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f76730740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b4a20 con 0x556e7dfa8490 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f76730740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b5e90 con 0x556e7dfb0670 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f76730740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681b50f0 con 0x556e7dfd1350 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f76730740 10 client.0 _ll_register_callbacks cb 0x7f2f58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfa8490 0x556e7e38c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f6e759640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfa8490 0x556e7e38c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55904/0 (socket says 172.21.15.31:55904) 2024-09-21T06:16:50.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f6e759640 1 -- 172.21.15.31:0/479065789 learned_addr learned my addr 172.21.15.31:0/479065789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f6d757640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfb0670 0x556e7e38c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.520+0000 7f2f6e759640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e386e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f6d757640 1 -- 172.21.15.31:0/479065789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 msgr2=0x556e7e386e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f6d757640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e386e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f6d757640 1 -- 172.21.15.31:0/479065789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfa8490 msgr2=0x556e7e38c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f6d757640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfa8490 0x556e7e38c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f6d757640 1 -- 172.21.15.31:0/479065789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e7e387360 con 0x556e7dfb0670 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f6d757640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfb0670 0x556e7e38c5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006010 tx=0x7f2f5c02fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db364cce27170f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/479065789 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c053020 con 0x556e7dfb0670 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.521+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/479065789 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f2f5c003d80 con 0x556e7dfb0670 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.522+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e7e2ec860 con 0x556e7dfb0670 2024-09-21T06:16:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.522+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/479065789 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2f5c05e050 con 0x556e7dfb0670 2024-09-21T06:16:50.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.522+0000 7f2f76730740 10 client.26363 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:50.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.522+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556e7e401690 con 0x556e7dfb0670 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.523+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/479065789 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5c036040 con 0x556e7dfb0670 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/479065789 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2f5c044070 con 0x556e7dfb0670 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f76730740 10 client.26363 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f76730740 10 client.26363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f76730740 20 client.26363 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f76730740 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df65080 0x556e7e403e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f76730740 4 client.26363 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:16:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e7e2ec860 con 0x556e7df65080 2024-09-21T06:16:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.524+0000 7f2f6e759640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df65080 0x556e7e403e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.525+0000 7f2f6e759640 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df65080 0x556e7e403e80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b912420dd7170d13 server_cookie=710a95611783dc2 in_seq=0 out_seq=0 2024-09-21T06:16:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.525+0000 7f2f4d7fa640 10 client.26363 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.526+0000 7f2f4d7fa640 1 -- 172.21.15.31:0/479065789 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e7e2ec860 con 0x556e7df65080 2024-09-21T06:16:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.526+0000 7f2f4d7fa640 10 client.26363 handle_command_reply: tid=0 2024-09-21T06:16:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.526+0000 7f2f76730740 1 client.26363 shutdown 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfb0670 msgr2=0x556e7e38c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfb0670 0x556e7e38c5b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006010 tx=0x7f2f5c02fe20 comp rx=0 tx=0).stop 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 shutdown_connections 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556e7df65080 0x556e7e403e80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e7dfd1350 0x556e7e386e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556e7dfb0670 0x556e7e38c5b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 --2- 172.21.15.31:0/479065789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556e7dfa8490 0x556e7e38c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 >> 172.21.15.31:0/479065789 conn(0x556e7dfc5740 msgr2=0x556e7e37b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f4d7fa640 0 client.26363 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.527+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 shutdown_connections 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.528+0000 7f2f76730740 1 -- 172.21.15.31:0/479065789 wait complete. 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.528+0000 7f2f76730740 20 client.26363 trim_cache size 0 max 16384 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "85354dac-139f-426b-98db-539d23ab1572", 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:16:50.531 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f500b6780 msgr2=0x7f2f500b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f500b6780 0x7f2f500b8c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f58000c00 tx=0x7f2f5800ce60 comp rx=0 tx=0).stop 2024-09-21T06:16:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 msgr2=0x7f2f6810e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 0x7f2f6810e300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e9f0 tx=0x7f2f5800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 shutdown_connections 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f68109e00 0x7f2f6810ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f500b6780 0x7f2f500b8c90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f681094b0 0x7f2f6810a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 --2- 172.21.15.31:0/3645140069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f68103bc0 0x7f2f6810e300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.529+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 >> 172.21.15.31:0/3645140069 conn(0x7f2f6807b910 msgr2=0x7f2f68105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.530+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 shutdown_connections 2024-09-21T06:16:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:50.530+0000 7f2f701bc640 1 -- 172.21.15.31:0/3645140069 wait complete. 2024-09-21T06:16:51.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:51 smithi031 ceph-mon[31237]: pgmap v477: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 889 op/s 2024-09-21T06:16:51.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:51 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:16:51.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:51 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:51.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:51 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:16:51.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:16:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:51 smithi110 ceph-mon[34550]: pgmap v477: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 889 op/s 2024-09-21T06:16:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:51 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:16:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:51 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:51 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:16:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:51 smithi107 ceph-mon[34380]: pgmap v477: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.8 MiB/s wr, 889 op/s 2024-09-21T06:16:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:51 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:16:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:51 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:51 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:16:51.814 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:52.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.509+0000 7f781535f640 1 -- 172.21.15.31:0/3959020907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7810111f40 msgr2=0x7f781010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.509+0000 7f781535f640 1 --2- 172.21.15.31:0/3959020907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7810111f40 0x7f781010a3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f780001a910 tx=0x7f780003b420 comp rx=0 tx=0).stop 2024-09-21T06:16:52.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:52 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:52.512 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:52 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:16:52.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.510+0000 7f781535f640 1 -- 172.21.15.31:0/3959020907 shutdown_connections 2024-09-21T06:16:52.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.510+0000 7f781535f640 1 --2- 172.21.15.31:0/3959020907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f781011cc30 0x7f781011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.510+0000 7f781535f640 1 --2- 172.21.15.31:0/3959020907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.510+0000 7f781535f640 1 --2- 172.21.15.31:0/3959020907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7810111f40 0x7f781010a3c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.510+0000 7f781535f640 1 -- 172.21.15.31:0/3959020907 >> 172.21.15.31:0/3959020907 conn(0x7f7810070ec0 msgr2=0x7f78100712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:52.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.513+0000 7f781535f640 1 -- 172.21.15.31:0/3959020907 shutdown_connections 2024-09-21T06:16:52.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.514+0000 7f781535f640 1 -- 172.21.15.31:0/3959020907 wait complete. 2024-09-21T06:16:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 Processor -- start 2024-09-21T06:16:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 -- start start 2024-09-21T06:16:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781011c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7810111f40 0x7f78101186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f781011cc30 0x7f7810118c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7810114c10 con 0x7f781010a9e0 2024-09-21T06:16:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7810114a90 con 0x7f781011cc30 2024-09-21T06:16:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.515+0000 7f781535f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7810114d90 con 0x7f7810111f40 2024-09-21T06:16:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.516+0000 7f780effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781011c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.516+0000 7f780effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781011c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55916/0 (socket says 172.21.15.31:55916) 2024-09-21T06:16:52.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.516+0000 7f780effd640 1 -- 172.21.15.31:0/2614073054 learned_addr learned my addr 172.21.15.31:0/2614073054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:52.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.519+0000 7f7807fff640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7810111f40 0x7f78101186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.519+0000 7f780f7fe640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f781011cc30 0x7f7810118c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.519+0000 7f780effd640 1 -- 172.21.15.31:0/2614073054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7810111f40 msgr2=0x7f78101186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.519+0000 7f780effd640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7810111f40 0x7f78101186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.519+0000 7f780effd640 1 -- 172.21.15.31:0/2614073054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f781011cc30 msgr2=0x7f7810118c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.519+0000 7f780effd640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f781011cc30 0x7f7810118c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.520+0000 7f780effd640 1 -- 172.21.15.31:0/2614073054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7810119430 con 0x7f781010a9e0 2024-09-21T06:16:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.522+0000 7f780effd640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781011c3f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f780003ba10 tx=0x7f7800004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.523+0000 7f780cff9640 1 -- 172.21.15.31:0/2614073054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780004f070 con 0x7f781010a9e0 2024-09-21T06:16:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.524+0000 7f780cff9640 1 -- 172.21.15.31:0/2614073054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78000409c0 con 0x7f781010a9e0 2024-09-21T06:16:52.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.524+0000 7f780cff9640 1 -- 172.21.15.31:0/2614073054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800046030 con 0x7f781010a9e0 2024-09-21T06:16:52.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.525+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78101acdd0 con 0x7f781010a9e0 2024-09-21T06:16:52.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:52 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:52.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:52 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:16:52.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.525+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101ad200 con 0x7f781010a9e0 2024-09-21T06:16:52.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.529+0000 7f781b8d3740 1 Processor -- start 2024-09-21T06:16:52.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.530+0000 7f781b8d3740 1 -- start start 2024-09-21T06:16:52.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.531+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac7074650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.531+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac7074b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.531+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac70750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.531+0000 7f781b8d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ac6feae50 con 0x564ac6ccd070 2024-09-21T06:16:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.531+0000 7f781b8d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ac6fda280 con 0x564ac6ca41b0 2024-09-21T06:16:52.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.531+0000 7f781b8d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ac6fde090 con 0x564ac6cac390 2024-09-21T06:16:52.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.533+0000 7f780cff9640 1 -- 172.21.15.31:0/2614073054 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7800052070 con 0x7f781010a9e0 2024-09-21T06:16:52.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.533+0000 7f780effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac7074b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.534+0000 7f7807fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac70750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.534+0000 7f7807fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac70750d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52330/0 (socket says 172.21.15.31:52330) 2024-09-21T06:16:52.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.534+0000 7f7807fff640 1 -- 172.21.15.31:0/989700975 learned_addr learned my addr 172.21.15.31:0/989700975 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:52.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.535+0000 7f780cff9640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77fc0b6880 0x7f77fc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.537+0000 7f780f7fe640 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac7074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:52 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:16:52.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:52 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:16:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.539+0000 7f780effd640 1 -- 172.21.15.31:0/989700975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 msgr2=0x564ac70750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.539+0000 7f780effd640 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac70750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.540+0000 7f780effd640 1 -- 172.21.15.31:0/989700975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 msgr2=0x564ac7074650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.540+0000 7f780f7fe640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77fc0b6880 0x7f77fc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.540+0000 7f780effd640 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac7074650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.540+0000 7f780effd640 1 -- 172.21.15.31:0/989700975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac6d38650 con 0x564ac6ca41b0 2024-09-21T06:16:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.540+0000 7f780effd640 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac7074b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7800115f90 tx=0x7f78000a2800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9674138ce3c8561 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.541+0000 7f780cff9640 1 -- 172.21.15.31:0/2614073054 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7800118860 con 0x7f781010a9e0 2024-09-21T06:16:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.541+0000 7f78057fa640 1 -- 172.21.15.31:0/989700975 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800042400 con 0x564ac6ca41b0 2024-09-21T06:16:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.541+0000 7f78057fa640 1 -- 172.21.15.31:0/989700975 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78000ded90 con 0x564ac6ca41b0 2024-09-21T06:16:52.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.541+0000 7f78057fa640 1 -- 172.21.15.31:0/989700975 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000e3170 con 0x564ac6ca41b0 2024-09-21T06:16:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.542+0000 7f780f7fe640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77fc0b6880 0x7f77fc0b8d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7808005c40 tx=0x7f7808009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.542+0000 7f781b8d3740 1 -- 172.21.15.31:0/989700975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 msgr2=0x564ac7074b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.542+0000 7f781b8d3740 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac7074b90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7800115f90 tx=0x7f78000a2800 comp rx=0 tx=0).stop 2024-09-21T06:16:52.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 -- 172.21.15.31:0/989700975 shutdown_connections 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac70750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac7074b90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 --2- 172.21.15.31:0/989700975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac7074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 -- 172.21.15.31:0/989700975 >> 172.21.15.31:0/989700975 conn(0x564ac6cc1460 msgr2=0x564ac7072540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 -- 172.21.15.31:0/989700975 shutdown_connections 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.543+0000 7f781b8d3740 1 -- 172.21.15.31:0/989700975 wait complete. 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 -- start start 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 Processor -- start 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac707fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac7080230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564ac6e3b700 con 0x564ac6ccd070 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ac6f75110 con 0x564ac6ca41b0 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564ac7082bf0 con 0x564ac6cac390 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.544+0000 7f781b8d3740 10 client.0 _ll_register_callbacks cb 0x564ac6f8e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f780f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f780f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70807b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55924/0 (socket says 172.21.15.31:55924) 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f780f7fe640 1 -- 172.21.15.31:0/2724607731 learned_addr learned my addr 172.21.15.31:0/2724607731 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f7807fff640 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac7080230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f7807fff640 1 -- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 msgr2=0x564ac707fcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f7807fff640 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac707fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f7807fff640 1 -- 172.21.15.31:0/2724607731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 msgr2=0x564ac70807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f7807fff640 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.545+0000 7f7807fff640 1 -- 172.21.15.31:0/2724607731 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac6fa2c40 con 0x564ac6cac390 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f7807fff640 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac7080230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f77f8012700 tx=0x7f77f8012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111693c97e2727ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f77f67fc640 1 -- 172.21.15.31:0/2724607731 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8005020 con 0x564ac6cac390 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78000ded90 con 0x564ac6cac390 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f77f67fc640 1 -- 172.21.15.31:0/2724607731 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f77f80051c0 con 0x564ac6cac390 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f77f67fc640 1 -- 172.21.15.31:0/2724607731 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f77f80053c0 con 0x564ac6cac390 2024-09-21T06:16:52.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f781b8d3740 10 client.26416 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.546+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564ac708e8b0 con 0x564ac6cac390 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.547+0000 7f77f67fc640 1 -- 172.21.15.31:0/2724607731 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78000ded90 con 0x564ac6cac390 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.548+0000 7f77f67fc640 1 -- 172.21.15.31:0/2724607731 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f77f80046c0 con 0x564ac6cac390 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.548+0000 7f781b8d3740 10 client.26416 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.548+0000 7f781b8d3740 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.549+0000 7f781b8d3740 20 client.26416 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.549+0000 7f781b8d3740 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c8b9f0 0x564ac70921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.549+0000 7f781b8d3740 4 client.26416 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.549+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ac6fecda0 con 0x564ac6c8b9f0 2024-09-21T06:16:52.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.549+0000 7f780effd640 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c8b9f0 0x564ac70921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.550+0000 7f77f67fc640 10 client.26416 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.550+0000 7f780effd640 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c8b9f0 0x564ac70921f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0a5345439bc2ba server_cookie=b111caaa49dc979b in_seq=0 out_seq=0 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f77f67fc640 1 -- 172.21.15.31:0/2724607731 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ac6fecda0 con 0x564ac6c8b9f0 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f77f67fc640 10 client.26416 handle_command_reply: tid=0 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 client.26416 shutdown 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 msgr2=0x564ac7080230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac7080230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f77f8012700 tx=0x7f77f8012b00 comp rx=0 tx=0).stop 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 shutdown_connections 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c8b9f0 0x564ac70921f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f77f67fc640 0 client.26416 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6cac390 0x564ac7080230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 --2- 172.21.15.31:0/2724607731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac707fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 >> 172.21.15.31:0/2724607731 conn(0x564ac6cc1460 msgr2=0x564ac70777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 shutdown_connections 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 1 -- 172.21.15.31:0/2724607731 wait complete. 2024-09-21T06:16:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.551+0000 7f781b8d3740 20 client.26416 trim_cache size 0 max 16384 2024-09-21T06:16:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.575+0000 7f781b8d3740 1 Processor -- start 2024-09-21T06:16:52.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.575+0000 7f781b8d3740 1 -- start start 2024-09-21T06:16:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ca41b0 0x564ac70801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 0x564ac7071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f7807fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 0x564ac7071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f7807fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 0x564ac7071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56254/0 (socket says 172.21.15.31:56254) 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f7807fff640 1 -- 172.21.15.31:0/746568972 learned_addr learned my addr 172.21.15.31:0/746568972 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77f000c700 con 0x564ac6ccd070 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77f000ef80 con 0x564ac6cac390 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.576+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77f000cd70 con 0x564ac6ca41b0 2024-09-21T06:16:52.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f780effd640 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ca41b0 0x564ac70801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f780f7fe640 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7807fff640 1 -- 172.21.15.31:0/746568972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ca41b0 msgr2=0x564ac70801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7807fff640 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ca41b0 0x564ac70801b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7807fff640 1 -- 172.21.15.31:0/746568972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 msgr2=0x564ac70720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7807fff640 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7807fff640 1 -- 172.21.15.31:0/746568972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac7073370 con 0x564ac6cac390 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f780f7fe640 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7807fff640 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 0x564ac7071b70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77f800f5c0 tx=0x7f77f8004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f7546709afc504 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7804ff9640 1 -- 172.21.15.31:0/746568972 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8029040 con 0x564ac6cac390 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.577+0000 7f7804ff9640 1 -- 172.21.15.31:0/746568972 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78000ded90 con 0x564ac6cac390 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.578+0000 7f7804ff9640 1 -- 172.21.15.31:0/746568972 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80056a0 con 0x564ac6cac390 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.578+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 msgr2=0x564ac7071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.578+0000 7f781b8d3740 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 0x564ac7071b70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77f800f5c0 tx=0x7f77f8004f10 comp rx=0 tx=0).stop 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 shutdown_connections 2024-09-21T06:16:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6ccd070 0x564ac70720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6cac390 0x564ac7071b70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 --2- 172.21.15.31:0/746568972 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ca41b0 0x564ac70801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 >> 172.21.15.31:0/746568972 conn(0x564ac6cc1460 msgr2=0x564ac707faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 shutdown_connections 2024-09-21T06:16:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 -- 172.21.15.31:0/746568972 wait complete. 2024-09-21T06:16:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 -- start start 2024-09-21T06:16:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.579+0000 7f781b8d3740 1 Processor -- start 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac70885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6cac390 0x564ac7083aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ccd070 0x564ac70841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77f000d440 con 0x564ac6cac390 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f77f000e8b0 con 0x564ac6ca41b0 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77f000db10 con 0x564ac6ccd070 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac70885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f781b8d3740 10 client.0 _ll_register_callbacks cb 0x7f77f8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac70885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56266/0 (socket says 172.21.15.31:56266) 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 -- 172.21.15.31:0/729365954 learned_addr learned my addr 172.21.15.31:0/729365954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780f7fe640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ccd070 0x564ac70841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 -- 172.21.15.31:0/729365954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ccd070 msgr2=0x564ac70841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ccd070 0x564ac70841b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f7807fff640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6cac390 0x564ac7083aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 -- 172.21.15.31:0/729365954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6cac390 msgr2=0x564ac7083aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6cac390 0x564ac7083aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780effd640 1 -- 172.21.15.31:0/729365954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac70fbbf0 con 0x564ac6ca41b0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.580+0000 7f780f7fe640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ccd070 0x564ac70841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.581+0000 7f780effd640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac70885b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f78000def40 tx=0x7f78000da6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28315cd660c74608 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.581+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.581+0000 7f77f67fc640 1 -- 172.21.15.31:0/729365954 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800059040 con 0x564ac6ca41b0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.581+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78000ded90 con 0x564ac6ca41b0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.581+0000 7f77f67fc640 1 -- 172.21.15.31:0/729365954 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f78000a3450 con 0x564ac6ca41b0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.581+0000 7f77f67fc640 1 -- 172.21.15.31:0/729365954 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f78000e3170 con 0x564ac6ca41b0 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 10 client.26378 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564ac70fccc0 con 0x564ac6ca41b0 2024-09-21T06:16:52.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f77f67fc640 1 -- 172.21.15.31:0/729365954 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78000ded90 con 0x564ac6ca41b0 2024-09-21T06:16:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f77f67fc640 1 -- 172.21.15.31:0/729365954 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f7800040b70 con 0x564ac6ca41b0 2024-09-21T06:16:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 10 client.26378 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:52.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 20 client.26378 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c60da0 0x564ac70ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 4 client.26378 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564ac6fe8850 con 0x564ac6c60da0 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.582+0000 7f7807fff640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c60da0 0x564ac70ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.583+0000 7f77f67fc640 10 client.26378 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.583+0000 7f7807fff640 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c60da0 0x564ac70ff240 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e64665109636404 server_cookie=e4cafa7300d8fb10 in_seq=0 out_seq=0 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.583+0000 7f77f67fc640 1 -- 172.21.15.31:0/729365954 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x564ac6fe8850 con 0x564ac6c60da0 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.583+0000 7f77f67fc640 10 client.26378 handle_command_reply: tid=0 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.583+0000 7f781b8d3740 1 client.26378 shutdown 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 msgr2=0x564ac70885b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac70885b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f78000def40 tx=0x7f78000da6f0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 shutdown_connections 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564ac6c60da0 0x564ac70ff240 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f77f67fc640 0 client.26378 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564ac6ccd070 0x564ac70841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564ac6cac390 0x564ac7083aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 --2- 172.21.15.31:0/729365954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ac6ca41b0 0x564ac70885b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 >> 172.21.15.31:0/729365954 conn(0x564ac6cc1460 msgr2=0x564ac7075680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 shutdown_connections 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 1 -- 172.21.15.31:0/729365954 wait complete. 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.584+0000 7f781b8d3740 20 client.26378 trim_cache size 0 max 16384 2024-09-21T06:16:52.589 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.590 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.592 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:16:52.593 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.594 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:16:52.597 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:16:52.599 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:16:52.600 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.585+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77fc0b6880 msgr2=0x7f77fc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.585+0000 7f781535f640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77fc0b6880 0x7f77fc0b8d90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7808005c40 tx=0x7f7808009500 comp rx=0 tx=0).stop 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.585+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 msgr2=0x7f781011c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.585+0000 7f781535f640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781011c3f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f780003ba10 tx=0x7f7800004960 comp rx=0 tx=0).stop 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 shutdown_connections 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f781011cc30 0x7f7810118c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7810111f40 0x7f78101186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f77fc0b6880 0x7f77fc0b8d90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 --2- 172.21.15.31:0/2614073054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f781010a9e0 0x7f781011c3f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 >> 172.21.15.31:0/2614073054 conn(0x7f7810070ec0 msgr2=0x7f781010ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 shutdown_connections 2024-09-21T06:16:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:52.586+0000 7f781535f640 1 -- 172.21.15.31:0/2614073054 wait complete. 2024-09-21T06:16:53.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:53 smithi031 ceph-mon[31237]: pgmap v478: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T06:16:53.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:53 smithi110 ceph-mon[34550]: pgmap v478: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T06:16:53.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:53 smithi107 ceph-mon[34380]: pgmap v478: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-21T06:16:53.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:16:53.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:16:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.358+0000 7f831be94640 1 -- 172.21.15.31:0/1084786752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831410ddc0 msgr2=0x7f8314110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.358+0000 7f831be94640 1 --2- 172.21.15.31:0/1084786752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831410ddc0 0x7f8314110290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f830c009ca0 tx=0x7f830c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.359+0000 7f831be94640 1 -- 172.21.15.31:0/1084786752 shutdown_connections 2024-09-21T06:16:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.359+0000 7f831be94640 1 --2- 172.21.15.31:0/1084786752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f831410ddc0 0x7f8314110290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.359+0000 7f831be94640 1 --2- 172.21.15.31:0/1084786752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83141051f0 0x7f831410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.359+0000 7f831be94640 1 --2- 172.21.15.31:0/1084786752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8314104810 0x7f8314104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.359+0000 7f831be94640 1 -- 172.21.15.31:0/1084786752 >> 172.21.15.31:0/1084786752 conn(0x7f83140fe3e0 msgr2=0x7f8314100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.360+0000 7f831be94640 1 -- 172.21.15.31:0/1084786752 shutdown_connections 2024-09-21T06:16:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.360+0000 7f831be94640 1 -- 172.21.15.31:0/1084786752 wait complete. 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.360+0000 7f831be94640 1 Processor -- start 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.360+0000 7f831be94640 1 -- start start 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831be94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8314104810 0x7f831419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831be94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83141051f0 0x7f831419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831be94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831410ddc0 0x7f831419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831be94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8314113a20 con 0x7f8314104810 2024-09-21T06:16:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831be94640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f83141138a0 con 0x7f83141051f0 2024-09-21T06:16:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831be94640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8314113ba0 con 0x7f831410ddc0 2024-09-21T06:16:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831942f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83141051f0 0x7f831419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f8319c30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8314104810 0x7f831419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831a431640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831410ddc0 0x7f831419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831942f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83141051f0 0x7f831419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56294/0 (socket says 172.21.15.31:56294) 2024-09-21T06:16:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.361+0000 7f831942f640 1 -- 172.21.15.31:0/3606460628 learned_addr learned my addr 172.21.15.31:0/3606460628 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f831a431640 1 -- 172.21.15.31:0/3606460628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83141051f0 msgr2=0x7f831419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f831a431640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83141051f0 0x7f831419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f831a431640 1 -- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8314104810 msgr2=0x7f831419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f831a431640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8314104810 0x7f831419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f831a431640 1 -- 172.21.15.31:0/3606460628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83141a0350 con 0x7f831410ddc0 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f831a431640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831410ddc0 0x7f831419fc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f830c02ade0 tx=0x7f830c031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.362+0000 7f830affd640 1 -- 172.21.15.31:0/3606460628 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c003b70 con 0x7f831410ddc0 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.363+0000 7f830affd640 1 -- 172.21.15.31:0/3606460628 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c031910 con 0x7f831410ddc0 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.363+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83141a05e0 con 0x7f831410ddc0 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.363+0000 7f830affd640 1 -- 172.21.15.31:0/3606460628 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c02f950 con 0x7f831410ddc0 2024-09-21T06:16:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.363+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83141a6dd0 con 0x7f831410ddc0 2024-09-21T06:16:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.366+0000 7f830affd640 1 -- 172.21.15.31:0/3606460628 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f830c02faf0 con 0x7f831410ddc0 2024-09-21T06:16:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.367+0000 7f8322e06740 1 Processor -- start 2024-09-21T06:16:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.367+0000 7f8322e06740 1 -- start start 2024-09-21T06:16:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8322e06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db71dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8322e06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 0x558d6db72300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8322e06740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d7a74d0 0x558d6db72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8322e06740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558d6dae26f0 con 0x558d6d7a74d0 2024-09-21T06:16:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8322e06740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d6dad1b20 con 0x558d6d7c81b0 2024-09-21T06:16:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8322e06740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d6dad5930 con 0x558d6d79f2f0 2024-09-21T06:16:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8319c30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 0x558d6db72300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f830affd640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f83000b6680 0x7f83000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8319c30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 0x558d6db72300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52406/0 (socket says 172.21.15.31:52406) 2024-09-21T06:16:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f8319c30640 1 -- 172.21.15.31:0/3134256069 learned_addr learned my addr 172.21.15.31:0/3134256069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.368+0000 7f831942f640 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db71dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.369+0000 7f830affd640 1 -- 172.21.15.31:0/3606460628 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f830c1087b0 con 0x7f831410ddc0 2024-09-21T06:16:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.369+0000 7f831942f640 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d7a74d0 0x558d6db72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.371+0000 7f8319c30640 1 -- 172.21.15.31:0/3134256069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 msgr2=0x558d6db71dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.371+0000 7f8319c30640 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db71dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.371+0000 7f8319c30640 1 -- 172.21.15.31:0/3134256069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d7a74d0 msgr2=0x558d6db72840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.371+0000 7f8319c30640 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d7a74d0 0x558d6db72840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.371+0000 7f8319c30640 1 -- 172.21.15.31:0/3134256069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6da97530 con 0x558d6d79f2f0 2024-09-21T06:16:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.374+0000 7f8319c30640 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 0x558d6db72300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8310009d50 tx=0x7f8310011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bb11c8e42d0a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.378+0000 7f82fffff640 1 -- 172.21.15.31:0/3134256069 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831001e070 con 0x558d6d79f2f0 2024-09-21T06:16:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.379+0000 7f82fffff640 1 -- 172.21.15.31:0/3134256069 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8310009e40 con 0x558d6d79f2f0 2024-09-21T06:16:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.379+0000 7f82fffff640 1 -- 172.21.15.31:0/3134256069 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000ea80 con 0x558d6d79f2f0 2024-09-21T06:16:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.388+0000 7f8322e06740 1 -- 172.21.15.31:0/3134256069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 msgr2=0x558d6db72300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.388+0000 7f8322e06740 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 0x558d6db72300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8310009d50 tx=0x7f8310011810 comp rx=0 tx=0).stop 2024-09-21T06:16:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.391+0000 7f831a431640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f83000b6680 0x7f83000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.396+0000 7f8322e06740 1 -- 172.21.15.31:0/3134256069 shutdown_connections 2024-09-21T06:16:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.396+0000 7f8322e06740 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d7a74d0 0x558d6db72840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.396+0000 7f8322e06740 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d79f2f0 0x558d6db72300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.396+0000 7f8322e06740 1 --2- 172.21.15.31:0/3134256069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db71dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.396+0000 7f8322e06740 1 -- 172.21.15.31:0/3134256069 >> 172.21.15.31:0/3134256069 conn(0x558d6d7bc300 msgr2=0x558d6db6fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.397+0000 7f831a431640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f83000b6680 0x7f83000b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f830c0ce5d0 tx=0x7f830c02a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.402+0000 7f8322e06740 1 -- 172.21.15.31:0/3134256069 shutdown_connections 2024-09-21T06:16:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.403+0000 7f8322e06740 1 -- 172.21.15.31:0/3134256069 wait complete. 2024-09-21T06:16:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.404+0000 7f8322e06740 1 -- start start 2024-09-21T06:16:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.404+0000 7f8322e06740 1 Processor -- start 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 0x558d6db827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558d6da09ab0 con 0x558d6d79f2f0 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d6da77730 con 0x558d6d7c81b0 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d6db80540 con 0x558d6d7a74d0 2024-09-21T06:16:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8322e06740 10 client.0 _ll_register_callbacks cb 0x558d6dacb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8319c30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 0x558d6db827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8319c30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 0x558d6db827a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52420/0 (socket says 172.21.15.31:52420) 2024-09-21T06:16:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f8319c30640 1 -- 172.21.15.31:0/1126147858 learned_addr learned my addr 172.21.15.31:0/1126147858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.405+0000 7f831942f640 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.409+0000 7f8319c30640 1 -- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 msgr2=0x558d6db877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.409+0000 7f8319c30640 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.409+0000 7f8319c30640 1 -- 172.21.15.31:0/1126147858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 msgr2=0x558d6db821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.409+0000 7f8319c30640 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.409+0000 7f8319c30640 1 -- 172.21.15.31:0/1126147858 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6d833790 con 0x558d6d7a74d0 2024-09-21T06:16:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.411+0000 7f8319c30640 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 0x558d6db827a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8310009710 tx=0x7f8310005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75199e52756dbbb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.411+0000 7f82fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.416+0000 7f82fd7fa640 1 -- 172.21.15.31:0/1126147858 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310026020 con 0x558d6d7a74d0 2024-09-21T06:16:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.416+0000 7f82fd7fa640 1 -- 172.21.15.31:0/1126147858 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f831000ec10 con 0x558d6d7a74d0 2024-09-21T06:16:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.416+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d6dae4640 con 0x558d6d7a74d0 2024-09-21T06:16:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.419+0000 7f82fd7fa640 1 -- 172.21.15.31:0/1126147858 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f831001e040 con 0x558d6d7a74d0 2024-09-21T06:16:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.423+0000 7f8322e06740 10 client.26434 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.423+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558d6d8ac620 con 0x558d6d7a74d0 2024-09-21T06:16:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.424+0000 7f82fd7fa640 1 -- 172.21.15.31:0/1126147858 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558d6dae4640 con 0x558d6d7a74d0 2024-09-21T06:16:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.424+0000 7f82fd7fa640 1 -- 172.21.15.31:0/1126147858 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8310019430 con 0x558d6d7a74d0 2024-09-21T06:16:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f8322e06740 10 client.26434 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f8322e06740 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f8322e06740 20 client.26434 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f8322e06740 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6d786d80 0x558d6dba4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f8322e06740 4 client.26434 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d6dae00f0 con 0x558d6d786d80 2024-09-21T06:16:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.425+0000 7f831942f640 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6d786d80 0x558d6dba4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.427+0000 7f831942f640 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6d786d80 0x558d6dba4900 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f482826d767e19a server_cookie=f95b511f4eec6385 in_seq=0 out_seq=0 2024-09-21T06:16:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.427+0000 7f82fd7fa640 10 client.26434 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.429+0000 7f82fd7fa640 1 -- 172.21.15.31:0/1126147858 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d6dae00f0 con 0x558d6d786d80 2024-09-21T06:16:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.429+0000 7f82fd7fa640 10 client.26434 handle_command_reply: tid=0 2024-09-21T06:16:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.429+0000 7f8322e06740 1 client.26434 shutdown 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 msgr2=0x558d6db827a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 0x558d6db827a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8310009710 tx=0x7f8310005500 comp rx=0 tx=0).stop 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 shutdown_connections 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6d786d80 0x558d6dba4900 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7a74d0 0x558d6db827a0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8310009710 tx=0x7f8310005500 comp rx=0 tx=0).stop 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 --2- 172.21.15.31:0/1126147858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 >> 172.21.15.31:0/1126147858 conn(0x558d6d7bc300 msgr2=0x558d6db72df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f82fd7fa640 0 client.26434 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 shutdown_connections 2024-09-21T06:16:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.430+0000 7f8322e06740 1 -- 172.21.15.31:0/1126147858 wait complete. 2024-09-21T06:16:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.433+0000 7f8322e06740 20 client.26434 trim_cache size 0 max 16384 2024-09-21T06:16:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.462+0000 7f8322e06740 1 Processor -- start 2024-09-21T06:16:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.462+0000 7f8322e06740 1 -- start start 2024-09-21T06:16:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.463+0000 7f8322e06740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.463+0000 7f8322e06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7c81b0 0x558d6db87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.463+0000 7f8322e06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d75c1c0 0x558d6db80810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.463+0000 7f8319c30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7c81b0 0x558d6db87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.463+0000 7f8319c30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7c81b0 0x558d6db87620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52430/0 (socket says 172.21.15.31:52430) 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.463+0000 7f8319c30640 1 -- 172.21.15.31:0/2851281053 learned_addr learned my addr 172.21.15.31:0/2851281053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d75c1c0 0x558d6db80810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8314071390 con 0x558d6d79f2f0 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8314073c10 con 0x558d6d75c1c0 2024-09-21T06:16:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8314071a00 con 0x558d6d7c81b0 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 -- 172.21.15.31:0/2851281053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7c81b0 msgr2=0x558d6db87620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7c81b0 0x558d6db87620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 -- 172.21.15.31:0/2851281053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 msgr2=0x558d6db7f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.464+0000 7f831942f640 1 -- 172.21.15.31:0/2851281053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6db73cf0 con 0x558d6d75c1c0 2024-09-21T06:16:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.465+0000 7f831942f640 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d75c1c0 0x558d6db80810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x558d6dba4e70 tx=0x7f8304013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec21c66a8a4f9ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.465+0000 7f82ff7fe640 1 -- 172.21.15.31:0/2851281053 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304005020 con 0x558d6d75c1c0 2024-09-21T06:16:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.465+0000 7f82ff7fe640 1 -- 172.21.15.31:0/2851281053 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83040047a0 con 0x558d6d75c1c0 2024-09-21T06:16:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.465+0000 7f82ff7fe640 1 -- 172.21.15.31:0/2851281053 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304003910 con 0x558d6d75c1c0 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.466+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d75c1c0 msgr2=0x558d6db80810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.466+0000 7f8322e06740 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d75c1c0 0x558d6db80810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x558d6dba4e70 tx=0x7f8304013040 comp rx=0 tx=0).stop 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 shutdown_connections 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d75c1c0 0x558d6db80810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d7c81b0 0x558d6db87620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 --2- 172.21.15.31:0/2851281053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 >> 172.21.15.31:0/2851281053 conn(0x558d6d7bc300 msgr2=0x558d6db821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 shutdown_connections 2024-09-21T06:16:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.467+0000 7f8322e06740 1 -- 172.21.15.31:0/2851281053 wait complete. 2024-09-21T06:16:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.468+0000 7f8322e06740 1 -- start start 2024-09-21T06:16:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.468+0000 7f8322e06740 1 Processor -- start 2024-09-21T06:16:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.469+0000 7f8322e06740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 0x558d6db7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.469+0000 7f8322e06740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.469+0000 7f8322e06740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db7f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f8319c30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f8319c30640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55982/0 (socket says 172.21.15.31:55982) 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.469+0000 7f831942f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 0x558d6db7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 0x558d6db7e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52438/0 (socket says 172.21.15.31:52438) 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f8319c30640 1 -- 172.21.15.31:0/202226697 learned_addr learned my addr 172.21.15.31:0/202226697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f83140720d0 con 0x558d6d79f2f0 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8314073540 con 0x558d6d7c81b0 2024-09-21T06:16:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83140727a0 con 0x558d6d75c1c0 2024-09-21T06:16:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db7f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 -- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 msgr2=0x558d6db7f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db7f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 -- 172.21.15.31:0/202226697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 msgr2=0x558d6db7edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.470+0000 7f831942f640 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6dc0d240 con 0x558d6d75c1c0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f8322e06740 10 client.0 _ll_register_callbacks cb 0x7f831001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83040047a0 con 0x558d6d75c1c0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f831942f640 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 0x558d6db7e8b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x558d6dba4e70 tx=0x7f8304005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfa0b4eca1172a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f82fd7fa640 10 client.26440 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f82fd7fa640 1 -- 172.21.15.31:0/202226697 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830403d050 con 0x558d6d75c1c0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f82fd7fa640 1 -- 172.21.15.31:0/202226697 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f8304005020 con 0x558d6d75c1c0 2024-09-21T06:16:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.471+0000 7f82fd7fa640 1 -- 172.21.15.31:0/202226697 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8304005220 con 0x558d6d75c1c0 2024-09-21T06:16:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.472+0000 7f8322e06740 10 client.26440 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.472+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x558d6dc0e1b0 con 0x558d6d75c1c0 2024-09-21T06:16:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f82fd7fa640 1 -- 172.21.15.31:0/202226697 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8304025070 con 0x558d6d75c1c0 2024-09-21T06:16:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8322e06740 10 client.26440 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8322e06740 10 client.26440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8322e06740 20 client.26440 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8322e06740 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6dc0e860 0x558d6dc10d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8322e06740 4 client.26440 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:16:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d6dadd520 con 0x558d6dc0e860 2024-09-21T06:16:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.473+0000 7f8319c30640 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6dc0e860 0x558d6dc10d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.475+0000 7f8319c30640 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6dc0e860 0x558d6dc10d70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4200bb7df428b513 server_cookie=884f0555569460ee in_seq=0 out_seq=0 2024-09-21T06:16:54.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.475+0000 7f82fd7fa640 10 client.26440 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.498+0000 7f82fd7fa640 1 -- 172.21.15.31:0/202226697 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d6dadd520 con 0x558d6dc0e860 2024-09-21T06:16:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.498+0000 7f82fd7fa640 10 client.26440 handle_command_reply: tid=0 2024-09-21T06:16:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.499+0000 7f8322e06740 1 client.26440 shutdown 2024-09-21T06:16:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 msgr2=0x558d6db7e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 0x558d6db7e8b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x558d6dba4e70 tx=0x7f8304005d30 comp rx=0 tx=0).stop 2024-09-21T06:16:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 shutdown_connections 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d6dc0e860 0x558d6dc10d70 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d6d7c81b0 0x558d6db7f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f82fd7fa640 0 client.26440 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d6d79f2f0 0x558d6db7edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 --2- 172.21.15.31:0/202226697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d6d75c1c0 0x558d6db7e8b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 >> 172.21.15.31:0/202226697 conn(0x558d6d7bc300 msgr2=0x558d6db80750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 shutdown_connections 2024-09-21T06:16:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 1 -- 172.21.15.31:0/202226697 wait complete. 2024-09-21T06:16:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.500+0000 7f8322e06740 20 client.26440 trim_cache size 0 max 16384 2024-09-21T06:16:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:16:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:16:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:16:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f83000b6680 msgr2=0x7f83000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f83000b6680 0x7f83000b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f830c0ce5d0 tx=0x7f830c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831410ddc0 msgr2=0x7f831419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831410ddc0 0x7f831419fc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f830c02ade0 tx=0x7f830c031660 comp rx=0 tx=0).stop 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 shutdown_connections 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831410ddc0 0x7f831419fc50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83141051f0 0x7f831419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f83000b6680 0x7f83000b8b90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 --2- 172.21.15.31:0/3606460628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8314104810 0x7f831419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 >> 172.21.15.31:0/3606460628 conn(0x7f83140fe3e0 msgr2=0x7f8314107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.501+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 shutdown_connections 2024-09-21T06:16:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:16:54.502+0000 7f831be94640 1 -- 172.21.15.31:0/3606460628 wait complete. 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 -- 172.21.15.31:0/1501755338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381027a0 msgr2=0x7f5938102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 --2- 172.21.15.31:0/1501755338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381027a0 0x7f5938102bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f592c009860 tx=0x7f592c02a340 comp rx=0 tx=0).stop 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 -- 172.21.15.31:0/1501755338 shutdown_connections 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 --2- 172.21.15.31:0/1501755338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5938107be0 0x7f59381080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 --2- 172.21.15.31:0/1501755338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59381030f0 0x7f5938107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 --2- 172.21.15.31:0/1501755338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381027a0 0x7f5938102bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.728+0000 7f593ee44640 1 -- 172.21.15.31:0/1501755338 >> 172.21.15.31:0/1501755338 conn(0x7f5938079730 msgr2=0x7f5938079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.729+0000 7f593ee44640 1 -- 172.21.15.31:0/1501755338 shutdown_connections 2024-09-21T06:16:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.729+0000 7f593ee44640 1 -- 172.21.15.31:0/1501755338 wait complete. 2024-09-21T06:16:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.729+0000 7f593ee44640 1 Processor -- start 2024-09-21T06:16:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 -- start start 2024-09-21T06:16:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 0x7f59381a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381030f0 0x7f59381a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5938107be0 0x7f59381a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f593811a100 con 0x7f59381027a0 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5938119f80 con 0x7f5938107be0 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593ee44640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f593811a280 con 0x7f59381030f0 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593cbe0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 0x7f59381a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593cbe0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 0x7f59381a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55988/0 (socket says 172.21.15.31:55988) 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593cbe0640 1 -- 172.21.15.31:0/3733107782 learned_addr learned my addr 172.21.15.31:0/3733107782 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f5937fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381030f0 0x7f59381a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593d3e1640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5938107be0 0x7f59381a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593d3e1640 1 -- 172.21.15.31:0/3733107782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381030f0 msgr2=0x7f59381a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593d3e1640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381030f0 0x7f59381a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593d3e1640 1 -- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 msgr2=0x7f59381a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:54.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593d3e1640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 0x7f59381a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:54.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593d3e1640 1 -- 172.21.15.31:0/3733107782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59381a6aa0 con 0x7f5938107be0 2024-09-21T06:16:54.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.730+0000 7f593cbe0640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 0x7f59381a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:54.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.732+0000 7f593d3e1640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5938107be0 0x7f59381a6370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5930009fd0 tx=0x7f593000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.733+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593000cd60 con 0x7f5938107be0 2024-09-21T06:16:54.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.733+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5930005020 con 0x7f5938107be0 2024-09-21T06:16:54.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.733+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930010690 con 0x7f5938107be0 2024-09-21T06:16:54.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.733+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59381a6d90 con 0x7f5938107be0 2024-09-21T06:16:54.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.733+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59381ad4f0 con 0x7f5938107be0 2024-09-21T06:16:54.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.735+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5938107620 con 0x7f5938107be0 2024-09-21T06:16:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.736+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5930004b20 con 0x7f5938107be0 2024-09-21T06:16:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.737+0000 7f5935ffb640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59280b6880 0x7f59280b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.738+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f59300e7ce0 con 0x7f5938107be0 2024-09-21T06:16:54.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.738+0000 7f593cbe0640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59280b6880 0x7f59280b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:54.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.739+0000 7f593cbe0640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59280b6880 0x7f59280b8d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f592c02a930 tx=0x7f592c02a840 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:54.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:54.742+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f59300ae210 con 0x7f5938107be0 2024-09-21T06:16:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.127+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59381ad6c0 con 0x7f5938107be0 2024-09-21T06:16:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.129+0000 7f5935ffb640 1 -- 172.21.15.31:0/3733107782 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f59300b25f0 con 0x7f5938107be0 2024-09-21T06:16:55.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:16:55.131 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:16:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:16:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59280b6880 msgr2=0x7f59280b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59280b6880 0x7f59280b8d90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f592c02a930 tx=0x7f592c02a840 comp rx=0 tx=0).stop 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5938107be0 msgr2=0x7f59381a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5938107be0 0x7f59381a6370 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5930009fd0 tx=0x7f593000ee50 comp rx=0 tx=0).stop 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 shutdown_connections 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5938107be0 0x7f59381a6370 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59381030f0 0x7f59381a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f59280b6880 0x7f59280b8d90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 --2- 172.21.15.31:0/3733107782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59381027a0 0x7f59381a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 >> 172.21.15.31:0/3733107782 conn(0x7f5938079730 msgr2=0x7f59381055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 shutdown_connections 2024-09-21T06:16:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:55.132+0000 7f593ee44640 1 -- 172.21.15.31:0/3733107782 wait complete. 2024-09-21T06:16:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 315.0 s, 2133.8 tps, lat 46.724 ms stddev 51.659 2024-09-21T06:16:55.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:55 smithi031 ceph-mon[31237]: pgmap v479: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 781 op/s 2024-09-21T06:16:55.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:55 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3733107782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:55.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:55 smithi110 ceph-mon[34550]: pgmap v479: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 781 op/s 2024-09-21T06:16:55.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:55 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3733107782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:55 smithi107 ceph-mon[34380]: pgmap v479: 97 pgs: 97 active+clean; 3.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 781 op/s 2024-09-21T06:16:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:55 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3733107782' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:16:55.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:16:56.353 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:16:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.222+0000 7f60d797b640 1 -- 172.21.15.31:0/2083954172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 msgr2=0x7f60d010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.222+0000 7f60d797b640 1 --2- 172.21.15.31:0/2083954172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 0x7f60d010f520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009ca0 tx=0x7f60c802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 -- 172.21.15.31:0/2083954172 shutdown_connections 2024-09-21T06:16:57.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 --2- 172.21.15.31:0/2083954172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 0x7f60d010f520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 --2- 172.21.15.31:0/2083954172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 --2- 172.21.15.31:0/2083954172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d0102a60 0x7f60d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 -- 172.21.15.31:0/2083954172 >> 172.21.15.31:0/2083954172 conn(0x7f60d0079730 msgr2=0x7f60d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 -- 172.21.15.31:0/2083954172 shutdown_connections 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.223+0000 7f60d797b640 1 -- 172.21.15.31:0/2083954172 wait complete. 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.224+0000 7f60d797b640 1 Processor -- start 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.224+0000 7f60d797b640 1 -- start start 2024-09-21T06:16:57.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.224+0000 7f60d797b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d0102a60 0x7f60d01a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.224+0000 7f60d797b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d797b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 0x7f60d01a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d797b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113a80 con 0x7f60d0102a60 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d797b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113900 con 0x7f60d01083a0 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d797b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113c00 con 0x7f60d0108cf0 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d5717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d0102a60 0x7f60d01a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56362/0 (socket says 172.21.15.31:56362) 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 -- 172.21.15.31:0/2035920837 learned_addr learned my addr 172.21.15.31:0/2035920837 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d5f18640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 0x7f60d01a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 -- 172.21.15.31:0/2035920837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 msgr2=0x7f60d01a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 0x7f60d01a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 -- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d0102a60 msgr2=0x7f60d01a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d0102a60 0x7f60d01a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.225+0000 7f60d4f16640 1 -- 172.21.15.31:0/2035920837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d01aca30 con 0x7f60d01083a0 2024-09-21T06:16:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.226+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01a1c60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60c000e9a0 tx=0x7f60c000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:57.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.226+0000 7f60c67fc640 1 -- 172.21.15.31:0/2035920837 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c000cd80 con 0x7f60d01083a0 2024-09-21T06:16:57.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.226+0000 7f60c67fc640 1 -- 172.21.15.31:0/2035920837 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c00035f0 con 0x7f60d01083a0 2024-09-21T06:16:57.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.226+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01acd20 con 0x7f60d01083a0 2024-09-21T06:16:57.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.226+0000 7f60c67fc640 1 -- 172.21.15.31:0/2035920837 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0010690 con 0x7f60d01083a0 2024-09-21T06:16:57.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.226+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01ad1a0 con 0x7f60d01083a0 2024-09-21T06:16:57.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.230+0000 7f60c67fc640 1 -- 172.21.15.31:0/2035920837 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f60c0010830 con 0x7f60d01083a0 2024-09-21T06:16:57.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.231+0000 7f60c67fc640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60bc0b6730 0x7f60bc0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.231+0000 7f60d5717640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60bc0b6730 0x7f60bc0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.231+0000 7f60c67fc640 1 -- 172.21.15.31:0/2035920837 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60c00e8850 con 0x7f60d01083a0 2024-09-21T06:16:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60d5717640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60bc0b6730 0x7f60bc0b8c40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f60d010f570 tx=0x7f60cc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 Processor -- start 2024-09-21T06:16:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 -- start start 2024-09-21T06:16:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73de070 0x55b2d77852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7785800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7785d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b2d76fbea0 con 0x55b2d73b51b0 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2d76eb2d0 con 0x55b2d73de070 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60ddeef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b2d76ef0e0 con 0x55b2d73bd390 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60d5f18640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7785d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60d5717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73de070 0x55b2d77852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60d4f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7785800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60d5f18640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7785d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52468/0 (socket says 172.21.15.31:52468) 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.232+0000 7f60d5f18640 1 -- 172.21.15.31:0/2860131015 learned_addr learned my addr 172.21.15.31:0/2860131015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:57.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5f18640 1 -- 172.21.15.31:0/2860131015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73de070 msgr2=0x55b2d77852c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5f18640 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73de070 0x55b2d77852c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5f18640 1 -- 172.21.15.31:0/2860131015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 msgr2=0x55b2d7785800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5f18640 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7785800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5f18640 1 -- 172.21.15.31:0/2860131015 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d7449650 con 0x55b2d73bd390 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5717640 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73de070 0x55b2d77852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60d5f18640 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7785d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009ca0 tx=0x7f60c8009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5454758198c12b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60bb7fe640 1 -- 172.21.15.31:0/2860131015 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c8031cf0 con 0x55b2d73bd390 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60bb7fe640 1 -- 172.21.15.31:0/2860131015 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c8036030 con 0x55b2d73bd390 2024-09-21T06:16:57.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.233+0000 7f60bb7fe640 1 -- 172.21.15.31:0/2860131015 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c80042a0 con 0x55b2d73bd390 2024-09-21T06:16:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.234+0000 7f60ddeef740 1 -- 172.21.15.31:0/2860131015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 msgr2=0x55b2d7785d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.234+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7785d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009ca0 tx=0x7f60c8009800 comp rx=0 tx=0).stop 2024-09-21T06:16:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.234+0000 7f60ddeef740 1 -- 172.21.15.31:0/2860131015 shutdown_connections 2024-09-21T06:16:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.234+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7785d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.234+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7785800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.235+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2860131015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73de070 0x55b2d77852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.235+0000 7f60ddeef740 1 -- 172.21.15.31:0/2860131015 >> 172.21.15.31:0/2860131015 conn(0x55b2d73d2460 msgr2=0x55b2d77831b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.235+0000 7f60ddeef740 1 -- 172.21.15.31:0/2860131015 shutdown_connections 2024-09-21T06:16:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.235+0000 7f60ddeef740 1 -- 172.21.15.31:0/2860131015 wait complete. 2024-09-21T06:16:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.235+0000 7f60ddeef740 1 -- start start 2024-09-21T06:16:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.235+0000 7f60ddeef740 1 Processor -- start 2024-09-21T06:16:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73b51b0 0x55b2d77908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7790e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73de070 0x55b2d7791360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b2d754c700 con 0x55b2d73de070 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b2d7686160 con 0x55b2d73b51b0 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b2d77938c0 con 0x55b2d73bd390 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7790e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7790e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52484/0 (socket says 172.21.15.31:52484) 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 -- 172.21.15.31:0/1760604078 learned_addr learned my addr 172.21.15.31:0/1760604078 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d5717640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73b51b0 0x55b2d77908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60ddeef740 10 client.0 _ll_register_callbacks cb 0x55b2d769f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d5f18640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73de070 0x55b2d7791360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 -- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73b51b0 msgr2=0x55b2d77908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73b51b0 0x55b2d77908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 -- 172.21.15.31:0/1760604078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73de070 msgr2=0x55b2d7791360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73de070 0x55b2d7791360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.236+0000 7f60d4f16640 1 -- 172.21.15.31:0/1760604078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d76b3c90 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60d4f16640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7790e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60c00ae380 tx=0x7f60c00adcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740f4c35e06d68e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60b8ff9640 1 -- 172.21.15.31:0/1760604078 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00ae190 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60b8ff9640 1 -- 172.21.15.31:0/1760604078 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60c00b7040 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c8036030 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60b8ff9640 1 -- 172.21.15.31:0/1760604078 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f60c00aed80 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60ddeef740 10 client.26458 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b2d779f510 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60b8ff9640 1 -- 172.21.15.31:0/1760604078 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c00b3160 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.237+0000 7f60b8ff9640 1 -- 172.21.15.31:0/1760604078 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f60c00b42f0 con 0x55b2d73bd390 2024-09-21T06:16:57.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60ddeef740 10 client.26458 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60ddeef740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60ddeef740 20 client.26458 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60ddeef740 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d739c9f0 0x55b2d77a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60ddeef740 4 client.26458 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f60c8036030 con 0x55b2d739c9f0 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60d5717640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d739c9f0 0x55b2d77a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60b8ff9640 10 client.26458 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.238+0000 7f60d5717640 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d739c9f0 0x55b2d77a2a60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97059387825756b server_cookie=bd55b121946d6fe in_seq=0 out_seq=0 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.239+0000 7f60b8ff9640 1 -- 172.21.15.31:0/1760604078 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f60c8036030 con 0x55b2d739c9f0 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.239+0000 7f60b8ff9640 10 client.26458 handle_command_reply: tid=0 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.239+0000 7f60ddeef740 1 client.26458 shutdown 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.239+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 msgr2=0x55b2d7790e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.239+0000 7f60ddeef740 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7790e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60c00ae380 tx=0x7f60c00adcb0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.239+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 shutdown_connections 2024-09-21T06:16:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d739c9f0 0x55b2d77a2a60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73de070 0x55b2d7791360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60b8ff9640 0 client.26458 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7790e20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 --2- 172.21.15.31:0/1760604078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73b51b0 0x55b2d77908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 >> 172.21.15.31:0/1760604078 conn(0x55b2d73d2460 msgr2=0x55b2d7788450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 shutdown_connections 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 1 -- 172.21.15.31:0/1760604078 wait complete. 2024-09-21T06:16:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.240+0000 7f60ddeef740 20 client.26458 trim_cache size 0 max 16384 2024-09-21T06:16:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60ddeef740 1 Processor -- start 2024-09-21T06:16:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60ddeef740 1 -- start start 2024-09-21T06:16:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d739c9f0 0x55b2d77928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73b51b0 0x55b2d7782760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60d5717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d739c9f0 0x55b2d77928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60d5717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d739c9f0 0x55b2d77928d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56044/0 (socket says 172.21.15.31:56044) 2024-09-21T06:16:57.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.260+0000 7f60d5717640 1 -- 172.21.15.31:0/651591363 learned_addr learned my addr 172.21.15.31:0/651591363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:57.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60ddeef740 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73bd390 0x55b2d7782ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400c6d0 con 0x55b2d739c9f0 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400ef50 con 0x55b2d73bd390 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400cd40 con 0x55b2d73b51b0 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d4f16640 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73b51b0 0x55b2d7782760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d5f18640 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73bd390 0x55b2d7782ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d4f16640 1 -- 172.21.15.31:0/651591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73bd390 msgr2=0x55b2d7782ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d4f16640 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73bd390 0x55b2d7782ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d4f16640 1 -- 172.21.15.31:0/651591363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d739c9f0 msgr2=0x55b2d77928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d4f16640 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d739c9f0 0x55b2d77928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.261+0000 7f60d4f16640 1 -- 172.21.15.31:0/651591363 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d77875b0 con 0x55b2d73b51b0 2024-09-21T06:16:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.262+0000 7f60d4f16640 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73b51b0 0x55b2d7782760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60c00ae610 tx=0x7f60c0009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f7bf5868894e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.262+0000 7f60baffd640 1 -- 172.21.15.31:0/651591363 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00aa520 con 0x55b2d73b51b0 2024-09-21T06:16:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.262+0000 7f60baffd640 1 -- 172.21.15.31:0/651591363 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c00ae190 con 0x55b2d73b51b0 2024-09-21T06:16:57.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.262+0000 7f60baffd640 1 -- 172.21.15.31:0/651591363 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00b3160 con 0x55b2d73b51b0 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73b51b0 msgr2=0x55b2d7782760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73b51b0 0x55b2d7782760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60c00ae610 tx=0x7f60c0009850 comp rx=0 tx=0).stop 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 shutdown_connections 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d73bd390 0x55b2d7782ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73b51b0 0x55b2d7782760 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 --2- 172.21.15.31:0/651591363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d739c9f0 0x55b2d77928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 >> 172.21.15.31:0/651591363 conn(0x55b2d73d2460 msgr2=0x55b2d7786cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 shutdown_connections 2024-09-21T06:16:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.264+0000 7f60ddeef740 1 -- 172.21.15.31:0/651591363 wait complete. 2024-09-21T06:16:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.265+0000 7f60ddeef740 1 -- start start 2024-09-21T06:16:57.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.265+0000 7f60ddeef740 1 Processor -- start 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 0x55b2d7794ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7794ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7795530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400d410 con 0x55b2d73b51b0 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400e880 con 0x55b2d739c9f0 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400dae0 con 0x55b2d73bd390 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60ddeef740 10 client.0 _ll_register_callbacks cb 0x7f60c000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:16:57.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60d5717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 0x55b2d7794ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60d5717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 0x55b2d7794ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56400/0 (socket says 172.21.15.31:56400) 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60d5717640 1 -- 172.21.15.31:0/2053524080 learned_addr learned my addr 172.21.15.31:0/2053524080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60d5f18640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7795530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.266+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7794ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5717640 1 -- 172.21.15.31:0/2053524080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 msgr2=0x55b2d7795530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5717640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7795530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5717640 1 -- 172.21.15.31:0/2053524080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 msgr2=0x55b2d7794ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5717640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7794ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5717640 1 -- 172.21.15.31:0/2053524080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d780c460 con 0x55b2d739c9f0 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5f18640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7795530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7794ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60d5717640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 0x55b2d7794ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e8c0 tx=0x7f60cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6136baa89b57da2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.267+0000 7f60b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:16:57.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60b8ff9640 1 -- 172.21.15.31:0/2053524080 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc034020 con 0x55b2d739c9f0 2024-09-21T06:16:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60b8ff9640 1 -- 172.21.15.31:0/2053524080 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60cc01d9a0 con 0x55b2d739c9f0 2024-09-21T06:16:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60c00ae190 con 0x55b2d739c9f0 2024-09-21T06:16:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60b8ff9640 1 -- 172.21.15.31:0/2053524080 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f60cc03f050 con 0x55b2d739c9f0 2024-09-21T06:16:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 10 client.26420 fetch_fsmap learned FSMap version 19 2024-09-21T06:16:57.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b2d780d6d0 con 0x55b2d739c9f0 2024-09-21T06:16:57.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60b8ff9640 1 -- 172.21.15.31:0/2053524080 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c00ae190 con 0x55b2d739c9f0 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60b8ff9640 1 -- 172.21.15.31:0/2053524080 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f60cc02a400 con 0x55b2d739c9f0 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 10 client.26420 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 20 client.26420 populate_metadata read hostname 'smithi031' 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d73de070 0x55b2d780fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 4 client.26420 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.268+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2d76fddf0 con 0x55b2d73de070 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d73de070 0x55b2d780fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60b8ff9640 10 client.26420 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60d4f16640 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d73de070 0x55b2d780fef0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d6dbaa4a86dcaa server_cookie=34351ffdc6baf246 in_seq=0 out_seq=0 2024-09-21T06:16:57.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60b8ff9640 1 -- 172.21.15.31:0/2053524080 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2d76fddf0 con 0x55b2d73de070 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60b8ff9640 10 client.26420 handle_command_reply: tid=0 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60ddeef740 1 client.26420 shutdown 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 msgr2=0x55b2d7794ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.269+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 0x55b2d7794ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00e8c0 tx=0x7f60cc004730 comp rx=0 tx=0).stop 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 shutdown_connections 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b2d73de070 0x55b2d780fef0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b2d73bd390 0x55b2d7795530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2d73b51b0 0x55b2d7794ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 --2- 172.21.15.31:0/2053524080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d739c9f0 0x55b2d7794ab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 >> 172.21.15.31:0/2053524080 conn(0x55b2d73d2460 msgr2=0x55b2d7786cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:57.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60b8ff9640 0 client.26420 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:16:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 shutdown_connections 2024-09-21T06:16:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 1 -- 172.21.15.31:0/2053524080 wait complete. 2024-09-21T06:16:57.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.270+0000 7f60ddeef740 20 client.26420 trim_cache size 0 max 16384 2024-09-21T06:16:57.276 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.271+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60bc0b6730 msgr2=0x7f60bc0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.271+0000 7f60d797b640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60bc0b6730 0x7f60bc0b8c40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f60d010f570 tx=0x7f60cc005f90 comp rx=0 tx=0).stop 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.271+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 msgr2=0x7f60d01a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01a1c60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60c000e9a0 tx=0x7f60c000ee60 comp rx=0 tx=0).stop 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 shutdown_connections 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60d0108cf0 0x7f60d01a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60d01083a0 0x7f60d01a1c60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60bc0b6730 0x7f60bc0b8c40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 --2- 172.21.15.31:0/2035920837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60d0102a60 0x7f60d01a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:16:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.272+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 >> 172.21.15.31:0/2035920837 conn(0x7f60d0079730 msgr2=0x7f60d0104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:16:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.273+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 shutdown_connections 2024-09-21T06:16:57.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:16:57.273+0000 7f60d797b640 1 -- 172.21.15.31:0/2035920837 wait complete. 2024-09-21T06:16:57.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:57 smithi031 ceph-mon[31237]: pgmap v480: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-09-21T06:16:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:57 smithi110 ceph-mon[34550]: pgmap v480: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-09-21T06:16:57.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:57 smithi107 ceph-mon[34380]: pgmap v480: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 16 MiB/s wr, 1.07k op/s 2024-09-21T06:16:59.396 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:16:59.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:16:59.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:16:59 smithi110 ceph-mon[34550]: pgmap v481: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 14 MiB/s wr, 815 op/s 2024-09-21T06:16:59.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:16:59 smithi107 ceph-mon[34380]: pgmap v481: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 14 MiB/s wr, 815 op/s 2024-09-21T06:16:59.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:16:59 smithi031 ceph-mon[31237]: pgmap v481: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 14 MiB/s wr, 815 op/s 2024-09-21T06:16:59.811 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 320.0 s, 2933.8 tps, lat 34.196 ms stddev 39.886 2024-09-21T06:17:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 -- 172.21.15.31:0/798834690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 msgr2=0x7ff42410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 --2- 172.21.15.31:0/798834690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff42410a3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009ca0 tx=0x7ff41c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 -- 172.21.15.31:0/798834690 shutdown_connections 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 --2- 172.21.15.31:0/798834690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff42411cc30 0x7ff42411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 --2- 172.21.15.31:0/798834690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff42410a9e0 0x7ff42410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 --2- 172.21.15.31:0/798834690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff42410a3c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 -- 172.21.15.31:0/798834690 >> 172.21.15.31:0/798834690 conn(0x7ff424070ec0 msgr2=0x7ff4240712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 -- 172.21.15.31:0/798834690 shutdown_connections 2024-09-21T06:17:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.411+0000 7ff4291d7640 1 -- 172.21.15.31:0/798834690 wait complete. 2024-09-21T06:17:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.415+0000 7ff4291d7640 1 Processor -- start 2024-09-21T06:17:00.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.415+0000 7ff4291d7640 1 -- start start 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.415+0000 7ff4291d7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff42410a9e0 0x7ff4241a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.415+0000 7ff4291d7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff4241a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.415+0000 7ff4291d7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff42411cc30 0x7ff4241aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff4291d7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241228b0 con 0x7ff42410a9e0 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff4291d7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff424122730 con 0x7ff42411cc30 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff4291d7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff424122a30 con 0x7ff424111f40 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff4241a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff42411cc30 0x7ff4241aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff4241a9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42586/0 (socket says 172.21.15.31:42586) 2024-09-21T06:17:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 -- 172.21.15.31:0/1636310109 learned_addr learned my addr 172.21.15.31:0/1636310109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff422d9d640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff42410a9e0 0x7ff4241a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 -- 172.21.15.31:0/1636310109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff42411cc30 msgr2=0x7ff4241aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff42411cc30 0x7ff4241aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 -- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff42410a9e0 msgr2=0x7ff4241a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff42410a9e0 0x7ff4241a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.416+0000 7ff42259c640 1 -- 172.21.15.31:0/1636310109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4241af250 con 0x7ff424111f40 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.417+0000 7ff42259c640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff4241a9880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff418005fd0 tx=0x7ff41800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.417+0000 7ff40bfff640 1 -- 172.21.15.31:0/1636310109 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418017070 con 0x7ff424111f40 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.417+0000 7ff40bfff640 1 -- 172.21.15.31:0/1636310109 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff418009c30 con 0x7ff424111f40 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.417+0000 7ff40bfff640 1 -- 172.21.15.31:0/1636310109 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418003d10 con 0x7ff424111f40 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.417+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4241af540 con 0x7ff424111f40 2024-09-21T06:17:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.417+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4241b5ca0 con 0x7ff424111f40 2024-09-21T06:17:00.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.420+0000 7ff40bfff640 1 -- 172.21.15.31:0/1636310109 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff418003eb0 con 0x7ff424111f40 2024-09-21T06:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.421+0000 7ff40bfff640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff40c0b67d0 0x7ff40c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.421+0000 7ff42f74b740 1 Processor -- start 2024-09-21T06:17:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.421+0000 7ff40bfff640 1 -- 172.21.15.31:0/1636310109 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff4180e7200 con 0x7ff424111f40 2024-09-21T06:17:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 -- start start 2024-09-21T06:17:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff422d9d640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff40c0b67d0 0x7ff40c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931ca350 0x561c93571430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93571970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931a9670 0x561c93571eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561c934e7e60 con 0x561c931ca350 2024-09-21T06:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c934d7290 con 0x561c931a9670 2024-09-21T06:17:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.422+0000 7ff42f74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c934db0a0 con 0x561c931a1490 2024-09-21T06:17:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.423+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931ca350 0x561c93571430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.423+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931ca350 0x561c93571430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38632/0 (socket says 172.21.15.31:38632) 2024-09-21T06:17:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.423+0000 7ff422d9d640 1 -- 172.21.15.31:0/1280185119 learned_addr learned my addr 172.21.15.31:0/1280185119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.424+0000 7ff42259c640 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93571970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.424+0000 7ff42359e640 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931a9670 0x561c93571eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.424+0000 7ff422d9d640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff40c0b67d0 0x7ff40c0b8ce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0094e0 tx=0x7ff41c0098c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff42259c640 1 -- 172.21.15.31:0/1280185119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931a9670 msgr2=0x561c93571eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff42259c640 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931a9670 0x561c93571eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff42259c640 1 -- 172.21.15.31:0/1280185119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931ca350 msgr2=0x561c93571430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff42259c640 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931ca350 0x561c93571430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff42259c640 1 -- 172.21.15.31:0/1280185119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c93235930 con 0x561c931a1490 2024-09-21T06:17:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff42259c640 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93571970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4180e4990 tx=0x7ff4180ac920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e9bf75b68ab1d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff4097fa640 1 -- 172.21.15.31:0/1280185119 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180ad790 con 0x561c931a1490 2024-09-21T06:17:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff4097fa640 1 -- 172.21.15.31:0/1280185119 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4180b1b70 con 0x561c931a1490 2024-09-21T06:17:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.425+0000 7ff4097fa640 1 -- 172.21.15.31:0/1280185119 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180abd60 con 0x561c931a1490 2024-09-21T06:17:00.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 -- 172.21.15.31:0/1280185119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 msgr2=0x561c93571970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93571970 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff4180e4990 tx=0x7ff4180ac920 comp rx=0 tx=0).stop 2024-09-21T06:17:00.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 -- 172.21.15.31:0/1280185119 shutdown_connections 2024-09-21T06:17:00.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931a9670 0x561c93571eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93571970 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1280185119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931ca350 0x561c93571430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 -- 172.21.15.31:0/1280185119 >> 172.21.15.31:0/1280185119 conn(0x561c931be740 msgr2=0x561c9356f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 -- 172.21.15.31:0/1280185119 shutdown_connections 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.426+0000 7ff42f74b740 1 -- 172.21.15.31:0/1280185119 wait complete. 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 -- start start 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 Processor -- start 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c9357d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931ca350 0x561c93588ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561c93338890 con 0x561c931a9670 2024-09-21T06:17:00.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c93471d40 con 0x561c931ca350 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561c9357fbb0 con 0x561c931a1490 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.427+0000 7ff42f74b740 10 client.0 _ll_register_callbacks cb 0x561c934d5c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c9357d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c9357d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42594/0 (socket says 172.21.15.31:42594) 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 -- 172.21.15.31:0/3340790136 learned_addr learned my addr 172.21.15.31:0/3340790136 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff42359e640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931ca350 0x561c93588ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff42259c640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 -- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931ca350 msgr2=0x561c93588ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931ca350 0x561c93588ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 -- 172.21.15.31:0/3340790136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 msgr2=0x561c9357d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 -- 172.21.15.31:0/3340790136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c935894b0 con 0x561c931a1490 2024-09-21T06:17:00.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff42259c640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff422d9d640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c9357d020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005bf0 tx=0x7ff41c02a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f158178fd9ce20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.428+0000 7ff406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.429+0000 7ff406ff5640 1 -- 172.21.15.31:0/3340790136 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c042040 con 0x561c931a1490 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.429+0000 7ff406ff5640 1 -- 172.21.15.31:0/3340790136 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff41c004040 con 0x561c931a1490 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.429+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4180b1b70 con 0x561c931a1490 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.430+0000 7ff406ff5640 1 -- 172.21.15.31:0/3340790136 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff41c003e40 con 0x561c931a1490 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.430+0000 7ff42f74b740 10 client.26476 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.430+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561c9358bde0 con 0x561c931a1490 2024-09-21T06:17:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.430+0000 7ff406ff5640 1 -- 172.21.15.31:0/3340790136 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c044070 con 0x561c931a1490 2024-09-21T06:17:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff406ff5640 1 -- 172.21.15.31:0/3340790136 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff41c031ac0 con 0x561c931a1490 2024-09-21T06:17:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42f74b740 10 client.26476 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42f74b740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42f74b740 20 client.26476 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:00.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42f74b740 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c93188cd0 0x561c9358f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42f74b740 4 client.26476 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4180b1b70 con 0x561c93188cd0 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42259c640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c93188cd0 0x561c9358f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff406ff5640 10 client.26476 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.431+0000 7ff42259c640 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c93188cd0 0x561c9358f2f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10447e28405b838c server_cookie=9943359531db6678 in_seq=0 out_seq=0 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff406ff5640 1 -- 172.21.15.31:0/3340790136 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff4180b1b70 con 0x561c93188cd0 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff406ff5640 10 client.26476 handle_command_reply: tid=0 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 client.26476 shutdown 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 msgr2=0x561c9357d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c9357d020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005bf0 tx=0x7ff41c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 shutdown_connections 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c93188cd0 0x561c9358f2f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c931ca350 0x561c93588ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff406ff5640 0 client.26476 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 --2- 172.21.15.31:0/3340790136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c9357d020 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.432+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 >> 172.21.15.31:0/3340790136 conn(0x561c931be740 msgr2=0x561c93574610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.433+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 shutdown_connections 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.433+0000 7ff42f74b740 1 -- 172.21.15.31:0/3340790136 wait complete. 2024-09-21T06:17:00.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.433+0000 7ff42f74b740 20 client.26476 trim_cache size 0 max 16384 2024-09-21T06:17:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 Processor -- start 2024-09-21T06:17:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 -- start start 2024-09-21T06:17:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93576510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93576a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b72e0 con 0x561c931a9670 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b6540 con 0x561c93188cd0 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.456+0000 7ff42f74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff424119040 con 0x561c931a1490 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93576510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff42259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93576a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93576510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60878/0 (socket says 172.21.15.31:60878) 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 -- 172.21.15.31:0/1689232820 learned_addr learned my addr 172.21.15.31:0/1689232820 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff42259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93576a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42598/0 (socket says 172.21.15.31:42598) 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff42359e640 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 -- 172.21.15.31:0/1689232820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 msgr2=0x561c93576a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93576a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 -- 172.21.15.31:0/1689232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 msgr2=0x561c9357a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 -- 172.21.15.31:0/1689232820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9357bd80 con 0x561c93188cd0 2024-09-21T06:17:00.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff422d9d640 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93576510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff41c02add0 tx=0x7ff41c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa993c51ba43b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:00.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.457+0000 7ff408ff9640 1 -- 172.21.15.31:0/1689232820 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c042070 con 0x561c93188cd0 2024-09-21T06:17:00.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff408ff9640 1 -- 172.21.15.31:0/1689232820 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c071070 con 0x561c93188cd0 2024-09-21T06:17:00.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff408ff9640 1 -- 172.21.15.31:0/1689232820 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c031820 con 0x561c93188cd0 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 -- 172.21.15.31:0/1689232820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 msgr2=0x561c93576510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93576510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff41c02add0 tx=0x7ff41c000f40 comp rx=0 tx=0).stop 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 -- 172.21.15.31:0/1689232820 shutdown_connections 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a9670 0x561c9357a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a1490 0x561c93576a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 --2- 172.21.15.31:0/1689232820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93576510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.458+0000 7ff42f74b740 1 -- 172.21.15.31:0/1689232820 >> 172.21.15.31:0/1689232820 conn(0x561c931be740 msgr2=0x561c93577fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.459+0000 7ff42f74b740 1 -- 172.21.15.31:0/1689232820 shutdown_connections 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.459+0000 7ff42f74b740 1 -- 172.21.15.31:0/1689232820 wait complete. 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.459+0000 7ff42f74b740 1 -- start start 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.459+0000 7ff42f74b740 1 Processor -- start 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93581e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a1490 0x561c93582370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 0x561c935828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff424119710 con 0x561c931a1490 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b5e70 con 0x561c93188cd0 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff424119de0 con 0x561c931a9670 2024-09-21T06:17:00.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42f74b740 10 client.0 _ll_register_callbacks cb 0x7ff41c0305a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.460+0000 7ff42359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 0x561c935828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 0x561c935828b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42614/0 (socket says 172.21.15.31:42614) 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 -- 172.21.15.31:0/994111521 learned_addr learned my addr 172.21.15.31:0/994111521 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42259c640 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a1490 0x561c93582370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 -- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 msgr2=0x561c93581e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93581e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 -- 172.21.15.31:0/994111521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a1490 msgr2=0x561c93582370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a1490 0x561c93582370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.461+0000 7ff42359e640 1 -- 172.21.15.31:0/994111521 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c935f9d10 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff42359e640 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 0x561c935828b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff4140128e0 tx=0x7ff414012da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d64b5f5c83a3d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff406ff5640 1 -- 172.21.15.31:0/994111521 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414022020 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff406ff5640 1 -- 172.21.15.31:0/994111521 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7ff414004250 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff41c071070 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff406ff5640 1 -- 172.21.15.31:0/994111521 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff414004450 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff42f74b740 10 client.26482 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.462+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561c935fb060 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff406ff5640 1 -- 172.21.15.31:0/994111521 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff41c071070 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff406ff5640 1 -- 172.21.15.31:0/994111521 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7ff414003c70 con 0x561c931a9670 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff42f74b740 10 client.26482 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff42f74b740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff42f74b740 20 client.26482 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff42f74b740 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c931ca350 0x561c935fd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff42f74b740 4 client.26482 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c934e9db0 con 0x561c931ca350 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.463+0000 7ff422d9d640 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c931ca350 0x561c935fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.464+0000 7ff406ff5640 10 client.26482 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:00.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.464+0000 7ff422d9d640 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c931ca350 0x561c935fd880 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3d0b66b07d6d9b server_cookie=9761d29b510eda46 in_seq=0 out_seq=0 2024-09-21T06:17:00.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.466+0000 7ff406ff5640 1 -- 172.21.15.31:0/994111521 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c934e9db0 con 0x561c931ca350 2024-09-21T06:17:00.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.466+0000 7ff406ff5640 10 client.26482 handle_command_reply: tid=0 2024-09-21T06:17:00.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 client.26482 shutdown 2024-09-21T06:17:00.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 msgr2=0x561c935828b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 0x561c935828b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff4140128e0 tx=0x7ff414012da0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 shutdown_connections 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561c931ca350 0x561c935fd880 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561c931a9670 0x561c935828b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c931a1490 0x561c93582370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 --2- 172.21.15.31:0/994111521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c93188cd0 0x561c93581e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff406ff5640 0 client.26482 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 >> 172.21.15.31:0/994111521 conn(0x561c931be740 msgr2=0x561c93572460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 shutdown_connections 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.467+0000 7ff42f74b740 1 -- 172.21.15.31:0/994111521 wait complete. 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff42f74b740 20 client.26482 trim_cache size 0 max 16384 2024-09-21T06:17:00.470 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:17:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "500d07c3-510a-4870-b876-5eb317d6c7ec", 2024-09-21T06:17:00.471 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:17:00.471 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:00.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff40c0b67d0 msgr2=0x7ff40c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff40c0b67d0 0x7ff40c0b8ce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0094e0 tx=0x7ff41c0098c0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 msgr2=0x7ff4241a9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff4241a9880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff418005fd0 tx=0x7ff41800dde0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 shutdown_connections 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff42411cc30 0x7ff4241aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff424111f40 0x7ff4241a9880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff40c0b67d0 0x7ff40c0b8ce0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 --2- 172.21.15.31:0/1636310109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff42410a9e0 0x7ff4241a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.468+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 >> 172.21.15.31:0/1636310109 conn(0x7ff424070ec0 msgr2=0x7ff42411e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.469+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 shutdown_connections 2024-09-21T06:17:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:00.469+0000 7ff4291d7640 1 -- 172.21.15.31:0/1636310109 wait complete. 2024-09-21T06:17:01.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:01 smithi110 ceph-mon[34550]: pgmap v482: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 866 op/s 2024-09-21T06:17:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:01 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:17:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:01 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:01 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:17:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:01 smithi107 ceph-mon[34380]: pgmap v482: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 866 op/s 2024-09-21T06:17:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:01 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:17:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:01 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:01 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:17:01.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:01 smithi031 ceph-mon[31237]: pgmap v482: 97 pgs: 97 active+clean; 4.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 18 MiB/s wr, 866 op/s 2024-09-21T06:17:01.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:01 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:17:01.630 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:01 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:01.631 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:01 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:17:02.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 -- 172.21.15.31:0/860428243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a8108d00 msgr2=0x7fc3a810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 --2- 172.21.15.31:0/860428243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a8108d00 0x7fc3a810f530 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009ca0 tx=0x7fc3a002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 -- 172.21.15.31:0/860428243 shutdown_connections 2024-09-21T06:17:02.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 --2- 172.21.15.31:0/860428243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a8108d00 0x7fc3a810f530 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 --2- 172.21.15.31:0/860428243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a81083b0 0x7fc3a81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 --2- 172.21.15.31:0/860428243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3a8102a70 0x7fc3a8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 -- 172.21.15.31:0/860428243 >> 172.21.15.31:0/860428243 conn(0x7fc3a8079730 msgr2=0x7fc3a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:02.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 -- 172.21.15.31:0/860428243 shutdown_connections 2024-09-21T06:17:02.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.039+0000 7fc3af754640 1 -- 172.21.15.31:0/860428243 wait complete. 2024-09-21T06:17:02.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.040+0000 7fc3af754640 1 Processor -- start 2024-09-21T06:17:02.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.040+0000 7fc3af754640 1 -- start start 2024-09-21T06:17:02.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.040+0000 7fc3af754640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 0x7fc3a80ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.040+0000 7fc3af754640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 0x7fc3a80ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.040+0000 7fc3af754640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3a8108d00 0x7fc3a8100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3af754640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8112970 con 0x7fc3a81083b0 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3af754640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81127f0 con 0x7fc3a8102a70 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3af754640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8112af0 con 0x7fc3a8108d00 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 0x7fc3a80ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 0x7fc3a80ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 0x7fc3a80ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:60920/0 (socket says 172.21.15.31:60920) 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 0x7fc3a80ffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38694/0 (socket says 172.21.15.31:38694) 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/2037923660 learned_addr learned my addr 172.21.15.31:0/2037923660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3adcf1640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3a8108d00 0x7fc3a8100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 -- 172.21.15.31:0/2037923660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3a8108d00 msgr2=0x7fc3a8100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3a8108d00 0x7fc3a8100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 -- 172.21.15.31:0/2037923660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 msgr2=0x7fc3a80ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 0x7fc3a80ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 -- 172.21.15.31:0/2037923660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a81a8710 con 0x7fc3a81083b0 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 0x7fc3a80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.041+0000 7fc3accef640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 0x7fc3a80ffb30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc39800e9c0 tx=0x7fc39800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.042+0000 7fc39e7fc640 1 -- 172.21.15.31:0/2037923660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39800ccc0 con 0x7fc3a81083b0 2024-09-21T06:17:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.042+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a81a8a00 con 0x7fc3a81083b0 2024-09-21T06:17:02.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.042+0000 7fc39e7fc640 1 -- 172.21.15.31:0/2037923660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3980108e0 con 0x7fc3a81083b0 2024-09-21T06:17:02.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.042+0000 7fc39e7fc640 1 -- 172.21.15.31:0/2037923660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3980187c0 con 0x7fc3a81083b0 2024-09-21T06:17:02.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.042+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a81a8f40 con 0x7fc3a81083b0 2024-09-21T06:17:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.045+0000 7fc3b5cc8740 1 Processor -- start 2024-09-21T06:17:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.045+0000 7fc3b5cc8740 1 -- start start 2024-09-21T06:17:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84be270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5581d8434e50 con 0x5581d8117070 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5581d8424280 con 0x5581d80f6390 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5581d8428090 con 0x5581d80ee1b0 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc39e7fc640 1 -- 172.21.15.31:0/2037923660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc398018960 con 0x7fc3a81083b0 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84be270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38702/0 (socket says 172.21.15.31:38702) 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 -- 172.21.15.31:0/2842028370 learned_addr learned my addr 172.21.15.31:0/2842028370 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3accef640 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 -- 172.21.15.31:0/2842028370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 msgr2=0x5581d84be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 -- 172.21.15.31:0/2842028370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 msgr2=0x5581d84becf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.046+0000 7fc3adcf1640 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84becf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc3adcf1640 1 -- 172.21.15.31:0/2842028370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d8182650 con 0x5581d8117070 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc3adcf1640 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84be270 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc3a002f410 tx=0x7fc3a002f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7602019d23698fac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc3937fe640 1 -- 172.21.15.31:0/2842028370 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0031cf0 con 0x5581d8117070 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc3937fe640 1 -- 172.21.15.31:0/2842028370 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a0036030 con 0x5581d8117070 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc3937fe640 1 -- 172.21.15.31:0/2842028370 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00042a0 con 0x5581d8117070 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc39e7fc640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3940b6680 0x7fc3940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3940b6680 0x7fc3940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.047+0000 7fc39e7fc640 1 -- 172.21.15.31:0/2037923660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc3980e7a50 con 0x7fc3a81083b0 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3940b6680 0x7fc3940b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5581d84bcef0 tx=0x7fc3a4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/2842028370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 msgr2=0x5581d84be270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84be270 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc3a002f410 tx=0x7fc3a002f440 comp rx=0 tx=0).stop 2024-09-21T06:17:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/2842028370 shutdown_connections 2024-09-21T06:17:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84becf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/2842028370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84be270 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/2842028370 >> 172.21.15.31:0/2842028370 conn(0x5581d810b460 msgr2=0x5581d84bc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/2842028370 shutdown_connections 2024-09-21T06:17:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.048+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/2842028370 wait complete. 2024-09-21T06:17:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.049+0000 7fc3b5cc8740 1 -- start start 2024-09-21T06:17:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.049+0000 7fc3b5cc8740 1 Processor -- start 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.049+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5581d8285700 con 0x5581d8117070 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5581d83bf110 con 0x5581d80f6390 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5581d84cc870 con 0x5581d80ee1b0 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3b5cc8740 10 client.0 _ll_register_callbacks cb 0x5581d83d8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42636/0 (socket says 172.21.15.31:42636) 2024-09-21T06:17:02.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1481902154 learned_addr learned my addr 172.21.15.31:0/1481902154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3accef640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3adcf1640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 msgr2=0x5581d84c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1481902154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 msgr2=0x5581d84d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84d5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1481902154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d83ecc40 con 0x5581d80ee1b0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84c98e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4013010 tx=0x7fc3a4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91f19d30d349e469 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.050+0000 7fc390ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc390ff9640 1 -- 172.21.15.31:0/1481902154 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4004050 con 0x5581d80ee1b0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc390ff9640 1 -- 172.21.15.31:0/1481902154 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc3a40041f0 con 0x5581d80ee1b0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a0036030 con 0x5581d80ee1b0 2024-09-21T06:17:02.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc390ff9640 1 -- 172.21.15.31:0/1481902154 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc3a40043f0 con 0x5581d80ee1b0 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc3b5cc8740 10 client.26494 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5581d84d85a0 con 0x5581d80ee1b0 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc390ff9640 1 -- 172.21.15.31:0/1481902154 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a0036030 con 0x5581d80ee1b0 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc390ff9640 1 -- 172.21.15.31:0/1481902154 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc3a4027020 con 0x5581d80ee1b0 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc3b5cc8740 10 client.26494 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.051+0000 7fc3b5cc8740 10 client.26494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc3b5cc8740 20 client.26494 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80d59f0 0x5581d84dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc3b5cc8740 4 client.26494 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:02.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581d8436da0 con 0x5581d80d59f0 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc3accef640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80d59f0 0x5581d84dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc390ff9640 10 client.26494 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.052+0000 7fc3accef640 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80d59f0 0x5581d84dbad0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e14d5cf5b706395 server_cookie=2d79f5050da31464 in_seq=0 out_seq=0 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc390ff9640 1 -- 172.21.15.31:0/1481902154 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5581d8436da0 con 0x5581d80d59f0 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc390ff9640 10 client.26494 handle_command_reply: tid=0 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 client.26494 shutdown 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 msgr2=0x5581d84c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84c98e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4013010 tx=0x7fc3a4004730 comp rx=0 tx=0).stop 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 shutdown_connections 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80d59f0 0x5581d84dbad0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d8117070 0x5581d84d5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d80f6390 0x5581d84c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1481902154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84c98e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc390ff9640 0 client.26494 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:02.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.053+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 >> 172.21.15.31:0/1481902154 conn(0x5581d810b460 msgr2=0x5581d84c1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:02.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.054+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 shutdown_connections 2024-09-21T06:17:02.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.054+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1481902154 wait complete. 2024-09-21T06:17:02.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.054+0000 7fc3b5cc8740 20 client.26494 trim_cache size 0 max 16384 2024-09-21T06:17:02.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 Processor -- start 2024-09-21T06:17:02.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 -- start start 2024-09-21T06:17:02.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 0x5581d84d5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80f6390 0x5581d84c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81af890 con 0x5581d80ee1b0 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81b2110 con 0x5581d8117070 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81aff00 con 0x5581d80f6390 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 0x5581d84d5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 0x5581d84d5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38718/0 (socket says 172.21.15.31:38718) 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/4249076708 learned_addr learned my addr 172.21.15.31:0/4249076708 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3accef640 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80f6390 0x5581d84c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.075+0000 7fc3adcf1640 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/4249076708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80f6390 msgr2=0x5581d84c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80f6390 0x5581d84c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/4249076708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 msgr2=0x5581d84c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/4249076708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581d84cc7b0 con 0x5581d80ee1b0 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3accef640 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80f6390 0x5581d84c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:02.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 0x5581d84d5a60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40092e0 tx=0x7fc3a4029440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d1312832a31545 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:02.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc392ffd640 1 -- 172.21.15.31:0/4249076708 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4003f70 con 0x5581d80ee1b0 2024-09-21T06:17:02.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc392ffd640 1 -- 172.21.15.31:0/4249076708 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a4037020 con 0x5581d80ee1b0 2024-09-21T06:17:02.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.076+0000 7fc392ffd640 1 -- 172.21.15.31:0/4249076708 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4011de0 con 0x5581d80ee1b0 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/4249076708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 msgr2=0x5581d84d5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 0x5581d84d5a60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40092e0 tx=0x7fc3a4029440 comp rx=0 tx=0).stop 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/4249076708 shutdown_connections 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80f6390 0x5581d84c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/4249076708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80ee1b0 0x5581d84d5a60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.077+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/4249076708 >> 172.21.15.31:0/4249076708 conn(0x5581d810b460 msgr2=0x5581d84c8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.078+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/4249076708 shutdown_connections 2024-09-21T06:17:02.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.078+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/4249076708 wait complete. 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.078+0000 7fc3b5cc8740 1 -- start start 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.078+0000 7fc3b5cc8740 1 Processor -- start 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80f6390 0x5581d84cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81b05d0 con 0x5581d80f6390 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81b1a40 con 0x5581d8117070 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a81b0ca0 con 0x5581d80ee1b0 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3b5cc8740 10 client.0 _ll_register_callbacks cb 0x7fc3a4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84d2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42650/0 (socket says 172.21.15.31:42650) 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3adcf1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1434717889 learned_addr learned my addr 172.21.15.31:0/1434717889 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:02.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3accef640 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80f6390 0x5581d84cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 msgr2=0x5581d84cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1434717889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80f6390 msgr2=0x5581d84cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80f6390 0x5581d84cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 -- 172.21.15.31:0/1434717889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581d84ce240 con 0x5581d80ee1b0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc3ad4f0640 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84d2190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4003e20 tx=0x7fc3a4004330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=904dbc53fdd9cbd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.079+0000 7fc390ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc390ff9640 1 -- 172.21.15.31:0/1434717889 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4011d70 con 0x5581d80ee1b0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc390ff9640 1 -- 172.21.15.31:0/1434717889 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fc3a40035f0 con 0x5581d80ee1b0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a4037020 con 0x5581d80ee1b0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc390ff9640 1 -- 172.21.15.31:0/1434717889 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc3a4003830 con 0x5581d80ee1b0 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 10 client.26500 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:02.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5581d85462d0 con 0x5581d80ee1b0 2024-09-21T06:17:02.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc390ff9640 1 -- 172.21.15.31:0/1434717889 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a4037020 con 0x5581d80ee1b0 2024-09-21T06:17:02.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc390ff9640 1 -- 172.21.15.31:0/1434717889 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc3a4027040 con 0x5581d80ee1b0 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 10 client.26500 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 20 client.26500 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80aada0 0x5581d8548920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 4 client.26500 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.080+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5581d8432850 con 0x5581d80aada0 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.081+0000 7fc3accef640 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80aada0 0x5581d8548920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.081+0000 7fc390ff9640 10 client.26500 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.081+0000 7fc3accef640 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80aada0 0x5581d8548920 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e8edf8e51c96c6 server_cookie=c6a773999c925321 in_seq=0 out_seq=0 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc390ff9640 1 -- 172.21.15.31:0/1434717889 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x5581d8432850 con 0x5581d80aada0 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc390ff9640 10 client.26500 handle_command_reply: tid=0 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 client.26500 shutdown 2024-09-21T06:17:02.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 msgr2=0x5581d84d2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84d2190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4003e20 tx=0x7fc3a4004330 comp rx=0 tx=0).stop 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 shutdown_connections 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5581d80aada0 0x5581d8548920 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581d8117070 0x5581d84cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5581d80f6390 0x5581d84cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 --2- 172.21.15.31:0/1434717889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5581d80ee1b0 0x5581d84d2190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 >> 172.21.15.31:0/1434717889 conn(0x5581d810b460 msgr2=0x5581d84c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc390ff9640 0 client.26500 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 shutdown_connections 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 1 -- 172.21.15.31:0/1434717889 wait complete. 2024-09-21T06:17:02.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.082+0000 7fc3b5cc8740 20 client.26500 trim_cache size 0 max 16384 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.090 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:17:02.091 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.092 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.093 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.094 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:17:02.095 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.096 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:17:02.097 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:02.098 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:02.099 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3940b6680 msgr2=0x7fc3940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3940b6680 0x7fc3940b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x5581d84bcef0 tx=0x7fc3a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 msgr2=0x7fc3a80ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 0x7fc3a80ffb30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc39800e9c0 tx=0x7fc39800ee80 comp rx=0 tx=0).stop 2024-09-21T06:17:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 shutdown_connections 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3a8108d00 0x7fc3a8100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc3940b6680 0x7fc3940b8b90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3a81083b0 0x7fc3a80ffb30 secure :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc39800e9c0 tx=0x7fc39800ee80 comp rx=0 tx=0).stop 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 --2- 172.21.15.31:0/2037923660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3a8102a70 0x7fc3a80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.083+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 >> 172.21.15.31:0/2037923660 conn(0x7fc3a8079730 msgr2=0x7fc3a8101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.084+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 shutdown_connections 2024-09-21T06:17:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:02.084+0000 7fc3af754640 1 -- 172.21.15.31:0/2037923660 wait complete. 2024-09-21T06:17:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:02 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:02.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:02 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:17:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:02 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:02.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:02 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:17:02.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:02 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:02.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:02 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:17:03.261 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:17:03.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:17:03.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:03 smithi110 ceph-mon[34550]: pgmap v483: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 22 MiB/s wr, 985 op/s 2024-09-21T06:17:03.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:03 smithi107 ceph-mon[34380]: pgmap v483: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 22 MiB/s wr, 985 op/s 2024-09-21T06:17:03.558 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:03 smithi031 ceph-mon[31237]: pgmap v483: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 22 MiB/s wr, 985 op/s 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 -- 172.21.15.31:0/1319433353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef81049d0 msgr2=0x7efef8104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 --2- 172.21.15.31:0/1319433353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef81049d0 0x7efef8104de0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efee802a410 comp rx=0 tx=0).stop 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 -- 172.21.15.31:0/1319433353 shutdown_connections 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 --2- 172.21.15.31:0/1319433353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef810de30 0x7efef8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 --2- 172.21.15.31:0/1319433353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 0x7efef810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 --2- 172.21.15.31:0/1319433353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef81049d0 0x7efef8104de0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 -- 172.21.15.31:0/1319433353 >> 172.21.15.31:0/1319433353 conn(0x7efef8079740 msgr2=0x7efef8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 -- 172.21.15.31:0/1319433353 shutdown_connections 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.151+0000 7efefe319640 1 -- 172.21.15.31:0/1319433353 wait complete. 2024-09-21T06:17:04.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.152+0000 7efefe319640 1 Processor -- start 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.152+0000 7efefe319640 1 -- start start 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefe319640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 0x7efef8102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefe319640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 0x7efef8102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefe319640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef810de30 0x7efef8103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefe319640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efef8113ab0 con 0x7efef810de30 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefe319640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efef8113930 con 0x7efef8105320 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefe319640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efef8113c30 con 0x7efef81049d0 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efef7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 0x7efef8102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efefc8b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef810de30 0x7efef8103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efef7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 0x7efef8102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42678/0 (socket says 172.21.15.31:42678) 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efef7fff640 1 -- 172.21.15.31:0/2737625215 learned_addr learned my addr 172.21.15.31:0/2737625215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.153+0000 7efef77fe640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 0x7efef8102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef7fff640 1 -- 172.21.15.31:0/2737625215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 msgr2=0x7efef8102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef7fff640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 0x7efef8102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef7fff640 1 -- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef810de30 msgr2=0x7efef8103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef7fff640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef810de30 0x7efef8103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef7fff640 1 -- 172.21.15.31:0/2737625215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef81a8860 con 0x7efef81049d0 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efefc8b6640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef810de30 0x7efef8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef77fe640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 0x7efef8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef7fff640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 0x7efef8102710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efee8006010 tx=0x7efee8004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee803f030 con 0x7efef81049d0 2024-09-21T06:17:04.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee8003aa0 con 0x7efef81049d0 2024-09-21T06:17:04.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef81a8af0 con 0x7efef81049d0 2024-09-21T06:17:04.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8035070 con 0x7efef81049d0 2024-09-21T06:17:04.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.154+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef81a8fd0 con 0x7efef81049d0 2024-09-21T06:17:04.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.155+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef8104de0 con 0x7efef81049d0 2024-09-21T06:17:04.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.157+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efee8042030 con 0x7efef81049d0 2024-09-21T06:17:04.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.158+0000 7efef57fa640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efee00b6680 0x7efee00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:04.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.158+0000 7efef77fe640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efee00b6680 0x7efee00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:04.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.158+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efee8107b10 con 0x7efef81049d0 2024-09-21T06:17:04.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.158+0000 7efef77fe640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efee00b6680 0x7efee00b8b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efef810fb30 tx=0x7efeec00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:04.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.164+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efee80d2480 con 0x7efef81049d0 2024-09-21T06:17:04.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.621+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efef8103a10 con 0x7efef81049d0 2024-09-21T06:17:04.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.625+0000 7efef57fa640 1 -- 172.21.15.31:0/2737625215 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7efee80d73c0 con 0x7efef81049d0 2024-09-21T06:17:04.630 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:17:04.630 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:17:04.630 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:17:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efee00b6680 msgr2=0x7efee00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efee00b6680 0x7efee00b8b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efef810fb30 tx=0x7efeec00b410 comp rx=0 tx=0).stop 2024-09-21T06:17:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 msgr2=0x7efef8102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 0x7efef8102710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efee8006010 tx=0x7efee8004060 comp rx=0 tx=0).stop 2024-09-21T06:17:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 shutdown_connections 2024-09-21T06:17:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efee00b6680 0x7efee00b8b90 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efef810fb30 tx=0x7efeec00b410 comp rx=0 tx=0).stop 2024-09-21T06:17:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efef810de30 0x7efef8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efef8105320 0x7efef8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 --2- 172.21.15.31:0/2737625215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efef81049d0 0x7efef8102710 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.635+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 >> 172.21.15.31:0/2737625215 conn(0x7efef8079740 msgr2=0x7efef81068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:04.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.640+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 shutdown_connections 2024-09-21T06:17:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:04.642+0000 7efefe319640 1 -- 172.21.15.31:0/2737625215 wait complete. 2024-09-21T06:17:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 325.0 s, 433.4 tps, lat 165.857 ms stddev 226.766 2024-09-21T06:17:05.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:17:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:05 smithi110 ceph-mon[34550]: pgmap v484: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 19 MiB/s wr, 772 op/s 2024-09-21T06:17:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:05 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2737625215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:05 smithi107 ceph-mon[34380]: pgmap v484: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 19 MiB/s wr, 772 op/s 2024-09-21T06:17:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:05 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2737625215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:05.604 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:05 smithi031 ceph-mon[31237]: pgmap v484: 97 pgs: 97 active+clean; 4.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 19 MiB/s wr, 772 op/s 2024-09-21T06:17:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:05 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2737625215' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 -- 172.21.15.31:0/1738138832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 msgr2=0x7f60701087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 --2- 172.21.15.31:0/1738138832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f60701087b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6064009900 tx=0x7f606402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 -- 172.21.15.31:0/1738138832 shutdown_connections 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 --2- 172.21.15.31:0/1738138832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6070108cf0 0x7f607010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 --2- 172.21.15.31:0/1738138832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f60701087b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 --2- 172.21.15.31:0/1738138832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6070102a60 0x7f6070102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 -- 172.21.15.31:0/1738138832 >> 172.21.15.31:0/1738138832 conn(0x7f6070079730 msgr2=0x7f6070079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 -- 172.21.15.31:0/1738138832 shutdown_connections 2024-09-21T06:17:06.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.098+0000 7f6076171640 1 -- 172.21.15.31:0/1738138832 wait complete. 2024-09-21T06:17:06.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.099+0000 7f6076171640 1 Processor -- start 2024-09-21T06:17:06.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.099+0000 7f6076171640 1 -- start start 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.099+0000 7f6076171640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6070102a60 0x7f607019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.099+0000 7f6076171640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f607019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f6076171640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6070108cf0 0x7f607019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f6076171640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6070112960 con 0x7f60701083a0 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f6076171640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60701127e0 con 0x7f6070108cf0 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f6076171640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6070112ae0 con 0x7f6070102a60 2024-09-21T06:17:06.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f607019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f607019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38756/0 (socket says 172.21.15.31:38756) 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 -- 172.21.15.31:0/3568791590 learned_addr learned my addr 172.21.15.31:0/3568791590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606ffff640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6070108cf0 0x7f607019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606f7fe640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6070102a60 0x7f607019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 -- 172.21.15.31:0/3568791590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6070102a60 msgr2=0x7f607019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6070102a60 0x7f607019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 -- 172.21.15.31:0/3568791590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6070108cf0 msgr2=0x7f607019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6070108cf0 0x7f607019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 -- 172.21.15.31:0/3568791590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60701a6530 con 0x7f60701083a0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606effd640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f607019b7b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6064032fd0 tx=0x7f6064031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.100+0000 7f606cff9640 1 -- 172.21.15.31:0/3568791590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606403f030 con 0x7f60701083a0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.101+0000 7f606cff9640 1 -- 172.21.15.31:0/3568791590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6064004490 con 0x7f60701083a0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.101+0000 7f606cff9640 1 -- 172.21.15.31:0/3568791590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064035070 con 0x7f60701083a0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.101+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60701a67c0 con 0x7f60701083a0 2024-09-21T06:17:06.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.101+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60701a6c50 con 0x7f60701083a0 2024-09-21T06:17:06.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.103+0000 7f607c6e5740 1 Processor -- start 2024-09-21T06:17:06.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.103+0000 7f607c6e5740 1 -- start start 2024-09-21T06:17:06.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.103+0000 7f606cff9640 1 -- 172.21.15.31:0/3568791590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6064042030 con 0x7f60701083a0 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089381be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089382120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f607c6e5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5620892f7ea0 con 0x562088fb9390 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f607c6e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5620892e72d0 con 0x562088fb11b0 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f607c6e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5620892eb0e0 con 0x562088fda070 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089381be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089381be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32790/0 (socket says 172.21.15.31:32790) 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 -- 172.21.15.31:0/430514733 learned_addr learned my addr 172.21.15.31:0/430514733 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606effd640 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089382120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 -- 172.21.15.31:0/430514733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 msgr2=0x5620893816a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893816a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 -- 172.21.15.31:0/430514733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 msgr2=0x562089382120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089382120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.104+0000 7f606f7fe640 1 -- 172.21.15.31:0/430514733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562089045650 con 0x562088fb11b0 2024-09-21T06:17:06.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f606cff9640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60580b6700 0x7f60580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f606f7fe640 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089381be0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6060009c90 tx=0x7f6060011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47e2d414da191bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f606ffff640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60580b6700 0x7f60580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f606cff9640 1 -- 172.21.15.31:0/3568791590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f6064107a00 con 0x7f60701083a0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f6055ffb640 1 -- 172.21.15.31:0/430514733 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060005220 con 0x562088fb11b0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f6055ffb640 1 -- 172.21.15.31:0/430514733 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6060004c50 con 0x562088fb11b0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.105+0000 7f6055ffb640 1 -- 172.21.15.31:0/430514733 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600196b0 con 0x562088fb11b0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 -- 172.21.15.31:0/430514733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 msgr2=0x562089381be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f606ffff640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60580b6700 0x7f60580b8c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6068005fd0 tx=0x7f6068009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089381be0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6060009c90 tx=0x7f6060011810 comp rx=0 tx=0).stop 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 -- 172.21.15.31:0/430514733 shutdown_connections 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089382120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089381be0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 --2- 172.21.15.31:0/430514733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893816a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 -- 172.21.15.31:0/430514733 >> 172.21.15.31:0/430514733 conn(0x562088fce460 msgr2=0x56208937f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:06.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 -- 172.21.15.31:0/430514733 shutdown_connections 2024-09-21T06:17:06.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 -- 172.21.15.31:0/430514733 wait complete. 2024-09-21T06:17:06.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.106+0000 7f607c6e5740 1 -- start start 2024-09-21T06:17:06.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 Processor -- start 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x5620893911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x5620893916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x56208939d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562089148700 con 0x562088fb9390 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562089282160 con 0x562088fb11b0 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5620893941b0 con 0x562088fda070 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f607c6e5740 10 client.0 _ll_register_callbacks cb 0x56208929b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:06.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x5620893911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f606effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x5620893916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f606ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x56208939d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x5620893911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32796/0 (socket says 172.21.15.31:32796) 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f606effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x5620893916e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38776/0 (socket says 172.21.15.31:38776) 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.107+0000 7f606f7fe640 1 -- 172.21.15.31:0/1526496441 learned_addr learned my addr 172.21.15.31:0/1526496441 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606ffff640 1 -- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 msgr2=0x5620893911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606ffff640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x5620893911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606ffff640 1 -- 172.21.15.31:0/1526496441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 msgr2=0x5620893916e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606ffff640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x5620893916e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606ffff640 1 -- 172.21.15.31:0/1526496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620892afc90 con 0x562088fda070 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606f7fe640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x5620893911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606effd640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x5620893916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:06.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f606ffff640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x56208939d4e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6068011e90 tx=0x7f6068004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5199ca1b48e1d8c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f604effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f604effd640 1 -- 172.21.15.31:0/1526496441 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606800f870 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6060004c50 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.108+0000 7f604effd640 1 -- 172.21.15.31:0/1526496441 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f606800fa10 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f604effd640 1 -- 172.21.15.31:0/1526496441 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f606800fc10 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 10 client.26518 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56208939ffe0 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f604effd640 1 -- 172.21.15.31:0/1526496441 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6060004c50 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f604effd640 1 -- 172.21.15.31:0/1526496441 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6068026070 con 0x562088fda070 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 10 client.26518 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 20 client.26518 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f989f0 0x5620893a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 4 client.26518 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.109+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620892f9df0 con 0x562088f989f0 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.110+0000 7f606f7fe640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f989f0 0x5620893a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.110+0000 7f604effd640 10 client.26518 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.110+0000 7f606f7fe640 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f989f0 0x5620893a3560 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901552750f5e1428 server_cookie=9207a5b72004ed3b in_seq=0 out_seq=0 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f604effd640 1 -- 172.21.15.31:0/1526496441 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620892f9df0 con 0x562088f989f0 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f604effd640 10 client.26518 handle_command_reply: tid=0 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 client.26518 shutdown 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 msgr2=0x56208939d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x56208939d4e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6068011e90 tx=0x7f6068004ab0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 shutdown_connections 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f989f0 0x5620893a3560 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x56208939d4e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x5620893916e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 --2- 172.21.15.31:0/1526496441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x5620893911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f604effd640 0 client.26518 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 >> 172.21.15.31:0/1526496441 conn(0x562088fce460 msgr2=0x5620893847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 shutdown_connections 2024-09-21T06:17:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 1 -- 172.21.15.31:0/1526496441 wait complete. 2024-09-21T06:17:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.111+0000 7f607c6e5740 20 client.26518 trim_cache size 0 max 16384 2024-09-21T06:17:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 Processor -- start 2024-09-21T06:17:06.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 -- start start 2024-09-21T06:17:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 0x562089391660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x56208939d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fda070 0x56208938e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60701ab3f0 con 0x562088fb9390 2024-09-21T06:17:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60701adc70 con 0x562088fda070 2024-09-21T06:17:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f607c6e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60701aba60 con 0x562088fb11b0 2024-09-21T06:17:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 0x562089391660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f606effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x56208939d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f606ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fda070 0x56208938e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 0x562089391660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42744/0 (socket says 172.21.15.31:42744) 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.125+0000 7f606f7fe640 1 -- 172.21.15.31:0/3479279758 learned_addr learned my addr 172.21.15.31:0/3479279758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606ffff640 1 -- 172.21.15.31:0/3479279758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 msgr2=0x562089391660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606ffff640 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 0x562089391660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606ffff640 1 -- 172.21.15.31:0/3479279758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 msgr2=0x56208939d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606ffff640 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x56208939d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606ffff640 1 -- 172.21.15.31:0/3479279758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620893844a0 con 0x562088fda070 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606f7fe640 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 0x562089391660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f606ffff640 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fda070 0x56208938e390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6068000f80 tx=0x7f6068009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed208eda34a2921 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:06.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f60557fa640 1 -- 172.21.15.31:0/3479279758 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068035070 con 0x562088fda070 2024-09-21T06:17:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f60557fa640 1 -- 172.21.15.31:0/3479279758 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6068028750 con 0x562088fda070 2024-09-21T06:17:06.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.126+0000 7f60557fa640 1 -- 172.21.15.31:0/3479279758 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068028e30 con 0x562088fda070 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 -- 172.21.15.31:0/3479279758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fda070 msgr2=0x56208938e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fda070 0x56208938e390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6068000f80 tx=0x7f6068009620 comp rx=0 tx=0).stop 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 -- 172.21.15.31:0/3479279758 shutdown_connections 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fda070 0x56208938e390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x56208939d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 --2- 172.21.15.31:0/3479279758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fb11b0 0x562089391660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 -- 172.21.15.31:0/3479279758 >> 172.21.15.31:0/3479279758 conn(0x562088fce460 msgr2=0x562089390f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 -- 172.21.15.31:0/3479279758 shutdown_connections 2024-09-21T06:17:06.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.127+0000 7f607c6e5740 1 -- 172.21.15.31:0/3479279758 wait complete. 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 -- start start 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 Processor -- start 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089399b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089395090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60701ac130 con 0x562088fb9390 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60701ad5a0 con 0x562088fb11b0 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60701ac800 con 0x562088fda070 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f607c6e5740 10 client.0 _ll_register_callbacks cb 0x7f6068009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f606effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089395090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089399b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089399b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32822/0 (socket says 172.21.15.31:32822) 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.128+0000 7f606f7fe640 1 -- 172.21.15.31:0/687262871 learned_addr learned my addr 172.21.15.31:0/687262871 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606ffff640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606f7fe640 1 -- 172.21.15.31:0/687262871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 msgr2=0x5620893955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606f7fe640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606f7fe640 1 -- 172.21.15.31:0/687262871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 msgr2=0x562089395090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606f7fe640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089395090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606f7fe640 1 -- 172.21.15.31:0/687262871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562089395cc0 con 0x562088fb11b0 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606effd640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089395090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606ffff640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f606f7fe640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089399b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6060006010 tx=0x7f60600097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44fa461e823b2dab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:06.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f604effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f604effd640 1 -- 172.21.15.31:0/687262871 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060004450 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f604effd640 1 -- 172.21.15.31:0/687262871 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f60600035f0 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6068028750 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f604effd640 1 -- 172.21.15.31:0/687262871 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f606003b050 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f607c6e5740 10 client.26477 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.129+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56208940ddc0 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f604effd640 1 -- 172.21.15.31:0/687262871 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6068028750 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f604effd640 1 -- 172.21.15.31:0/687262871 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6060019840 con 0x562088fb11b0 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f607c6e5740 10 client.26477 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f607c6e5740 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f607c6e5740 20 client.26477 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:06.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f607c6e5740 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f6dda0 0x5620894103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f607c6e5740 4 client.26477 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620892f58a0 con 0x562088f6dda0 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f606effd640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f6dda0 0x5620894103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f606effd640 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f6dda0 0x5620894103c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4f0d228c942bd7 server_cookie=2fd9c287c37e87b2 in_seq=0 out_seq=0 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.130+0000 7f604effd640 10 client.26477 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f604effd640 1 -- 172.21.15.31:0/687262871 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620892f58a0 con 0x562088f6dda0 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f604effd640 10 client.26477 handle_command_reply: tid=0 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 client.26477 shutdown 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 msgr2=0x562089399b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089399b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6060006010 tx=0x7f60600097c0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 shutdown_connections 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562088f6dda0 0x5620894103c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562088fda070 0x5620893955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562088fb9390 0x562089395090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f604effd640 0 client.26477 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 --2- 172.21.15.31:0/687262871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562088fb11b0 0x562089399b90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 >> 172.21.15.31:0/687262871 conn(0x562088fce460 msgr2=0x562089392650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 shutdown_connections 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 1 -- 172.21.15.31:0/687262871 wait complete. 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.131+0000 7f607c6e5740 20 client.26477 trim_cache size 0 max 16384 2024-09-21T06:17:06.139 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60580b6700 msgr2=0x7f60580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60580b6700 0x7f60580b8c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6068005fd0 tx=0x7f6068009500 comp rx=0 tx=0).stop 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 msgr2=0x7f607019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f607019b7b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6064032fd0 tx=0x7f6064031a20 comp rx=0 tx=0).stop 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 shutdown_connections 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6070108cf0 0x7f607019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60580b6700 0x7f60580b8c10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60701083a0 0x7f607019b7b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 --2- 172.21.15.31:0/3568791590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6070102a60 0x7f607019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 >> 172.21.15.31:0/3568791590 conn(0x7f6070079730 msgr2=0x7f6070104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 shutdown_connections 2024-09-21T06:17:06.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:06.132+0000 7f6076171640 1 -- 172.21.15.31:0/3568791590 wait complete. 2024-09-21T06:17:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:07 smithi110 ceph-mon[34550]: pgmap v485: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 34 MiB/s wr, 1.36k op/s 2024-09-21T06:17:07.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:07 smithi107 ceph-mon[34380]: pgmap v485: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 34 MiB/s wr, 1.36k op/s 2024-09-21T06:17:07.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:07 smithi031 ceph-mon[31237]: pgmap v485: 97 pgs: 97 active+clean; 4.2 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 34 MiB/s wr, 1.36k op/s 2024-09-21T06:17:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 -- 172.21.15.31:0/3859019898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe480108c30 msgr2=0x7fe48010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 --2- 172.21.15.31:0/3859019898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe480108c30 0x7fe48010f460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe478009ca0 tx=0x7fe47802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 -- 172.21.15.31:0/3859019898 shutdown_connections 2024-09-21T06:17:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 --2- 172.21.15.31:0/3859019898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe480108c30 0x7fe48010f460 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 --2- 172.21.15.31:0/3859019898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4801082e0 0x7fe4801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 --2- 172.21.15.31:0/3859019898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 0x7fe480102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.656+0000 7fe48812b640 1 -- 172.21.15.31:0/3859019898 >> 172.21.15.31:0/3859019898 conn(0x7fe48007a740 msgr2=0x7fe48007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.657+0000 7fe48812b640 1 -- 172.21.15.31:0/3859019898 shutdown_connections 2024-09-21T06:17:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.657+0000 7fe48812b640 1 -- 172.21.15.31:0/3859019898 wait complete. 2024-09-21T06:17:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 Processor -- start 2024-09-21T06:17:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 -- start start 2024-09-21T06:17:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 0x7fe4800750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4801082e0 0x7fe480073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 0x7fe480073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe48006d6a0 con 0x7fe480108c30 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe48006d520 con 0x7fe4801029b0 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe48812b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe48006d820 con 0x7fe4801082e0 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 0x7fe4800750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe4866c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 0x7fe480073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe4866c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 0x7fe480073cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38808/0 (socket says 172.21.15.31:38808) 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.658+0000 7fe4866c8640 1 -- 172.21.15.31:0/2825254057 learned_addr learned my addr 172.21.15.31:0/2825254057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.659+0000 7fe4856c6640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4801082e0 0x7fe480073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.659+0000 7fe485ec7640 1 -- 172.21.15.31:0/2825254057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4801082e0 msgr2=0x7fe480073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.659+0000 7fe485ec7640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4801082e0 0x7fe480073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.659+0000 7fe485ec7640 1 -- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 msgr2=0x7fe480073cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.659+0000 7fe485ec7640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 0x7fe480073cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.659+0000 7fe485ec7640 1 -- 172.21.15.31:0/2825254057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4801b0ef0 con 0x7fe4801029b0 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.660+0000 7fe4856c6640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4801082e0 0x7fe480073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.660+0000 7fe4866c8640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 0x7fe480073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.660+0000 7fe485ec7640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 0x7fe4800750a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e9f0 tx=0x7fe47c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.660+0000 7fe472ffd640 1 -- 172.21.15.31:0/2825254057 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00ccc0 con 0x7fe4801029b0 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.660+0000 7fe472ffd640 1 -- 172.21.15.31:0/2825254057 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47c0108e0 con 0x7fe4801029b0 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.660+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4801b1180 con 0x7fe4801029b0 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.661+0000 7fe472ffd640 1 -- 172.21.15.31:0/2825254057 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c018880 con 0x7fe4801029b0 2024-09-21T06:17:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.661+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4801b1720 con 0x7fe4801029b0 2024-09-21T06:17:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.665+0000 7fe472ffd640 1 -- 172.21.15.31:0/2825254057 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe47c018a20 con 0x7fe4801029b0 2024-09-21T06:17:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe472ffd640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe46c0b6700 0x7fe46c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 Processor -- start 2024-09-21T06:17:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe472ffd640 1 -- 172.21.15.31:0/2825254057 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe47c0e79a0 con 0x7fe4801029b0 2024-09-21T06:17:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 -- start start 2024-09-21T06:17:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe4856c6640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe46c0b6700 0x7fe46c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514d31b0 0x558e518811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514aa2f0 0x558e51881700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e51881c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558e517ed6f0 con 0x558e514aa2f0 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558e517dcb20 con 0x558e514b24d0 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.667+0000 7fe48f09c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558e517e0930 con 0x558e514d31b0 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe4866c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e51881c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe4866c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e51881c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32842/0 (socket says 172.21.15.31:32842) 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe4866c8640 1 -- 172.21.15.31:0/301573828 learned_addr learned my addr 172.21.15.31:0/301573828 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe485ec7640 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514d31b0 0x558e518811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe485ec7640 1 -- 172.21.15.31:0/301573828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 msgr2=0x558e51881c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe485ec7640 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e51881c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe485ec7640 1 -- 172.21.15.31:0/301573828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514aa2f0 msgr2=0x558e51881700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe485ec7640 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514aa2f0 0x558e51881700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.668+0000 7fe485ec7640 1 -- 172.21.15.31:0/301573828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e517a2530 con 0x558e514d31b0 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.669+0000 7fe485ec7640 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514d31b0 0x558e518811c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0e5100 tx=0x7fe47c0ad3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a19740071cc57f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.669+0000 7fe4856c6640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe46c0b6700 0x7fe46c0b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe48007ab90 tx=0x7fe4740108b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.669+0000 7fe46bfff640 1 -- 172.21.15.31:0/301573828 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0b7250 con 0x558e514d31b0 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.669+0000 7fe46bfff640 1 -- 172.21.15.31:0/301573828 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47c0e53b0 con 0x558e514d31b0 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.669+0000 7fe46bfff640 1 -- 172.21.15.31:0/301573828 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0a9490 con 0x558e514d31b0 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 -- 172.21.15.31:0/301573828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514d31b0 msgr2=0x558e518811c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514d31b0 0x558e518811c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0e5100 tx=0x7fe47c0ad3f0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 -- 172.21.15.31:0/301573828 shutdown_connections 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e51881c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514aa2f0 0x558e51881700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 --2- 172.21.15.31:0/301573828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514d31b0 0x558e518811c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.670+0000 7fe48f09c740 1 -- 172.21.15.31:0/301573828 >> 172.21.15.31:0/301573828 conn(0x558e514c7300 msgr2=0x558e5187f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.671+0000 7fe48f09c740 1 -- 172.21.15.31:0/301573828 shutdown_connections 2024-09-21T06:17:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.671+0000 7fe48f09c740 1 -- 172.21.15.31:0/301573828 wait complete. 2024-09-21T06:17:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.672+0000 7fe48f09c740 1 -- start start 2024-09-21T06:17:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.672+0000 7fe48f09c740 1 Processor -- start 2024-09-21T06:17:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.673+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e5188c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.673+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e5188cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.673+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e5188c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.673+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e5188c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42774/0 (socket says 172.21.15.31:42774) 2024-09-21T06:17:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.673+0000 7fe485ec7640 1 -- 172.21.15.31:0/4113454057 learned_addr learned my addr 172.21.15.31:0/4113454057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.673+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51898b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558e51714ab0 con 0x558e514d31b0 2024-09-21T06:17:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4866c8640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51898b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558e51782730 con 0x558e514b24d0 2024-09-21T06:17:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558e5188f880 con 0x558e514aa2f0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4856c6640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e5188cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4856c6640 1 -- 172.21.15.31:0/4113454057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 msgr2=0x558e5188c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4856c6640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e5188c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4856c6640 1 -- 172.21.15.31:0/4113454057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 msgr2=0x558e51898b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4856c6640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51898b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.674+0000 7fe4856c6640 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5153e790 con 0x558e514b24d0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.675+0000 7fe48f09c740 10 client.0 _ll_register_callbacks cb 0x558e517d66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.675+0000 7fe4856c6640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e5188cf30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe474016000 tx=0x7fe474004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169f442389e8359e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.675+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47c0e53b0 con 0x558e514b24d0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.675+0000 7fe4697fa640 10 client.26489 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.675+0000 7fe4697fa640 1 -- 172.21.15.31:0/4113454057 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47402c050 con 0x558e514b24d0 2024-09-21T06:17:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.675+0000 7fe4697fa640 1 -- 172.21.15.31:0/4113454057 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe474004020 con 0x558e514b24d0 2024-09-21T06:17:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe4697fa640 1 -- 172.21.15.31:0/4113454057 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe474023030 con 0x558e514b24d0 2024-09-21T06:17:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 10 client.26489 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x558e515b7620 con 0x558e514b24d0 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe4697fa640 1 -- 172.21.15.31:0/4113454057 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe47402f020 con 0x558e514b24d0 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 10 client.26489 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 10 client.26489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 20 client.26489 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e51491d80 0x558e5189e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 4 client.26489 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.676+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e517ef640 con 0x558e51491d80 2024-09-21T06:17:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.677+0000 7fe485ec7640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e51491d80 0x558e5189e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.678+0000 7fe485ec7640 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e51491d80 0x558e5189e8b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a33c14eaef786f server_cookie=7260edf6a9cbaa52 in_seq=0 out_seq=0 2024-09-21T06:17:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.678+0000 7fe4697fa640 10 client.26489 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.679+0000 7fe4697fa640 1 -- 172.21.15.31:0/4113454057 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e517ef640 con 0x558e51491d80 2024-09-21T06:17:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.679+0000 7fe4697fa640 10 client.26489 handle_command_reply: tid=0 2024-09-21T06:17:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.679+0000 7fe48f09c740 1 client.26489 shutdown 2024-09-21T06:17:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 msgr2=0x558e5188cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e5188cf30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe474016000 tx=0x7fe474004730 comp rx=0 tx=0).stop 2024-09-21T06:17:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 shutdown_connections 2024-09-21T06:17:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e51491d80 0x558e5189e8b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51898b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514b24d0 0x558e5188cf30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 --2- 172.21.15.31:0/4113454057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e5188c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 >> 172.21.15.31:0/4113454057 conn(0x558e514c7300 msgr2=0x558e51874ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe4697fa640 0 client.26489 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 shutdown_connections 2024-09-21T06:17:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 1 -- 172.21.15.31:0/4113454057 wait complete. 2024-09-21T06:17:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.680+0000 7fe48f09c740 20 client.26489 trim_cache size 0 max 16384 2024-09-21T06:17:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.707+0000 7fe48f09c740 1 Processor -- start 2024-09-21T06:17:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.707+0000 7fe48f09c740 1 -- start start 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e518880a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514b24d0 0x558e51886990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514d31b0 0x558e51898a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe48f09c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b3ca0 con 0x558e514b24d0 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe48f09c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b6520 con 0x558e514d31b0 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe48f09c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b4310 con 0x558e514aa2f0 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e518880a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe4866c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514d31b0 0x558e51898a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e518880a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42784/0 (socket says 172.21.15.31:42784) 2024-09-21T06:17:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 -- 172.21.15.31:0/3880794176 learned_addr learned my addr 172.21.15.31:0/3880794176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe4856c6640 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514b24d0 0x558e51886990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 -- 172.21.15.31:0/3880794176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514d31b0 msgr2=0x558e51898a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514d31b0 0x558e51898a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 -- 172.21.15.31:0/3880794176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514b24d0 msgr2=0x558e51886990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514b24d0 0x558e51886990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.708+0000 7fe485ec7640 1 -- 172.21.15.31:0/3880794176 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e51875970 con 0x558e514aa2f0 2024-09-21T06:17:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.709+0000 7fe485ec7640 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e518880a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0ad5e0 tx=0x7fe47c0a98b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650314b1c67c7faa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.709+0000 7fe46b7fe640 1 -- 172.21.15.31:0/3880794176 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0aeb60 con 0x558e514aa2f0 2024-09-21T06:17:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.709+0000 7fe46b7fe640 1 -- 172.21.15.31:0/3880794176 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47c0aed00 con 0x558e514aa2f0 2024-09-21T06:17:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.709+0000 7fe46b7fe640 1 -- 172.21.15.31:0/3880794176 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0a9490 con 0x558e514aa2f0 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 -- 172.21.15.31:0/3880794176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 msgr2=0x558e518880a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e518880a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0ad5e0 tx=0x7fe47c0a98b0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 -- 172.21.15.31:0/3880794176 shutdown_connections 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514d31b0 0x558e51898a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514b24d0 0x558e51886990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3880794176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514aa2f0 0x558e518880a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 -- 172.21.15.31:0/3880794176 >> 172.21.15.31:0/3880794176 conn(0x558e514c7300 msgr2=0x558e51886280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 -- 172.21.15.31:0/3880794176 shutdown_connections 2024-09-21T06:17:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 -- 172.21.15.31:0/3880794176 wait complete. 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 -- start start 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.711+0000 7fe48f09c740 1 Processor -- start 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 0x558e51894ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514b24d0 0x558e518950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51902740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b49e0 con 0x558e514d31b0 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b5e50 con 0x558e514aa2f0 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801b50b0 con 0x558e514b24d0 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe48f09c740 10 client.0 _ll_register_callbacks cb 0x7fe47400bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 0x558e51894ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe485ec7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 0x558e51894ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32864/0 (socket says 172.21.15.31:32864) 2024-09-21T06:17:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.712+0000 7fe485ec7640 1 -- 172.21.15.31:0/3568741198 learned_addr learned my addr 172.21.15.31:0/3568741198 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe4856c6640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514b24d0 0x558e518950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe4866c8640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51902740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe485ec7640 1 -- 172.21.15.31:0/3568741198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514b24d0 msgr2=0x558e518950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe485ec7640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514b24d0 0x558e518950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe485ec7640 1 -- 172.21.15.31:0/3568741198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 msgr2=0x558e51902740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe485ec7640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51902740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe485ec7640 1 -- 172.21.15.31:0/3568741198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e51902c80 con 0x558e514aa2f0 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe485ec7640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 0x558e51894ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0a9d90 tx=0x7fe47c00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565fafe46eac24bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:07.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe4697fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe4697fa640 1 -- 172.21.15.31:0/3568741198 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0aeb60 con 0x558e514aa2f0 2024-09-21T06:17:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.713+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47c0aed00 con 0x558e514aa2f0 2024-09-21T06:17:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.714+0000 7fe4697fa640 1 -- 172.21.15.31:0/3568741198 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe47c0acd50 con 0x558e514aa2f0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.714+0000 7fe4697fa640 1 -- 172.21.15.31:0/3568741198 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe47c0cf320 con 0x558e514aa2f0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.714+0000 7fe48f09c740 10 client.26495 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.714+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558e5187a690 con 0x558e514aa2f0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.714+0000 7fe4697fa640 1 -- 172.21.15.31:0/3568741198 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe47c0aed00 con 0x558e514aa2f0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.714+0000 7fe4697fa640 1 -- 172.21.15.31:0/3568741198 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe47c0ac5b0 con 0x558e514aa2f0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe48f09c740 10 client.26495 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe48f09c740 10 client.26495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe48f09c740 20 client.26495 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e514671c0 0x558e5187c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe48f09c740 4 client.26495 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e517eb0f0 con 0x558e514671c0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe4856c6640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e514671c0 0x558e5187c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.715+0000 7fe4697fa640 10 client.26495 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.716+0000 7fe4856c6640 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e514671c0 0x558e5187c9e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cacbcd3ce918e0 server_cookie=e778d78dbf3f76f8 in_seq=0 out_seq=0 2024-09-21T06:17:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe4697fa640 1 -- 172.21.15.31:0/3568741198 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e517eb0f0 con 0x558e514671c0 2024-09-21T06:17:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe4697fa640 10 client.26495 handle_command_reply: tid=0 2024-09-21T06:17:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 client.26495 shutdown 2024-09-21T06:17:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 msgr2=0x558e51894ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 0x558e51894ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0a9d90 tx=0x7fe47c00ce60 comp rx=0 tx=0).stop 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 shutdown_connections 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558e514671c0 0x558e5187c9e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558e514d31b0 0x558e51902740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558e514b24d0 0x558e518950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 --2- 172.21.15.31:0/3568741198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558e514aa2f0 0x558e51894ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe4697fa640 0 client.26495 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 >> 172.21.15.31:0/3568741198 conn(0x558e514c7300 msgr2=0x558e51882bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 shutdown_connections 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 1 -- 172.21.15.31:0/3568741198 wait complete. 2024-09-21T06:17:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.739+0000 7fe48f09c740 20 client.26495 trim_cache size 0 max 16384 2024-09-21T06:17:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:17:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:17:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:17:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe46c0b6700 msgr2=0x7fe46c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe46c0b6700 0x7fe46c0b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe48007ab90 tx=0x7fe4740108b0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 msgr2=0x7fe4800750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 0x7fe4800750a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e9f0 tx=0x7fe47c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 shutdown_connections 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe46c0b6700 0x7fe46c0b8c10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe480108c30 0x7fe480073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4801082e0 0x7fe480073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 --2- 172.21.15.31:0/2825254057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4801029b0 0x7fe4800750a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 >> 172.21.15.31:0/2825254057 conn(0x7fe48007a740 msgr2=0x7fe480106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 shutdown_connections 2024-09-21T06:17:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:07.740+0000 7fe48812b640 1 -- 172.21.15.31:0/2825254057 wait complete. 2024-09-21T06:17:07.942 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:17:07.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:17:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.904+0000 7f330cfd2640 1 -- 172.21.15.31:0/4294923671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330806a0e0 msgr2=0x7f3308108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.904+0000 7f330cfd2640 1 --2- 172.21.15.31:0/4294923671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330806a0e0 0x7f3308108030 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3300009ca0 tx=0x7f330002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 -- 172.21.15.31:0/4294923671 shutdown_connections 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 --2- 172.21.15.31:0/4294923671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330806a0e0 0x7f3308108030 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 --2- 172.21.15.31:0/4294923671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33081076e0 0x7f3308107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 --2- 172.21.15.31:0/4294923671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3308101da0 0x7f33081021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 -- 172.21.15.31:0/4294923671 >> 172.21.15.31:0/4294923671 conn(0x7f3308077800 msgr2=0x7f3308077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 -- 172.21.15.31:0/4294923671 shutdown_connections 2024-09-21T06:17:08.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.905+0000 7f330cfd2640 1 -- 172.21.15.31:0/4294923671 wait complete. 2024-09-21T06:17:08.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.906+0000 7f330cfd2640 1 Processor -- start 2024-09-21T06:17:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 -- start start 2024-09-21T06:17:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330806a0e0 0x7f33081990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3308101da0 0x7f33081995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 0x7f33081a4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3308113be0 con 0x7f3308101da0 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3308113a60 con 0x7f330806a0e0 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330cfd2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3308113d60 con 0x7f33081076e0 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 0x7f33081a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 0x7f33081a4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42812/0 (socket says 172.21.15.31:42812) 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.907+0000 7f330659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330806a0e0 0x7f33081990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.908+0000 7f330659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330806a0e0 0x7f33081990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32880/0 (socket says 172.21.15.31:32880) 2024-09-21T06:17:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.908+0000 7f330659c640 1 -- 172.21.15.31:0/2998638685 learned_addr learned my addr 172.21.15.31:0/2998638685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.909+0000 7f3306d9d640 1 -- 172.21.15.31:0/2998638685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330806a0e0 msgr2=0x7f33081990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.909+0000 7f3306d9d640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330806a0e0 0x7f33081990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.909+0000 7f3306d9d640 1 -- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3308101da0 msgr2=0x7f33081995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.909+0000 7f3306d9d640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3308101da0 0x7f33081995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.909+0000 7f3306d9d640 1 -- 172.21.15.31:0/2998638685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3308199e90 con 0x7f33081076e0 2024-09-21T06:17:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.910+0000 7f3306d9d640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 0x7f33081a4130 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f330002aee0 tx=0x7f3300003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.911+0000 7f32f77fe640 1 -- 172.21.15.31:0/2998638685 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300004040 con 0x7f33081076e0 2024-09-21T06:17:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.911+0000 7f32f77fe640 1 -- 172.21.15.31:0/2998638685 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f330002d070 con 0x7f33081076e0 2024-09-21T06:17:08.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.912+0000 7f32f77fe640 1 -- 172.21.15.31:0/2998638685 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300030d20 con 0x7f33081076e0 2024-09-21T06:17:08.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.912+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33081a46d0 con 0x7f33081076e0 2024-09-21T06:17:08.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.912+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a4c10 con 0x7f33081076e0 2024-09-21T06:17:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.918+0000 7f3313546740 1 Processor -- start 2024-09-21T06:17:08.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.921+0000 7f3313546740 1 -- start start 2024-09-21T06:17:08.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.921+0000 7f32f77fe640 1 -- 172.21.15.31:0/2998638685 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f33000041e0 con 0x7f33081076e0 2024-09-21T06:17:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.922+0000 7f3313546740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce33350 0x5566fd1da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.922+0000 7f3313546740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f330659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f330659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1da970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42816/0 (socket says 172.21.15.31:42816) 2024-09-21T06:17:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f3313546740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1daeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f3313546740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5566fd150e60 con 0x5566fce12670 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f3313546740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5566fd140290 con 0x5566fce33350 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f3313546740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5566fd1440a0 con 0x5566fce0a490 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f3305d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce33350 0x5566fd1da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.923+0000 7f330659c640 1 -- 172.21.15.31:0/2064535870 learned_addr learned my addr 172.21.15.31:0/2064535870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.924+0000 7f3305d9b640 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1daeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.924+0000 7f3305d9b640 1 -- 172.21.15.31:0/2064535870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 msgr2=0x5566fd1da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.924+0000 7f3305d9b640 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1da970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.924+0000 7f3305d9b640 1 -- 172.21.15.31:0/2064535870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 msgr2=0x5566fd1daeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.924+0000 7f3305d9b640 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1daeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.924+0000 7f3305d9b640 1 -- 172.21.15.31:0/2064535870 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fce9e930 con 0x5566fce33350 2024-09-21T06:17:08.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.925+0000 7f32f77fe640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f32f00b66a0 0x7f32f00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.925+0000 7f3305d9b640 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce33350 0x5566fd1da430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f330806cc00 tx=0x7f32fc009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25975382cd7a9514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.925+0000 7f330659c640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f32f00b66a0 0x7f32f00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.926+0000 7f32f77fe640 1 -- 172.21.15.31:0/2998638685 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f3300108d70 con 0x7f33081076e0 2024-09-21T06:17:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.926+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2064535870 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00c050 con 0x5566fce33350 2024-09-21T06:17:08.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.926+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2064535870 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32fc010040 con 0x5566fce33350 2024-09-21T06:17:08.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.927+0000 7f3313546740 1 -- 172.21.15.31:0/2064535870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce33350 msgr2=0x5566fd1da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.927+0000 7f3313546740 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce33350 0x5566fd1da430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f330806cc00 tx=0x7f32fc009230 comp rx=0 tx=0).stop 2024-09-21T06:17:08.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.927+0000 7f3313546740 1 -- 172.21.15.31:0/2064535870 shutdown_connections 2024-09-21T06:17:08.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.927+0000 7f3313546740 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1daeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.927+0000 7f3313546740 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.928+0000 7f3313546740 1 --2- 172.21.15.31:0/2064535870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce33350 0x5566fd1da430 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.928+0000 7f3313546740 1 -- 172.21.15.31:0/2064535870 >> 172.21.15.31:0/2064535870 conn(0x5566fce27740 msgr2=0x5566fd1d8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.928+0000 7f3313546740 1 -- 172.21.15.31:0/2064535870 shutdown_connections 2024-09-21T06:17:08.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.928+0000 7f3313546740 1 -- 172.21.15.31:0/2064535870 wait complete. 2024-09-21T06:17:08.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.929+0000 7f330659c640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f32f00b66a0 0x7f32f00b8bb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5566fd1d90b0 tx=0x7f32f8008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.929+0000 7f3313546740 1 -- start start 2024-09-21T06:17:08.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.929+0000 7f3313546740 1 Processor -- start 2024-09-21T06:17:08.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.930+0000 7f3313546740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 0x5566fd1e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f330659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 0x5566fd1e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f330659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 0x5566fd1e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:32900/0 (socket says 172.21.15.31:32900) 2024-09-21T06:17:08.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3313546740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f330659c640 1 -- 172.21.15.31:0/3700850040 learned_addr learned my addr 172.21.15.31:0/3700850040 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:08.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3305d9b640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5566fcfa1890 con 0x5566fce12670 2024-09-21T06:17:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5566fd0dad40 con 0x5566fce0a490 2024-09-21T06:17:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5566fd1e8c60 con 0x5566fce33350 2024-09-21T06:17:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.931+0000 7f3313546740 10 client.0 _ll_register_callbacks cb 0x5566fd13ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 -- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 msgr2=0x5566fd1e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 0x5566fd1e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 -- 172.21.15.31:0/3700850040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 msgr2=0x5566fd1e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fd1f24e0 con 0x5566fce33350 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f330659c640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 0x5566fd1e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f32edffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.932+0000 7f3306d9d640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1f1fa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33001064a0 tx=0x7f33000ceaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4176a681a381afae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.933+0000 7f32edffb640 1 -- 172.21.15.31:0/3700850040 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300106920 con 0x5566fce33350 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.933+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32fc010040 con 0x5566fce33350 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.933+0000 7f32edffb640 1 -- 172.21.15.31:0/3700850040 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f3300106720 con 0x5566fce33350 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.934+0000 7f32edffb640 1 -- 172.21.15.31:0/3700850040 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f33000305f0 con 0x5566fce33350 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.934+0000 7f3313546740 10 client.26551 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.934+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5566fd1f4de0 con 0x5566fce33350 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.934+0000 7f32edffb640 1 -- 172.21.15.31:0/3700850040 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32fc010040 con 0x5566fce33350 2024-09-21T06:17:08.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.934+0000 7f32edffb640 1 -- 172.21.15.31:0/3700850040 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f33000d8130 con 0x5566fce33350 2024-09-21T06:17:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f3313546740 10 client.26551 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f3313546740 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f3313546740 20 client.26551 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdf1cd0 0x5566fd1f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f3313546740 4 client.26551 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566fd152db0 con 0x5566fcdf1cd0 2024-09-21T06:17:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.935+0000 7f330659c640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdf1cd0 0x5566fd1f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.936+0000 7f32edffb640 10 client.26551 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.936+0000 7f330659c640 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdf1cd0 0x5566fd1f82a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a07ba5e7a9ca45b server_cookie=4f41489ed109b2db in_seq=0 out_seq=0 2024-09-21T06:17:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f32edffb640 1 -- 172.21.15.31:0/3700850040 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566fd152db0 con 0x5566fcdf1cd0 2024-09-21T06:17:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f32edffb640 10 client.26551 handle_command_reply: tid=0 2024-09-21T06:17:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 client.26551 shutdown 2024-09-21T06:17:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 msgr2=0x5566fd1f1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1f1fa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33001064a0 tx=0x7f33000ceaa0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 shutdown_connections 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdf1cd0 0x5566fd1f82a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1f1fa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f32edffb640 0 client.26551 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce12670 0x5566fd1e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 --2- 172.21.15.31:0/3700850040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce0a490 0x5566fd1e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 >> 172.21.15.31:0/3700850040 conn(0x5566fce27740 msgr2=0x5566fd1dd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 shutdown_connections 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.937+0000 7f3313546740 1 -- 172.21.15.31:0/3700850040 wait complete. 2024-09-21T06:17:08.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.938+0000 7f3313546740 20 client.26551 trim_cache size 0 max 16384 2024-09-21T06:17:08.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.967+0000 7f3313546740 1 Processor -- start 2024-09-21T06:17:08.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.967+0000 7f3313546740 1 -- start start 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3313546740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 0x5566fd1e0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3313546740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3313546740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3313546740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33080725d0 con 0x5566fce0a490 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3313546740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3308074eb0 con 0x5566fce12670 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3313546740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3308072ca0 con 0x5566fce33350 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 0x5566fd1e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 0x5566fd1e0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38890/0 (socket says 172.21.15.31:38890) 2024-09-21T06:17:08.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 -- 172.21.15.31:0/1723221936 learned_addr learned my addr 172.21.15.31:0/1723221936 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3306d9d640 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f3305d9b640 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 -- 172.21.15.31:0/1723221936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 msgr2=0x5566fd1e2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1e2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 -- 172.21.15.31:0/1723221936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 msgr2=0x5566fd1e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1e7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.968+0000 7f330659c640 1 -- 172.21.15.31:0/1723221936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fd1e4ca0 con 0x5566fce0a490 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.969+0000 7f330659c640 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 0x5566fd1e0550 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007670 tx=0x7f32f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c700f76305535e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.969+0000 7f32effff640 1 -- 172.21.15.31:0/1723221936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f802f070 con 0x5566fce0a490 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.969+0000 7f32effff640 1 -- 172.21.15.31:0/1723221936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f801d040 con 0x5566fce0a490 2024-09-21T06:17:08.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.969+0000 7f32effff640 1 -- 172.21.15.31:0/1723221936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f801c040 con 0x5566fce0a490 2024-09-21T06:17:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 -- 172.21.15.31:0/1723221936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 msgr2=0x5566fd1e0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 0x5566fd1e0550 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007670 tx=0x7f32f8016000 comp rx=0 tx=0).stop 2024-09-21T06:17:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 -- 172.21.15.31:0/1723221936 shutdown_connections 2024-09-21T06:17:08.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce33350 0x5566fd1e2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 --2- 172.21.15.31:0/1723221936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce0a490 0x5566fd1e0550 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 -- 172.21.15.31:0/1723221936 >> 172.21.15.31:0/1723221936 conn(0x5566fce27740 msgr2=0x5566fd1e7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 -- 172.21.15.31:0/1723221936 shutdown_connections 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.970+0000 7f3313546740 1 -- 172.21.15.31:0/1723221936 wait complete. 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.971+0000 7f3313546740 1 -- start start 2024-09-21T06:17:08.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.971+0000 7f3313546740 1 Processor -- start 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce33350 0x5566fd1eaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3308073370 con 0x5566fce33350 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f33080747e0 con 0x5566fce12670 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3308073a40 con 0x5566fce0a490 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f3313546740 10 client.0 _ll_register_callbacks cb 0x7f3300009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f330659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f330659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1ee460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42850/0 (socket says 172.21.15.31:42850) 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.972+0000 7f330659c640 1 -- 172.21.15.31:0/3979653109 learned_addr learned my addr 172.21.15.31:0/3979653109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:08.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.973+0000 7f330659c640 1 -- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 msgr2=0x5566fd1ea9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:17:08.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.973+0000 7f330659c640 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1ea9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.973+0000 7f330659c640 1 -- 172.21.15.31:0/3979653109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce33350 msgr2=0x5566fd1eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:08.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.973+0000 7f330659c640 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce33350 0x5566fd1eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.973+0000 7f330659c640 1 -- 172.21.15.31:0/3979653109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fd1eb440 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f330659c640 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1ee460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32f8000c00 tx=0x7f32f8023660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb956bae2f514d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f32edffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f32edffb640 1 -- 172.21.15.31:0/3979653109 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8038060 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f32edffb640 1 -- 172.21.15.31:0/3979653109 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f32f8033450 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f801d040 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f32edffb640 1 -- 172.21.15.31:0/3979653109 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f32f802f070 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f3313546740 10 client.26557 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.974+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5566fd263980 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f32edffb640 1 -- 172.21.15.31:0/3979653109 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f801d040 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f32edffb640 1 -- 172.21.15.31:0/3979653109 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f32f8013480 con 0x5566fce0a490 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f3313546740 10 client.26557 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f3313546740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:08.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f3313546740 20 client.26557 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f3313546740 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdc7080 0x5566fd2660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f3313546740 4 client.26557 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.975+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5566fd14e860 con 0x5566fcdc7080 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.976+0000 7f3305d9b640 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdc7080 0x5566fd2660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.976+0000 7f32edffb640 10 client.26557 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.976+0000 7f3305d9b640 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdc7080 0x5566fd2660c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993aebbb71ab7f6b server_cookie=3f1c6ff3cb82fc91 in_seq=0 out_seq=0 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.977+0000 7f32edffb640 1 -- 172.21.15.31:0/3979653109 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5566fd14e860 con 0x5566fcdc7080 2024-09-21T06:17:08.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.977+0000 7f32edffb640 10 client.26557 handle_command_reply: tid=0 2024-09-21T06:17:08.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.978+0000 7f3313546740 1 client.26557 shutdown 2024-09-21T06:17:08.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.978+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 msgr2=0x5566fd1ee460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.978+0000 7f3313546740 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1ee460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32f8000c00 tx=0x7f32f8023660 comp rx=0 tx=0).stop 2024-09-21T06:17:08.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 shutdown_connections 2024-09-21T06:17:08.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566fcdc7080 0x5566fd2660c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566fce33350 0x5566fd1eaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f32edffb640 0 client.26557 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:08.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566fce12670 0x5566fd1ea9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 --2- 172.21.15.31:0/3979653109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566fce0a490 0x5566fd1ee460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 >> 172.21.15.31:0/3979653109 conn(0x5566fce27740 msgr2=0x5566fd1dbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 shutdown_connections 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 1 -- 172.21.15.31:0/3979653109 wait complete. 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.979+0000 7f3313546740 20 client.26557 trim_cache size 0 max 16384 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:17:08.983 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "3fc27b03-07a0-480b-8d21-34b6a99891c5", 2024-09-21T06:17:08.984 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:17:08.984 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f32f00b66a0 msgr2=0x7f32f00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f32f00b66a0 0x7f32f00b8bb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5566fd1d90b0 tx=0x7f32f8008040 comp rx=0 tx=0).stop 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 msgr2=0x7f33081a4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 0x7f33081a4130 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f330002aee0 tx=0x7f3300003c10 comp rx=0 tx=0).stop 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 shutdown_connections 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33081076e0 0x7f33081a4130 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f32f00b66a0 0x7f32f00b8bb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3308101da0 0x7f33081995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 --2- 172.21.15.31:0/2998638685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330806a0e0 0x7f33081990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.980+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 >> 172.21.15.31:0/2998638685 conn(0x7f3308077800 msgr2=0x7f33081044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.981+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 shutdown_connections 2024-09-21T06:17:08.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:08.981+0000 7f330cfd2640 1 -- 172.21.15.31:0/2998638685 wait complete. 2024-09-21T06:17:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:09 smithi110 ceph-mon[34550]: pgmap v486: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 34 MiB/s wr, 1.29k op/s 2024-09-21T06:17:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:09 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:17:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:09 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:09 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:17:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:09 smithi107 ceph-mon[34380]: pgmap v486: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 34 MiB/s wr, 1.29k op/s 2024-09-21T06:17:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:09 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:17:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:09 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:09 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:17:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:09 smithi031 ceph-mon[31237]: pgmap v486: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 34 MiB/s wr, 1.29k op/s 2024-09-21T06:17:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:09 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:17:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:09 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:09.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:09 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:17:09.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 330.0 s, 1904.9 tps, lat 67.034 ms stddev 259.656 2024-09-21T06:17:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:10 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:10.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:10 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 -- 172.21.15.31:0/3321952474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e40760a0 msgr2=0x7f61e410fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 --2- 172.21.15.31:0/3321952474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e40760a0 0x7f61e410fc00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009900 tx=0x7f61cc02a420 comp rx=0 tx=0).stop 2024-09-21T06:17:10.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 -- 172.21.15.31:0/3321952474 shutdown_connections 2024-09-21T06:17:10.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 --2- 172.21.15.31:0/3321952474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e4110140 0x7f61e4112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 --2- 172.21.15.31:0/3321952474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e40760a0 0x7f61e410fc00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 --2- 172.21.15.31:0/3321952474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 0x7f61e4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 -- 172.21.15.31:0/3321952474 >> 172.21.15.31:0/3321952474 conn(0x7f61e40ff610 msgr2=0x7f61e4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 -- 172.21.15.31:0/3321952474 shutdown_connections 2024-09-21T06:17:10.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.680+0000 7f61ea681640 1 -- 172.21.15.31:0/3321952474 wait complete. 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 Processor -- start 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 -- start start 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 0x7f61e419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e40760a0 0x7f61e419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4110140 0x7f61e41a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4115d00 con 0x7f61e4110140 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4115b80 con 0x7f61e40760a0 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61ea681640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4115e80 con 0x7f61e4075750 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61e8c1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4110140 0x7f61e41a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 0x7f61e419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61e8c1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4110140 0x7f61e41a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34204/0 (socket says 172.21.15.31:34204) 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61e8c1e640 1 -- 172.21.15.31:0/786180620 learned_addr learned my addr 172.21.15.31:0/786180620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.681+0000 7f61dbfff640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e40760a0 0x7f61e419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e3fff640 1 -- 172.21.15.31:0/786180620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e40760a0 msgr2=0x7f61e419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e3fff640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e40760a0 0x7f61e419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e3fff640 1 -- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4110140 msgr2=0x7f61e41a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e3fff640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4110140 0x7f61e41a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e3fff640 1 -- 172.21.15.31:0/786180620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e41a2600 con 0x7f61e4075750 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e3fff640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 0x7f61e419c690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f61d400d6d0 tx=0x7f61d400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61e1ffb640 1 -- 172.21.15.31:0/786180620 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017070 con 0x7f61e4075750 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.682+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e41a28f0 con 0x7f61e4075750 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.683+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a9050 con 0x7f61e4075750 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.683+0000 7f61e1ffb640 1 -- 172.21.15.31:0/786180620 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4009820 con 0x7f61e4075750 2024-09-21T06:17:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.683+0000 7f61e1ffb640 1 -- 172.21.15.31:0/786180620 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4004c40 con 0x7f61e4075750 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 Processor -- start 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 -- start start 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9d93e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 0x55e6d9d948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9d01e50 con 0x55e6d99bb1b0 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9cf1280 con 0x55e6d99e4070 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9cf5090 con 0x55e6d99c3390 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 0x55e6d9d948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 0x55e6d9d948e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:56984/0 (socket says 172.21.15.31:56984) 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61dbfff640 1 -- 172.21.15.31:0/790042910 learned_addr learned my addr 172.21.15.31:0/790042910 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.686+0000 7f61e8c1e640 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9d93e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61e1ffb640 1 -- 172.21.15.31:0/786180620 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f61d4005220 con 0x7f61e4075750 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61dbfff640 1 -- 172.21.15.31:0/790042910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 msgr2=0x55e6d9d93e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61dbfff640 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9d93e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61dbfff640 1 -- 172.21.15.31:0/790042910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 msgr2=0x55e6d9d943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61dbfff640 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61dbfff640 1 -- 172.21.15.31:0/790042910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d9a4f650 con 0x55e6d99c3390 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.687+0000 7f61dbfff640 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 0x55e6d9d948e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61cc02aa10 tx=0x7f61cc004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7577b93bb8670015 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.688+0000 7f61da7fc640 1 -- 172.21.15.31:0/790042910 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc035040 con 0x55e6d99c3390 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.688+0000 7f61da7fc640 1 -- 172.21.15.31:0/790042910 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc004700 con 0x55e6d99c3390 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.688+0000 7f61da7fc640 1 -- 172.21.15.31:0/790042910 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0037f0 con 0x55e6d99c3390 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.688+0000 7f61e1ffb640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f61d00b6730 0x7f61d00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.689+0000 7f61e1ffb640 1 -- 172.21.15.31:0/786180620 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f61d40e6f70 con 0x7f61e4075750 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.689+0000 7f61e8c1e640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f61d00b6730 0x7f61d00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.690+0000 7f61f0bf5740 1 -- 172.21.15.31:0/790042910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 msgr2=0x55e6d9d948e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.690+0000 7f61e8c1e640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f61d00b6730 0x7f61d00b8c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009270 tx=0x7f61dc009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.690+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 0x55e6d9d948e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61cc02aa10 tx=0x7f61cc004310 comp rx=0 tx=0).stop 2024-09-21T06:17:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.690+0000 7f61f0bf5740 1 -- 172.21.15.31:0/790042910 shutdown_connections 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.691+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99c3390 0x55e6d9d948e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.691+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.691+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/790042910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9d93e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.691+0000 7f61f0bf5740 1 -- 172.21.15.31:0/790042910 >> 172.21.15.31:0/790042910 conn(0x55e6d99d8460 msgr2=0x55e6d9d91d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.692+0000 7f61f0bf5740 1 -- 172.21.15.31:0/790042910 shutdown_connections 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.692+0000 7f61f0bf5740 1 -- 172.21.15.31:0/790042910 wait complete. 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 -- start start 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 Processor -- start 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9d9fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9dab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9b52700 con 0x55e6d99bb1b0 2024-09-21T06:17:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9c8c110 con 0x55e6d99c3390 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e6d9da25e0 con 0x55e6d99e4070 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61f0bf5740 10 client.0 _ll_register_callbacks cb 0x55e6d9ca5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9d9fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9d9fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44540/0 (socket says 172.21.15.31:44540) 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 -- 172.21.15.31:0/53299920 learned_addr learned my addr 172.21.15.31:0/53299920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.693+0000 7f61e8c1e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9dab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61e3fff640 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 -- 172.21.15.31:0/53299920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 msgr2=0x55e6d9dab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9dab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 -- 172.21.15.31:0/53299920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 msgr2=0x55e6d9d9f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 -- 172.21.15.31:0/53299920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d9cb9c40 con 0x55e6d99c3390 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61dbfff640 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9d9fae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004ae0 tx=0x7f61cc02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776cbdbf2aa968 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61c37fe640 1 -- 172.21.15.31:0/53299920 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc053060 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.694+0000 7f61c37fe640 1 -- 172.21.15.31:0/53299920 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f61cc005000 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc004700 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61c37fe640 1 -- 172.21.15.31:0/53299920 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f61cc035050 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 10 client.26516 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e6d9dae320 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61c37fe640 1 -- 172.21.15.31:0/53299920 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc004700 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61c37fe640 1 -- 172.21.15.31:0/53299920 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f61cc031c70 con 0x55e6d99c3390 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 10 client.26516 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 20 client.26516 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d99a29f0 0x55e6d9db1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 4 client.26516 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.695+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6d9d03da0 con 0x55e6d99a29f0 2024-09-21T06:17:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.696+0000 7f61e3fff640 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d99a29f0 0x55e6d9db1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.696+0000 7f61c37fe640 10 client.26516 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.696+0000 7f61e3fff640 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d99a29f0 0x55e6d9db1680 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c794cbbed548bdc6 server_cookie=322666852590682f in_seq=0 out_seq=0 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.697+0000 7f61c37fe640 1 -- 172.21.15.31:0/53299920 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6d9d03da0 con 0x55e6d99a29f0 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.697+0000 7f61c37fe640 10 client.26516 handle_command_reply: tid=0 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.697+0000 7f61f0bf5740 1 client.26516 shutdown 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.697+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 msgr2=0x55e6d9d9fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.697+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9d9fae0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004ae0 tx=0x7f61cc02a420 comp rx=0 tx=0).stop 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 shutdown_connections 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d99a29f0 0x55e6d9db1680 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9dab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9d9fae0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61c37fe640 0 client.26516 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/53299920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 >> 172.21.15.31:0/53299920 conn(0x55e6d99d8460 msgr2=0x55e6d9d97040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 shutdown_connections 2024-09-21T06:17:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.698+0000 7f61f0bf5740 1 -- 172.21.15.31:0/53299920 wait complete. 2024-09-21T06:17:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.699+0000 7f61f0bf5740 20 client.26516 trim_cache size 0 max 16384 2024-09-21T06:17:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.723+0000 7f61f0bf5740 1 Processor -- start 2024-09-21T06:17:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 -- start start 2024-09-21T06:17:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9dab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9d9c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ab540 con 0x55e6d99bb1b0 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41adce0 con 0x55e6d99c3390 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.724+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41abb90 con 0x55e6d99e4070 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9dab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9dab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44552/0 (socket says 172.21.15.31:44552) 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 -- 172.21.15.31:0/3058277841 learned_addr learned my addr 172.21.15.31:0/3058277841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61e8c1e640 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9d9c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 -- 172.21.15.31:0/3058277841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 msgr2=0x55e6d9d9c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9d9c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 -- 172.21.15.31:0/3058277841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 msgr2=0x55e6d9d9fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 -- 172.21.15.31:0/3058277841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d9d9e500 con 0x55e6d99c3390 2024-09-21T06:17:10.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61dbfff640 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9dab840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f61cc02aa10 tx=0x7f61cc0059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a474e699dbcdeac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.725+0000 7f61d9ffb640 1 -- 172.21.15.31:0/3058277841 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc053030 con 0x55e6d99c3390 2024-09-21T06:17:10.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.726+0000 7f61d9ffb640 1 -- 172.21.15.31:0/3058277841 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc005df0 con 0x55e6d99c3390 2024-09-21T06:17:10.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.726+0000 7f61d9ffb640 1 -- 172.21.15.31:0/3058277841 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0055d0 con 0x55e6d99c3390 2024-09-21T06:17:10.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 -- 172.21.15.31:0/3058277841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 msgr2=0x55e6d9dab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9dab840 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f61cc02aa10 tx=0x7f61cc0059d0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 -- 172.21.15.31:0/3058277841 shutdown_connections 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99e4070 0x55e6d9d9c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99c3390 0x55e6d9dab840 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/3058277841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99bb1b0 0x55e6d9d9fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 -- 172.21.15.31:0/3058277841 >> 172.21.15.31:0/3058277841 conn(0x55e6d99d8460 msgr2=0x55e6d9d9f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 -- 172.21.15.31:0/3058277841 shutdown_connections 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 -- 172.21.15.31:0/3058277841 wait complete. 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.727+0000 7f61f0bf5740 1 -- start start 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 Processor -- start 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 0x55e6d9db2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99c3390 0x55e6d9db2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9db3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ac1e0 con 0x55e6d99c3390 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ad610 con 0x55e6d99e4070 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41ac870 con 0x55e6d99bb1b0 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61f0bf5740 10 client.0 _ll_register_callbacks cb 0x7f61dc009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 0x55e6d9db2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99c3390 0x55e6d9db2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 0x55e6d9db2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:57006/0 (socket says 172.21.15.31:57006) 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61e3fff640 1 -- 172.21.15.31:0/4273753604 learned_addr learned my addr 172.21.15.31:0/4273753604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.728+0000 7f61e8c1e640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9db3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e3fff640 1 -- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 msgr2=0x55e6d9db3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e3fff640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9db3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e3fff640 1 -- 172.21.15.31:0/4273753604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99c3390 msgr2=0x55e6d9db2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:10.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e3fff640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99c3390 0x55e6d9db2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e3fff640 1 -- 172.21.15.31:0/4273753604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d9e1b000 con 0x55e6d99bb1b0 2024-09-21T06:17:10.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e8c1e640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9db3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:10.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61e3fff640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 0x55e6d9db2880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009610 tx=0x7f61d40accd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760be866375f6276 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:10.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:10.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61c37fe640 1 -- 172.21.15.31:0/4273753604 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40b0070 con 0x55e6d99bb1b0 2024-09-21T06:17:10.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61c37fe640 1 -- 172.21.15.31:0/4273753604 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f61d4009cd0 con 0x55e6d99bb1b0 2024-09-21T06:17:10.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.729+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc005df0 con 0x55e6d99bb1b0 2024-09-21T06:17:10.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.730+0000 7f61c37fe640 1 -- 172.21.15.31:0/4273753604 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f61d4020040 con 0x55e6d99bb1b0 2024-09-21T06:17:10.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.730+0000 7f61f0bf5740 10 client.26575 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:10.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.730+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e6d9e1c2d0 con 0x55e6d99bb1b0 2024-09-21T06:17:10.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.730+0000 7f61c37fe640 1 -- 172.21.15.31:0/4273753604 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc005df0 con 0x55e6d99bb1b0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.730+0000 7f61c37fe640 1 -- 172.21.15.31:0/4273753604 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f61d40a90c0 con 0x55e6d99bb1b0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.731+0000 7f61f0bf5740 10 client.26575 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.731+0000 7f61f0bf5740 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.731+0000 7f61f0bf5740 20 client.26575 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.731+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d9977da0 0x55e6d9e1e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.732+0000 7f61f0bf5740 4 client.26575 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.732+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6d9cff850 con 0x55e6d9977da0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.732+0000 7f61dbfff640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d9977da0 0x55e6d9e1e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61dbfff640 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d9977da0 0x55e6d9e1e9f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d69ab0cf315755b server_cookie=3f1db318713c129f in_seq=0 out_seq=0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61c37fe640 10 client.26575 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61c37fe640 1 -- 172.21.15.31:0/4273753604 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x55e6d9cff850 con 0x55e6d9977da0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61c37fe640 10 client.26575 handle_command_reply: tid=0 2024-09-21T06:17:10.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 client.26575 shutdown 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 msgr2=0x55e6d9db2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 0x55e6d9db2880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009610 tx=0x7f61d40accd0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 shutdown_connections 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e6d9977da0 0x55e6d9e1e9f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6d99e4070 0x55e6d9db3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6d99c3390 0x55e6d9db2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 --2- 172.21.15.31:0/4273753604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6d99bb1b0 0x55e6d9db2880 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.733+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 >> 172.21.15.31:0/4273753604 conn(0x55e6d99d8460 msgr2=0x55e6d9d94e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.734+0000 7f61c37fe640 0 client.26575 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.734+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 shutdown_connections 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.734+0000 7f61f0bf5740 1 -- 172.21.15.31:0/4273753604 wait complete. 2024-09-21T06:17:10.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.734+0000 7f61f0bf5740 20 client.26575 trim_cache size 0 max 16384 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "3fc27b03-07a0-480b-8d21-34b6a99891c5", 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:17:10.742 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:17:10.745 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:17:10.747 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.748 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:17:10.752 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f61d00b6730 msgr2=0x7f61d00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f61d00b6730 0x7f61d00b8c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009270 tx=0x7f61dc009680 comp rx=0 tx=0).stop 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 msgr2=0x7f61e419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 0x7f61e419c690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f61d400d6d0 tx=0x7f61d400db90 comp rx=0 tx=0).stop 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 shutdown_connections 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f61d00b6730 0x7f61d00b8c40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61e4110140 0x7f61e41a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61e40760a0 0x7f61e419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 --2- 172.21.15.31:0/786180620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61e4075750 0x7f61e419c690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 >> 172.21.15.31:0/786180620 conn(0x7f61e40ff610 msgr2=0x7f61e4107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 shutdown_connections 2024-09-21T06:17:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:10.735+0000 7f61ea681640 1 -- 172.21.15.31:0/786180620 wait complete. 2024-09-21T06:17:10.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:10 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:11.476 INFO:tasks.cephfs.filesystem:scrub status for tag:3fc27b03-07a0-480b-8d21-34b6a99891c5 - {'path': '/', 'tag': '3fc27b03-07a0-480b-8d21-34b6a99891c5', 'options': 'recursive,force'} 2024-09-21T06:17:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:11 smithi107 ceph-mon[34380]: pgmap v487: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-09-21T06:17:11.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:11 smithi031 ceph-mon[31237]: pgmap v487: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-09-21T06:17:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:11 smithi110 ceph-mon[34550]: pgmap v487: 97 pgs: 97 active+clean; 4.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 39 MiB/s wr, 1.47k op/s 2024-09-21T06:17:12.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:12 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:17:12.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:12 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:17:12.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:12 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:17:13.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:13 smithi031 ceph-mon[31237]: pgmap v488: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 40 MiB/s wr, 1.73k op/s 2024-09-21T06:17:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:13 smithi110 ceph-mon[34550]: pgmap v488: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 40 MiB/s wr, 1.73k op/s 2024-09-21T06:17:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:13 smithi107 ceph-mon[34380]: pgmap v488: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 40 MiB/s wr, 1.73k op/s 2024-09-21T06:17:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 335.0 s, 2422.2 tps, lat 41.405 ms stddev 47.969 2024-09-21T06:17:15.579 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:15 smithi110 ceph-mon[34550]: pgmap v489: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 36 MiB/s wr, 1.57k op/s 2024-09-21T06:17:15.590 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:15 smithi107 ceph-mon[34380]: pgmap v489: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 36 MiB/s wr, 1.57k op/s 2024-09-21T06:17:15.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:15 smithi031 ceph-mon[31237]: pgmap v489: 97 pgs: 97 active+clean; 4.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 36 MiB/s wr, 1.57k op/s 2024-09-21T06:17:17.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:17 smithi031 ceph-mon[31237]: pgmap v490: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2024-09-21T06:17:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:17 smithi110 ceph-mon[34550]: pgmap v490: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2024-09-21T06:17:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:17 smithi107 ceph-mon[34380]: pgmap v490: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 40 MiB/s wr, 1.83k op/s 2024-09-21T06:17:19.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:19 smithi031 ceph-mon[31237]: pgmap v491: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 27 MiB/s wr, 1.31k op/s 2024-09-21T06:17:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:19 smithi110 ceph-mon[34550]: pgmap v491: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 27 MiB/s wr, 1.31k op/s 2024-09-21T06:17:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:19 smithi107 ceph-mon[34380]: pgmap v491: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 27 MiB/s wr, 1.31k op/s 2024-09-21T06:17:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 340.0 s, 2488.6 tps, lat 40.098 ms stddev 46.313 2024-09-21T06:17:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.893+0000 7fa8b4ece640 1 -- 172.21.15.31:0/1331166494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 msgr2=0x7fa8b010f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.893+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/1331166494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b010f480 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009ca0 tx=0x7fa8a802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 -- 172.21.15.31:0/1331166494 shutdown_connections 2024-09-21T06:17:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/1331166494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b010f480 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/1331166494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b0108300 0x7fa8b0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/1331166494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b01029d0 0x7fa8b0102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 -- 172.21.15.31:0/1331166494 >> 172.21.15.31:0/1331166494 conn(0x7fa8b007a6f0 msgr2=0x7fa8b007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 -- 172.21.15.31:0/1331166494 shutdown_connections 2024-09-21T06:17:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.894+0000 7fa8b4ece640 1 -- 172.21.15.31:0/1331166494 wait complete. 2024-09-21T06:17:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.895+0000 7fa8b4ece640 1 Processor -- start 2024-09-21T06:17:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 -- start start 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b01029d0 0x7fa8b010fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b0108300 0x7fa8b0110320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b0110860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0113a70 con 0x7fa8b0108c50 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01138f0 con 0x7fa8b0108300 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8b4ece640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0113bf0 con 0x7fa8b01029d0 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8aed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b0110860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8aed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b0110860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35690/0 (socket says 172.21.15.31:35690) 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8aed9d640 1 -- 172.21.15.31:0/3622116015 learned_addr learned my addr 172.21.15.31:0/3622116015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8add9b640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b0108300 0x7fa8b0110320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.896+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b01029d0 0x7fa8b010fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8aed9d640 1 -- 172.21.15.31:0/3622116015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b01029d0 msgr2=0x7fa8b010fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8aed9d640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b01029d0 0x7fa8b010fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8aed9d640 1 -- 172.21.15.31:0/3622116015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b0108300 msgr2=0x7fa8b0110320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8aed9d640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b0108300 0x7fa8b0110320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8aed9d640 1 -- 172.21.15.31:0/3622116015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b01a8720 con 0x7fa8b0108c50 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8add9b640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b0108300 0x7fa8b0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.897+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b01029d0 0x7fa8b010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.898+0000 7fa8aed9d640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b0110860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009430 tx=0x7fa8a8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.898+0000 7fa89f7fe640 1 -- 172.21.15.31:0/3622116015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8003d90 con 0x7fa8b0108c50 2024-09-21T06:17:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.898+0000 7fa89f7fe640 1 -- 172.21.15.31:0/3622116015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a80044f0 con 0x7fa8b0108c50 2024-09-21T06:17:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.898+0000 7fa89f7fe640 1 -- 172.21.15.31:0/3622116015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8030db0 con 0x7fa8b0108c50 2024-09-21T06:17:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.898+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b01a89b0 con 0x7fa8b0108c50 2024-09-21T06:17:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.898+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01a8eb0 con 0x7fa8b0108c50 2024-09-21T06:17:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.902+0000 7fa8bbe36740 1 Processor -- start 2024-09-21T06:17:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.903+0000 7fa89f7fe640 1 -- 172.21.15.31:0/3622116015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa8a8004040 con 0x7fa8b0108c50 2024-09-21T06:17:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 -- start start 2024-09-21T06:17:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce5219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce521ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce522430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564bce4966f0 con 0x564bce1532f0 2024-09-21T06:17:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564bce485b20 con 0x564bce17c1b0 2024-09-21T06:17:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.904+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564bce489930 con 0x564bce15b4d0 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce5219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce5219b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47798/0 (socket says 172.21.15.31:47798) 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 -- 172.21.15.31:0/2688640192 learned_addr learned my addr 172.21.15.31:0/2688640192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce522430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce521ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 -- 172.21.15.31:0/2688640192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 msgr2=0x564bce5219b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce5219b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 -- 172.21.15.31:0/2688640192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 msgr2=0x564bce521ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce521ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa8add9b640 1 -- 172.21.15.31:0/2688640192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bce44b530 con 0x564bce15b4d0 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.905+0000 7fa89f7fe640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8980b6730 0x7fa8980b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.906+0000 7fa8add9b640 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce522430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4011b90 tx=0x7fa8a4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc261aad62ae536e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.906+0000 7fa89cff9640 1 -- 172.21.15.31:0/2688640192 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4011d40 con 0x564bce15b4d0 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.906+0000 7fa89f7fe640 1 -- 172.21.15.31:0/3622116015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa8a81087c0 con 0x7fa8b0108c50 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.906+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8980b6730 0x7fa8980b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.906+0000 7fa89cff9640 1 -- 172.21.15.31:0/2688640192 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a4011820 con 0x564bce15b4d0 2024-09-21T06:17:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.906+0000 7fa89cff9640 1 -- 172.21.15.31:0/2688640192 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4004a90 con 0x564bce15b4d0 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8980b6730 0x7fa8980b8c40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x564bce520630 tx=0x7fa8a000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 -- 172.21.15.31:0/2688640192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 msgr2=0x564bce522430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce522430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4011b90 tx=0x7fa8a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 -- 172.21.15.31:0/2688640192 shutdown_connections 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce522430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce521ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/2688640192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce5219b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.907+0000 7fa8bbe36740 1 -- 172.21.15.31:0/2688640192 >> 172.21.15.31:0/2688640192 conn(0x564bce170300 msgr2=0x564bce51f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.909+0000 7fa8bbe36740 1 -- 172.21.15.31:0/2688640192 shutdown_connections 2024-09-21T06:17:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.909+0000 7fa8bbe36740 1 -- 172.21.15.31:0/2688640192 wait complete. 2024-09-21T06:17:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.910+0000 7fa8bbe36740 1 -- start start 2024-09-21T06:17:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.910+0000 7fa8bbe36740 1 Processor -- start 2024-09-21T06:17:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce535870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce535db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce541c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce535870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce535db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce535db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38190/0 (socket says 172.21.15.31:38190) 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce535870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35708/0 (socket says 172.21.15.31:35708) 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 -- 172.21.15.31:0/1908817738 learned_addr learned my addr 172.21.15.31:0/1908817738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564bce3bdab0 con 0x564bce1532f0 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564bce42b730 con 0x564bce17c1b0 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564bce5388f0 con 0x564bce15b4d0 2024-09-21T06:17:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8aed9d640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce541c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8bbe36740 10 client.0 _ll_register_callbacks cb 0x564bce47f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 -- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 msgr2=0x564bce541c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce541c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 -- 172.21.15.31:0/1908817738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 msgr2=0x564bce535870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce535870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8add9b640 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bce1e7790 con 0x564bce15b4d0 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.911+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce535870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.912+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce535db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4011740 tx=0x7fa8a40054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40a9260f3f8b8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.912+0000 7fa895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.912+0000 7fa895ffb640 1 -- 172.21.15.31:0/1908817738 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400fc50 con 0x564bce15b4d0 2024-09-21T06:17:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.912+0000 7fa895ffb640 1 -- 172.21.15.31:0/1908817738 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa8a4004240 con 0x564bce15b4d0 2024-09-21T06:17:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.912+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a4011820 con 0x564bce15b4d0 2024-09-21T06:17:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa895ffb640 1 -- 172.21.15.31:0/1908817738 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa8a402d050 con 0x564bce15b4d0 2024-09-21T06:17:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa8bbe36740 10 client.26587 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564bce260620 con 0x564bce15b4d0 2024-09-21T06:17:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa895ffb640 1 -- 172.21.15.31:0/1908817738 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a4011820 con 0x564bce15b4d0 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa895ffb640 1 -- 172.21.15.31:0/1908817738 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa8a401d030 con 0x564bce15b4d0 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa8bbe36740 10 client.26587 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa8bbe36740 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa8bbe36740 20 client.26587 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.913+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce13ad80 0x564bce547820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.914+0000 7fa8bbe36740 4 client.26587 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.914+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bce498640 con 0x564bce13ad80 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.914+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce13ad80 0x564bce547820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.914+0000 7fa895ffb640 10 client.26587 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.914+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce13ad80 0x564bce547820 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1957a8752ef1c2 server_cookie=379df87269cd24f2 in_seq=0 out_seq=0 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa895ffb640 1 -- 172.21.15.31:0/1908817738 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564bce498640 con 0x564bce13ad80 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa895ffb640 10 client.26587 handle_command_reply: tid=0 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 client.26587 shutdown 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 msgr2=0x564bce535db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce535db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4011740 tx=0x7fa8a40054e0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 shutdown_connections 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce13ad80 0x564bce547820 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce17c1b0 0x564bce541c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce535db0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1908817738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce535870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.915+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 >> 172.21.15.31:0/1908817738 conn(0x564bce170300 msgr2=0x564bce524bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.916+0000 7fa895ffb640 0 client.26587 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.917+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 shutdown_connections 2024-09-21T06:17:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.917+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1908817738 wait complete. 2024-09-21T06:17:20.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.917+0000 7fa8bbe36740 20 client.26587 trim_cache size 0 max 16384 2024-09-21T06:17:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.942+0000 7fa8bbe36740 1 Processor -- start 2024-09-21T06:17:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.942+0000 7fa8bbe36740 1 -- start start 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce1532f0 0x564bce537750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce525190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce17c1b0 0x564bce541b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01ab450 con 0x564bce17c1b0 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01adcd0 con 0x564bce1532f0 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8ae59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce1532f0 0x564bce537750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8ae59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce1532f0 0x564bce537750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47814/0 (socket says 172.21.15.31:47814) 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8ae59c640 1 -- 172.21.15.31:0/1430800828 learned_addr learned my addr 172.21.15.31:0/1430800828 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1430800828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01abac0 con 0x564bce15b4d0 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8aed9d640 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce17c1b0 0x564bce541b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.943+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce525190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa8add9b640 1 -- 172.21.15.31:0/1430800828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce1532f0 msgr2=0x564bce537750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce1532f0 0x564bce537750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa8add9b640 1 -- 172.21.15.31:0/1430800828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce17c1b0 msgr2=0x564bce541b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce17c1b0 0x564bce541b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa8add9b640 1 -- 172.21.15.31:0/1430800828 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bce13afb0 con 0x564bce15b4d0 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa8add9b640 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce525190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009b50 tx=0x7fa8a400fe80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e351089e2cdc687 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.944+0000 7fa897fff640 1 -- 172.21.15.31:0/1430800828 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4009d80 con 0x564bce15b4d0 2024-09-21T06:17:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.945+0000 7fa897fff640 1 -- 172.21.15.31:0/1430800828 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a4005550 con 0x564bce15b4d0 2024-09-21T06:17:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.945+0000 7fa897fff640 1 -- 172.21.15.31:0/1430800828 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40114d0 con 0x564bce15b4d0 2024-09-21T06:17:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1430800828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 msgr2=0x564bce525190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce525190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009b50 tx=0x7fa8a400fe80 comp rx=0 tx=0).stop 2024-09-21T06:17:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1430800828 shutdown_connections 2024-09-21T06:17:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce17c1b0 0x564bce541b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce15b4d0 0x564bce525190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/1430800828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce1532f0 0x564bce537750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1430800828 >> 172.21.15.31:0/1430800828 conn(0x564bce170300 msgr2=0x564bce524a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1430800828 shutdown_connections 2024-09-21T06:17:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.946+0000 7fa8bbe36740 1 -- 172.21.15.31:0/1430800828 wait complete. 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.947+0000 7fa8bbe36740 1 -- start start 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.947+0000 7fa8bbe36740 1 Processor -- start 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.947+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce538fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce15b4d0 0x564bce539520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8bbe36740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce17c1b0 0x564bce5b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01ac190 con 0x564bce1532f0 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01ad600 con 0x564bce15b4d0 2024-09-21T06:17:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8bbe36740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01ac860 con 0x564bce17c1b0 2024-09-21T06:17:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce538fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce538fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35730/0 (socket says 172.21.15.31:35730) 2024-09-21T06:17:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 -- 172.21.15.31:0/223828768 learned_addr learned my addr 172.21.15.31:0/223828768 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8bbe36740 10 client.0 _ll_register_callbacks cb 0x7fa8a000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8aed9d640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce17c1b0 0x564bce5b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.951 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8add9b640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce15b4d0 0x564bce539520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 -- 172.21.15.31:0/223828768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce17c1b0 msgr2=0x564bce5b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce17c1b0 0x564bce5b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 -- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce15b4d0 msgr2=0x564bce539520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce15b4d0 0x564bce539520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.948+0000 7fa8ae59c640 1 -- 172.21.15.31:0/223828768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bce5b05b0 con 0x564bce1532f0 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.949+0000 7fa8ae59c640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce538fe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0010b50 tx=0x7fa8a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4514b7bb9282d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.949+0000 7fa895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.950+0000 7fa895ffb640 1 -- 172.21.15.31:0/223828768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a001e700 con 0x564bce1532f0 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.950+0000 7fa895ffb640 1 -- 172.21.15.31:0/223828768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fa8a001e8a0 con 0x564bce1532f0 2024-09-21T06:17:20.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.950+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a4005550 con 0x564bce1532f0 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.950+0000 7fa895ffb640 1 -- 172.21.15.31:0/223828768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa8a001eaa0 con 0x564bce1532f0 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 10 client.16809 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564bce5b1740 con 0x564bce1532f0 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa895ffb640 1 -- 172.21.15.31:0/223828768 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a4005550 con 0x564bce1532f0 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa895ffb640 1 -- 172.21.15.31:0/223828768 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa8a00303f0 con 0x564bce1532f0 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 10 client.16809 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 20 client.16809 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce1101c0 0x564bce5b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 4 client.16809 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bce4940f0 con 0x564bce1101c0 2024-09-21T06:17:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.951+0000 7fa8add9b640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce1101c0 0x564bce5b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.952+0000 7fa8add9b640 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce1101c0 0x564bce5b3f30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff45a2216c1bda2a server_cookie=5c6bfd2f79c7e3a1 in_seq=0 out_seq=0 2024-09-21T06:17:20.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.952+0000 7fa895ffb640 10 client.16809 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.997+0000 7fa895ffb640 1 -- 172.21.15.31:0/223828768 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bce4940f0 con 0x564bce1101c0 2024-09-21T06:17:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.997+0000 7fa895ffb640 10 client.16809 handle_command_reply: tid=0 2024-09-21T06:17:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.997+0000 7fa8bbe36740 1 client.16809 shutdown 2024-09-21T06:17:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.999+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 msgr2=0x564bce538fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:20.999+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce538fe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0010b50 tx=0x7fa8a00046c0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.000+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 shutdown_connections 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.000+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564bce1101c0 0x564bce5b3f30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.001+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bce17c1b0 0x564bce5b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.001+0000 7fa895ffb640 0 client.16809 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.001+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564bce15b4d0 0x564bce539520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.001+0000 7fa8bbe36740 1 --2- 172.21.15.31:0/223828768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564bce1532f0 0x564bce538fe0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.001+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 >> 172.21.15.31:0/223828768 conn(0x564bce170300 msgr2=0x564bce530c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.001+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 shutdown_connections 2024-09-21T06:17:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.002+0000 7fa8bbe36740 1 -- 172.21.15.31:0/223828768 wait complete. 2024-09-21T06:17:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.002+0000 7fa8bbe36740 20 client.16809 trim_cache size 0 max 16384 2024-09-21T06:17:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:17:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:17:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:17:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:17:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.003+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8980b6730 msgr2=0x7fa8980b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.003+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8980b6730 0x7fa8980b8c40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x564bce520630 tx=0x7fa8a000b410 comp rx=0 tx=0).stop 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.003+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 msgr2=0x7fa8b0110860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.003+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b0110860 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009430 tx=0x7fa8a8003970 comp rx=0 tx=0).stop 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 shutdown_connections 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa8980b6730 0x7fa8980b8c40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8b0108c50 0x7fa8b0110860 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b0108300 0x7fa8b0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 --2- 172.21.15.31:0/3622116015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8b01029d0 0x7fa8b010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 >> 172.21.15.31:0/3622116015 conn(0x7fa8b007a6f0 msgr2=0x7fa8b0106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 shutdown_connections 2024-09-21T06:17:21.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:21.004+0000 7fa8b4ece640 1 -- 172.21.15.31:0/3622116015 wait complete. 2024-09-21T06:17:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:21 smithi031 ceph-mon[31237]: pgmap v492: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-09-21T06:17:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:21 smithi110 ceph-mon[34550]: pgmap v492: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-09-21T06:17:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:21 smithi107 ceph-mon[34380]: pgmap v492: 97 pgs: 97 active+clean; 4.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 22 MiB/s wr, 1.14k op/s 2024-09-21T06:17:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:23 smithi031 ceph-mon[31237]: pgmap v493: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-09-21T06:17:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:23 smithi110 ceph-mon[34550]: pgmap v493: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-09-21T06:17:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:23 smithi107 ceph-mon[34380]: pgmap v493: 97 pgs: 97 active+clean; 4.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-09-21T06:17:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 345.0 s, 2541.0 tps, lat 39.365 ms stddev 69.909 2024-09-21T06:17:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:25 smithi031 ceph-mon[31237]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 898 op/s 2024-09-21T06:17:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:25 smithi110 ceph-mon[34550]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 898 op/s 2024-09-21T06:17:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:25 smithi107 ceph-mon[34380]: pgmap v494: 97 pgs: 97 active+clean; 4.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 17 MiB/s wr, 898 op/s 2024-09-21T06:17:27.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:27 smithi031 ceph-mon[31237]: pgmap v495: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-09-21T06:17:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:27 smithi110 ceph-mon[34550]: pgmap v495: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-09-21T06:17:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:27 smithi107 ceph-mon[34380]: pgmap v495: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-09-21T06:17:29.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:29 smithi031 ceph-mon[31237]: pgmap v496: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 28 MiB/s wr, 1.80k op/s 2024-09-21T06:17:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:29 smithi110 ceph-mon[34550]: pgmap v496: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 28 MiB/s wr, 1.80k op/s 2024-09-21T06:17:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:29 smithi107 ceph-mon[34380]: pgmap v496: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 28 MiB/s wr, 1.80k op/s 2024-09-21T06:17:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 350.0 s, 2879.6 tps, lat 34.693 ms stddev 39.310 2024-09-21T06:17:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:31 smithi031 ceph-mon[31237]: pgmap v497: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-09-21T06:17:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:31 smithi110 ceph-mon[34550]: pgmap v497: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-09-21T06:17:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:31 smithi107 ceph-mon[34380]: pgmap v497: 97 pgs: 97 active+clean; 4.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 27 MiB/s wr, 1.76k op/s 2024-09-21T06:17:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:33 smithi031 ceph-mon[31237]: pgmap v498: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 28 MiB/s wr, 1.89k op/s 2024-09-21T06:17:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:33 smithi110 ceph-mon[34550]: pgmap v498: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 28 MiB/s wr, 1.89k op/s 2024-09-21T06:17:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:33 smithi107 ceph-mon[34380]: pgmap v498: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 28 MiB/s wr, 1.89k op/s 2024-09-21T06:17:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 -- 172.21.15.31:0/3427046573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0103ae0 msgr2=0x7feef0103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 --2- 172.21.15.31:0/3427046573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0103ae0 0x7feef0103ef0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feee0009900 tx=0x7feee002a410 comp rx=0 tx=0).stop 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 -- 172.21.15.31:0/3427046573 shutdown_connections 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 --2- 172.21.15.31:0/3427046573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109d60 0x7feef0110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 --2- 172.21.15.31:0/3427046573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef0109410 0x7feef0109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 --2- 172.21.15.31:0/3427046573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0103ae0 0x7feef0103ef0 secure :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feee0009900 tx=0x7feee002a410 comp rx=0 tx=0).stop 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 -- 172.21.15.31:0/3427046573 >> 172.21.15.31:0/3427046573 conn(0x7feef00ff520 msgr2=0x7feef0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 -- 172.21.15.31:0/3427046573 shutdown_connections 2024-09-21T06:17:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.153+0000 7feef5263640 1 -- 172.21.15.31:0/3427046573 wait complete. 2024-09-21T06:17:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.154+0000 7feef5263640 1 Processor -- start 2024-09-21T06:17:34.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feef5263640 1 -- start start 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feef5263640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109410 0x7feef019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feef5263640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 0x7feef019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feef5263640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef019fd50 0x7feef01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feef5263640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feef0113a20 con 0x7feef0109d60 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 0x7feef019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.155+0000 7feef5263640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feef01138a0 con 0x7feef019fd50 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 0x7feef019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45970/0 (socket says 172.21.15.31:45970) 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeee7fc640 1 -- 172.21.15.31:0/3511070550 learned_addr learned my addr 172.21.15.31:0/3511070550 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:34.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feef0113ba0 con 0x7feef0109410 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef019fd50 0x7feef01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeeeffd640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109410 0x7feef019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 -- 172.21.15.31:0/3511070550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109410 msgr2=0x7feef019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109410 0x7feef019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 -- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 msgr2=0x7feef019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 0x7feef019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 -- 172.21.15.31:0/3511070550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef01a0930 con 0x7feef019fd50 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeeeffd640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109410 0x7feef019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeee7fc640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 0x7feef019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.156+0000 7feeef7fe640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef019fd50 0x7feef01a0160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feee8009fd0 tx=0x7feee800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.157+0000 7feed7fff640 1 -- 172.21.15.31:0/3511070550 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee800ce20 con 0x7feef019fd50 2024-09-21T06:17:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.157+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef01a6ed0 con 0x7feef019fd50 2024-09-21T06:17:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.157+0000 7feed7fff640 1 -- 172.21.15.31:0/3511070550 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee8005020 con 0x7feef019fd50 2024-09-21T06:17:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.157+0000 7feed7fff640 1 -- 172.21.15.31:0/3511070550 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8010690 con 0x7feef019fd50 2024-09-21T06:17:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.157+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a72d0 con 0x7feef019fd50 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.160+0000 7feefc1d3740 1 Processor -- start 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 -- start start 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd03410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f9372f0 0x56105fd03950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f93f4d0 0x56105fd15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56105fc7a6f0 con 0x56105f9601b0 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56105fc69b20 con 0x56105f9372f0 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feefc1d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56105fc6d930 con 0x56105f93f4d0 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd03410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd03410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45986/0 (socket says 172.21.15.31:45986) 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feeee7fc640 1 -- 172.21.15.31:0/4094979493 learned_addr learned my addr 172.21.15.31:0/4094979493 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.161+0000 7feeee7fc640 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f93f4d0 0x56105fd15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeeeffd640 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f9372f0 0x56105fd03950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeee7fc640 1 -- 172.21.15.31:0/4094979493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f93f4d0 msgr2=0x56105fd15dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeee7fc640 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f93f4d0 0x56105fd15dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeee7fc640 1 -- 172.21.15.31:0/4094979493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f9372f0 msgr2=0x56105fd03950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeee7fc640 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f9372f0 0x56105fd03950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeee7fc640 1 -- 172.21.15.31:0/4094979493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105fc2f530 con 0x56105f9601b0 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeeeffd640 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f9372f0 0x56105fd03950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feeee7fc640 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd03410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feee4012c40 tx=0x7feee400bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca1fbbdeaa6c702 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feed57fa640 1 -- 172.21.15.31:0/4094979493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4012df0 con 0x56105f9601b0 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feed57fa640 1 -- 172.21.15.31:0/4094979493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee400ebe0 con 0x56105f9601b0 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.162+0000 7feed57fa640 1 -- 172.21.15.31:0/4094979493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4003a80 con 0x56105f9601b0 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.163+0000 7feed7fff640 1 -- 172.21.15.31:0/3511070550 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7feee80054d0 con 0x7feef019fd50 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feed7fff640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed80b6680 0x7feed80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 -- 172.21.15.31:0/4094979493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 msgr2=0x56105fd03410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd03410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feee4012c40 tx=0x7feee400bbc0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feed7fff640 1 -- 172.21.15.31:0/3511070550 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feee80e7dc0 con 0x7feef019fd50 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feeeeffd640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed80b6680 0x7feed80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 -- 172.21.15.31:0/4094979493 shutdown_connections 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f93f4d0 0x56105fd15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f9372f0 0x56105fd03950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4094979493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd03410 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.164+0000 7feefc1d3740 1 -- 172.21.15.31:0/4094979493 >> 172.21.15.31:0/4094979493 conn(0x56105f954300 msgr2=0x56105fd05920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.165+0000 7feefc1d3740 1 -- 172.21.15.31:0/4094979493 shutdown_connections 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.165+0000 7feefc1d3740 1 -- 172.21.15.31:0/4094979493 wait complete. 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.165+0000 7feeeeffd640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed80b6680 0x7feed80b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feee0005940 tx=0x7feee00058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.165+0000 7feefc1d3740 1 -- start start 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.165+0000 7feefc1d3740 1 Processor -- start 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd1a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56105fba1ab0 con 0x56105f9372f0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56105fc0f730 con 0x56105f93f4d0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56105fd167e0 con 0x56105f9601b0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feefc1d3740 10 client.0 _ll_register_callbacks cb 0x56105fc636b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd1a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeef7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd1a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45998/0 (socket says 172.21.15.31:45998) 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeeeffd640 1 -- 172.21.15.31:0/4034392219 learned_addr learned my addr 172.21.15.31:0/4034392219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeee7fc640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeef7fe640 1 -- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 msgr2=0x56105fd1a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeef7fe640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeef7fe640 1 -- 172.21.15.31:0/4034392219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 msgr2=0x56105fd1a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeef7fe640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd1a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeef7fe640 1 -- 172.21.15.31:0/4034392219 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105f9cb790 con 0x56105f9601b0 2024-09-21T06:17:34.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.166+0000 7feeee7fc640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feeef7fe640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1f690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feee80e5550 tx=0x7feee80ad760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f65e6bb38880d27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feed2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feed2ff5640 1 -- 172.21.15.31:0/4034392219 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80d12d0 con 0x56105f9601b0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee400ebe0 con 0x56105f9601b0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feed2ff5640 1 -- 172.21.15.31:0/4034392219 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feee80aceb0 con 0x56105f9601b0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feed2ff5640 1 -- 172.21.15.31:0/4034392219 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7feee80b2730 con 0x56105f9601b0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feefc1d3740 10 client.26605 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56105fa44620 con 0x56105f9601b0 2024-09-21T06:17:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feed2ff5640 1 -- 172.21.15.31:0/4034392219 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee400ebe0 con 0x56105f9601b0 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.167+0000 7feed2ff5640 1 -- 172.21.15.31:0/4034392219 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7feee80b5ab0 con 0x56105f9601b0 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feefc1d3740 10 client.26605 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feefc1d3740 10 client.26605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feefc1d3740 20 client.26605 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f91ed80 0x56105fd2b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feefc1d3740 4 client.26605 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56105fc7c640 con 0x56105f91ed80 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.168+0000 7feeeeffd640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f91ed80 0x56105fd2b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.169+0000 7feeeeffd640 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f91ed80 0x56105fd2b8e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925d8920ac37a692 server_cookie=39defe0b5676107c in_seq=0 out_seq=0 2024-09-21T06:17:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.169+0000 7feed2ff5640 10 client.26605 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:34.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feed2ff5640 1 -- 172.21.15.31:0/4034392219 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56105fc7c640 con 0x56105f91ed80 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feed2ff5640 10 client.26605 handle_command_reply: tid=0 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 client.26605 shutdown 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 msgr2=0x56105fd1f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1f690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feee80e5550 tx=0x7feee80ad760 comp rx=0 tx=0).stop 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 shutdown_connections 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f91ed80 0x56105fd2b8e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1f690 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feed2ff5640 0 client.26605 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 --2- 172.21.15.31:0/4034392219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd1a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 >> 172.21.15.31:0/4034392219 conn(0x56105f954300 msgr2=0x56105fd029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 shutdown_connections 2024-09-21T06:17:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.171+0000 7feefc1d3740 1 -- 172.21.15.31:0/4034392219 wait complete. 2024-09-21T06:17:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.172+0000 7feefc1d3740 20 client.26605 trim_cache size 0 max 16384 2024-09-21T06:17:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 Processor -- start 2024-09-21T06:17:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 -- start start 2024-09-21T06:17:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feef01aba20 con 0x56105f9372f0 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ae280 con 0x56105f93f4d0 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feefc1d3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ac070 con 0x56105f9601b0 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.193+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd14e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46012/0 (socket says 172.21.15.31:46012) 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeeeffd640 1 -- 172.21.15.31:0/878202485 learned_addr learned my addr 172.21.15.31:0/878202485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeef7fe640 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeee7fc640 1 -- 172.21.15.31:0/878202485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 msgr2=0x56105fd1ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeee7fc640 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeee7fc640 1 -- 172.21.15.31:0/878202485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 msgr2=0x56105fd14e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeee7fc640 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd14e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeee7fc640 1 -- 172.21.15.31:0/878202485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56105fd041b0 con 0x56105f93f4d0 2024-09-21T06:17:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feeee7fc640 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1e4f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feee4005d00 tx=0x7feee4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5300598dabeea723 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feed4ff9640 1 -- 172.21.15.31:0/878202485 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee400edb0 con 0x56105f93f4d0 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feed4ff9640 1 -- 172.21.15.31:0/878202485 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee4023070 con 0x56105f93f4d0 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.194+0000 7feed4ff9640 1 -- 172.21.15.31:0/878202485 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4003cb0 con 0x56105f93f4d0 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 -- 172.21.15.31:0/878202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 msgr2=0x56105fd1e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1e4f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feee4005d00 tx=0x7feee4005850 comp rx=0 tx=0).stop 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 -- 172.21.15.31:0/878202485 shutdown_connections 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9601b0 0x56105fd1ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd1e4f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 --2- 172.21.15.31:0/878202485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9372f0 0x56105fd14e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 -- 172.21.15.31:0/878202485 >> 172.21.15.31:0/878202485 conn(0x56105f954300 msgr2=0x56105fd186f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 -- 172.21.15.31:0/878202485 shutdown_connections 2024-09-21T06:17:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.196+0000 7feefc1d3740 1 -- 172.21.15.31:0/878202485 wait complete. 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feefc1d3740 1 -- start start 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feefc1d3740 1 Processor -- start 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9372f0 0x56105fd31900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feefc1d3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9372f0 0x56105fd31900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feeeeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9372f0 0x56105fd31900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34278/0 (socket says 172.21.15.31:34278) 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.197+0000 7feeee7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35142/0 (socket says 172.21.15.31:35142) 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 -- 172.21.15.31:0/2356373476 learned_addr learned my addr 172.21.15.31:0/2356373476 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ac740 con 0x56105f9601b0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feef01adbb0 con 0x56105f93f4d0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feef01ace10 con 0x56105f9372f0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeef7fe640 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feefc1d3740 10 client.0 _ll_register_callbacks cb 0x7feee80aa2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 -- 172.21.15.31:0/2356373476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9372f0 msgr2=0x56105fd31900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9372f0 0x56105fd31900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 -- 172.21.15.31:0/2356373476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 msgr2=0x56105fd056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56105fd05e30 con 0x56105f93f4d0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feeee7fc640 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd050c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feee4012820 tx=0x7feee400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4213b093ad0dcf83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feed2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feed2ff5640 1 -- 172.21.15.31:0/2356373476 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4005460 con 0x56105f93f4d0 2024-09-21T06:17:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feed2ff5640 1 -- 172.21.15.31:0/2356373476 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7feee40038f0 con 0x56105f93f4d0 2024-09-21T06:17:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.198+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee4023070 con 0x56105f93f4d0 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.199+0000 7feed2ff5640 1 -- 172.21.15.31:0/2356373476 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7feee4003b30 con 0x56105f93f4d0 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.199+0000 7feefc1d3740 10 client.26558 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.199+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56105fd07550 con 0x56105f93f4d0 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.199+0000 7feed2ff5640 1 -- 172.21.15.31:0/2356373476 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee4023070 con 0x56105f93f4d0 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.199+0000 7feed2ff5640 1 -- 172.21.15.31:0/2356373476 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7feee402f070 con 0x56105f93f4d0 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feefc1d3740 10 client.26558 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feefc1d3740 10 client.26558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feefc1d3740 20 client.26558 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feefc1d3740 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f8f41c0 0x56105fd09a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feefc1d3740 4 client.26558 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56105fc780f0 con 0x56105f8f41c0 2024-09-21T06:17:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.200+0000 7feeeeffd640 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f8f41c0 0x56105fd09a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.202+0000 7feed2ff5640 10 client.26558 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.202+0000 7feeeeffd640 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f8f41c0 0x56105fd09a60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3416c272b44f384 server_cookie=d75f8810244133ae in_seq=0 out_seq=0 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.253+0000 7feed2ff5640 1 -- 172.21.15.31:0/2356373476 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56105fc780f0 con 0x56105f8f41c0 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.253+0000 7feed2ff5640 10 client.26558 handle_command_reply: tid=0 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.253+0000 7feefc1d3740 1 client.26558 shutdown 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 msgr2=0x56105fd050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd050c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feee4012820 tx=0x7feee400bcf0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 shutdown_connections 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56105f8f41c0 0x56105fd09a60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56105f9601b0 0x56105fd056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105f93f4d0 0x56105fd050c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feed2ff5640 0 client.26558 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 --2- 172.21.15.31:0/2356373476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56105f9372f0 0x56105fd31900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 >> 172.21.15.31:0/2356373476 conn(0x56105f954300 msgr2=0x56105fd1d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 shutdown_connections 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 1 -- 172.21.15.31:0/2356373476 wait complete. 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.254+0000 7feefc1d3740 20 client.26558 trim_cache size 0 max 16384 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:17:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.255+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed80b6680 msgr2=0x7feed80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed80b6680 0x7feed80b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feee0005940 tx=0x7feee00058d0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef019fd50 msgr2=0x7feef01a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef019fd50 0x7feef01a0160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feee8009fd0 tx=0x7feee800ef10 comp rx=0 tx=0).stop 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 shutdown_connections 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef019fd50 0x7feef01a0160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7feed80b6680 0x7feed80b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feef0109d60 0x7feef019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 --2- 172.21.15.31:0/3511070550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feef0109410 0x7feef019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 >> 172.21.15.31:0/3511070550 conn(0x7feef00ff520 msgr2=0x7feef0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 shutdown_connections 2024-09-21T06:17:34.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:34.256+0000 7feef5263640 1 -- 172.21.15.31:0/3511070550 wait complete. 2024-09-21T06:17:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 355.0 s, 2708.2 tps, lat 36.864 ms stddev 42.066 2024-09-21T06:17:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:35 smithi031 ceph-mon[31237]: pgmap v499: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-09-21T06:17:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:35 smithi110 ceph-mon[34550]: pgmap v499: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-09-21T06:17:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:35 smithi107 ceph-mon[34380]: pgmap v499: 97 pgs: 97 active+clean; 4.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 24 MiB/s wr, 1.70k op/s 2024-09-21T06:17:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:37 smithi031 ceph-mon[31237]: pgmap v500: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2024-09-21T06:17:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:37 smithi110 ceph-mon[34550]: pgmap v500: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2024-09-21T06:17:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:37 smithi107 ceph-mon[34380]: pgmap v500: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 25 MiB/s wr, 1.78k op/s 2024-09-21T06:17:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:39 smithi031 ceph-mon[31237]: pgmap v501: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 893 op/s 2024-09-21T06:17:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:39 smithi110 ceph-mon[34550]: pgmap v501: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 893 op/s 2024-09-21T06:17:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:39 smithi107 ceph-mon[34380]: pgmap v501: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 13 MiB/s wr, 893 op/s 2024-09-21T06:17:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 360.0 s, 2913.8 tps, lat 34.549 ms stddev 41.279 2024-09-21T06:17:41.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:41.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:41 smithi031 ceph-mon[31237]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 667 op/s 2024-09-21T06:17:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:41 smithi110 ceph-mon[34550]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 667 op/s 2024-09-21T06:17:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:41 smithi107 ceph-mon[34380]: pgmap v502: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 10 MiB/s wr, 667 op/s 2024-09-21T06:17:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.498+0000 7f1d32617640 1 -- 172.21.15.31:0/930840865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c103ba0 msgr2=0x7f1d2c103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.498+0000 7f1d32617640 1 --2- 172.21.15.31:0/930840865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c103fb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009900 tx=0x7f1d2402a410 comp rx=0 tx=0).stop 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 -- 172.21.15.31:0/930840865 shutdown_connections 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 --2- 172.21.15.31:0/930840865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d2c109e30 0x7f1d2c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 --2- 172.21.15.31:0/930840865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c1094e0 0x7f1d2c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 --2- 172.21.15.31:0/930840865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c103fb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 -- 172.21.15.31:0/930840865 >> 172.21.15.31:0/930840865 conn(0x7f1d2c0ff630 msgr2=0x7f1d2c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:42.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 -- 172.21.15.31:0/930840865 shutdown_connections 2024-09-21T06:17:42.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.499+0000 7f1d32617640 1 -- 172.21.15.31:0/930840865 wait complete. 2024-09-21T06:17:42.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.500+0000 7f1d32617640 1 Processor -- start 2024-09-21T06:17:42.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 -- start start 2024-09-21T06:17:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c1094e0 0x7f1d2c06e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d2c109e30 0x7f1d2c06e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c113af0 con 0x7f1d2c1094e0 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c113970 con 0x7f1d2c109e30 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d32617640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c113c70 con 0x7f1d2c103ba0 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d2bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d2bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47784/0 (socket says 172.21.15.31:47784) 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d2bfff640 1 -- 172.21.15.31:0/4126117056 learned_addr learned my addr 172.21.15.31:0/4126117056 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c1094e0 0x7f1d2c06e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.501+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d2c109e30 0x7f1d2c06e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d2bfff640 1 -- 172.21.15.31:0/4126117056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d2c109e30 msgr2=0x7f1d2c06e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d2bfff640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d2c109e30 0x7f1d2c06e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d2bfff640 1 -- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c1094e0 msgr2=0x7f1d2c06e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d2bfff640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c1094e0 0x7f1d2c06e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d2bfff640 1 -- 172.21.15.31:0/4126117056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c06ee90 con 0x7f1d2c103ba0 2024-09-21T06:17:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d2bfff640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c074e40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1d24006010 tx=0x7f1d240045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d297fa640 1 -- 172.21.15.31:0/4126117056 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24040070 con 0x7f1d2c103ba0 2024-09-21T06:17:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d297fa640 1 -- 172.21.15.31:0/4126117056 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d240316d0 con 0x7f1d2c103ba0 2024-09-21T06:17:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c073ab0 con 0x7f1d2c103ba0 2024-09-21T06:17:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d297fa640 1 -- 172.21.15.31:0/4126117056 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24035020 con 0x7f1d2c103ba0 2024-09-21T06:17:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.502+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c073f90 con 0x7f1d2c103ba0 2024-09-21T06:17:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.505+0000 7f1d38b8b740 1 Processor -- start 2024-09-21T06:17:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 -- start start 2024-09-21T06:17:42.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d297fa640 1 -- 172.21.15.31:0/4126117056 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1d24043070 con 0x7f1d2c103ba0 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da101650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da101b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d4da077e50 con 0x55d4d9d39390 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d4da067280 con 0x55d4d9d311b0 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d4da06b090 con 0x55d4d9d5a070 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d30bb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d30bb4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47794/0 (socket says 172.21.15.31:47794) 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d30bb4640 1 -- 172.21.15.31:0/3167938338 learned_addr learned my addr 172.21.15.31:0/3167938338 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.506+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d30bb4640 1 -- 172.21.15.31:0/3167938338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 msgr2=0x55d4da101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d30bb4640 1 -- 172.21.15.31:0/3167938338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 msgr2=0x55d4da1020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da1020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d30bb4640 1 -- 172.21.15.31:0/3167938338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d9dc5650 con 0x55d4d9d5a070 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da101650 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009fc0 tx=0x7f1d1c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afea621c49f7b360 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d297fa640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1d180b6680 0x7f1d180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.507+0000 7f1d227fc640 1 -- 172.21.15.31:0/3167938338 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c003b70 con 0x55d4d9d5a070 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.508+0000 7f1d227fc640 1 -- 172.21.15.31:0/3167938338 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c003d10 con 0x55d4d9d5a070 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.508+0000 7f1d2bfff640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1d180b6680 0x7f1d180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.508+0000 7f1d297fa640 1 -- 172.21.15.31:0/4126117056 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1d241081b0 con 0x7f1d2c103ba0 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.508+0000 7f1d227fc640 1 -- 172.21.15.31:0/3167938338 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c011a40 con 0x55d4d9d5a070 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.509+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3167938338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 msgr2=0x55d4da101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.509+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da101650 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009fc0 tx=0x7f1d1c010860 comp rx=0 tx=0).stop 2024-09-21T06:17:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3167938338 shutdown_connections 2024-09-21T06:17:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3167938338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da101650 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3167938338 >> 172.21.15.31:0/3167938338 conn(0x55d4d9d4e460 msgr2=0x55d4da0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d2bfff640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1d180b6680 0x7f1d180b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55d4da1002d0 tx=0x7f1d2410bc60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.510+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3167938338 shutdown_connections 2024-09-21T06:17:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.511+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3167938338 wait complete. 2024-09-21T06:17:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.511+0000 7f1d38b8b740 1 -- start start 2024-09-21T06:17:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.512+0000 7f1d38b8b740 1 Processor -- start 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da1154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d39390 0x55d4da115a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da115f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d30bb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da1154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d4d9ec8700 con 0x55d4d9d311b0 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d39390 0x55d4da115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d30bb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da1154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57630/0 (socket says 172.21.15.31:57630) 2024-09-21T06:17:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d30bb4640 1 -- 172.21.15.31:0/29132 learned_addr learned my addr 172.21.15.31:0/29132 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d4da002110 con 0x55d4d9d39390 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d4da118600 con 0x55d4d9d5a070 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d38b8b740 10 client.0 _ll_register_callbacks cb 0x55d4da01b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da115f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/29132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 msgr2=0x55d4da115f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da115f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/29132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 msgr2=0x55d4da1154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da1154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.513+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/29132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4da02fc40 con 0x55d4d9d39390 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d39390 0x55d4da115a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c06a790 tx=0x7f1d14012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fab948a149c4216 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/29132 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400f8f0 con 0x55d4d9d39390 2024-09-21T06:17:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/29132 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1d1400fa90 con 0x55d4d9d39390 2024-09-21T06:17:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1c003d10 con 0x55d4d9d39390 2024-09-21T06:17:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/29132 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1d1400fc90 con 0x55d4d9d39390 2024-09-21T06:17:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d38b8b740 10 client.26570 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.514+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d4da124330 con 0x55d4d9d39390 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.515+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/29132 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1c003d10 con 0x55d4d9d39390 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.515+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/29132 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1d1401c420 con 0x55d4d9d39390 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.515+0000 7f1d38b8b740 10 client.26570 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.515+0000 7f1d38b8b740 10 client.26570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d38b8b740 20 client.26570 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9d189f0 0x55d4da1279c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d38b8b740 4 client.26570 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4da079da0 con 0x55d4d9d189f0 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9d189f0 0x55d4da1279c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9d189f0 0x55d4da1279c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f4077fac63c6f2 server_cookie=b983ffaf2c0182b7 in_seq=0 out_seq=0 2024-09-21T06:17:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.516+0000 7f1d0b7fe640 10 client.26570 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.517+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/29132 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4da079da0 con 0x55d4d9d189f0 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.517+0000 7f1d0b7fe640 10 client.26570 handle_command_reply: tid=0 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.517+0000 7f1d38b8b740 1 client.26570 shutdown 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.517+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d39390 msgr2=0x55d4da115a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.517+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d39390 0x55d4da115a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c06a790 tx=0x7f1d14012b60 comp rx=0 tx=0).stop 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 shutdown_connections 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9d189f0 0x55d4da1279c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d39390 0x55d4da115a30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/29132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da1154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 >> 172.21.15.31:0/29132 conn(0x55d4d9d4e460 msgr2=0x55d4da1047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d0b7fe640 0 client.26570 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 shutdown_connections 2024-09-21T06:17:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 1 -- 172.21.15.31:0/29132 wait complete. 2024-09-21T06:17:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.518+0000 7f1d38b8b740 20 client.26570 trim_cache size 0 max 16384 2024-09-21T06:17:42.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.547+0000 7f1d38b8b740 1 Processor -- start 2024-09-21T06:17:42.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.547+0000 7f1d38b8b740 1 -- start start 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.547+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da110ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.547+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 0x55d4da117120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d30bb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da110ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d5a070 0x55d4da1126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1b3d10 con 0x55d4d9d311b0 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1b6590 con 0x55d4d9d5a070 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 0x55d4da117120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 0x55d4da117120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47800/0 (socket says 172.21.15.31:47800) 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3780890332 learned_addr learned my addr 172.21.15.31:0/3780890332 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1b4380 con 0x55d4d9d39390 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d5a070 0x55d4da1126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3780890332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d5a070 msgr2=0x55d4da1126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d5a070 0x55d4da1126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3780890332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 msgr2=0x55d4da110ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da110ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3780890332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4da116300 con 0x55d4d9d39390 2024-09-21T06:17:42.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.548+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:42.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.549+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 0x55d4da117120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1d14001eb0 tx=0x7f1d140096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=543c80994f4d15e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:42.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.549+0000 7f1d21ffb640 1 -- 172.21.15.31:0/3780890332 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14029030 con 0x55d4d9d39390 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.549+0000 7f1d21ffb640 1 -- 172.21.15.31:0/3780890332 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d1400f8f0 con 0x55d4d9d39390 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.549+0000 7f1d21ffb640 1 -- 172.21.15.31:0/3780890332 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14005270 con 0x55d4d9d39390 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.550+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3780890332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 msgr2=0x55d4da117120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 0x55d4da117120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1d14001eb0 tx=0x7f1d140096c0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3780890332 shutdown_connections 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d5a070 0x55d4da1126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d39390 0x55d4da117120 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3780890332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d311b0 0x55d4da110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3780890332 >> 172.21.15.31:0/3780890332 conn(0x55d4d9d4e460 msgr2=0x55d4da116a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:42.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3780890332 shutdown_connections 2024-09-21T06:17:42.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.551+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3780890332 wait complete. 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 -- start start 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 Processor -- start 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da128480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da128a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da109b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1b4a50 con 0x55d4d9d39390 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1b5ec0 con 0x55d4d9d311b0 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1b5120 con 0x55d4d9d5a070 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d38b8b740 10 client.0 _ll_register_callbacks cb 0x7f1d14009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:42.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da128a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d2b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da128a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57654/0 (socket says 172.21.15.31:57654) 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3017775003 learned_addr learned my addr 172.21.15.31:0/3017775003 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.552+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da128480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da109b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3017775003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 msgr2=0x55d4da109b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da109b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 msgr2=0x55d4da128480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da128480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d2b7fe640 1 -- 172.21.15.31:0/3017775003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4da10a150 con 0x55d4d9d39390 2024-09-21T06:17:42.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da128480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d2b7fe640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da128a20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005270 tx=0x7f1d140052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdfa3696946f783 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.553+0000 7f1d0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:17:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.554+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/3017775003 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14029040 con 0x55d4d9d39390 2024-09-21T06:17:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.554+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/3017775003 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1d140054c0 con 0x55d4d9d39390 2024-09-21T06:17:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.554+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d1400f8f0 con 0x55d4d9d39390 2024-09-21T06:17:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.555+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/3017775003 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1d1400faa0 con 0x55d4d9d39390 2024-09-21T06:17:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.555+0000 7f1d38b8b740 10 client.16845 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.555+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d4da10f580 con 0x55d4d9d39390 2024-09-21T06:17:42.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.556+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/3017775003 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d14003aa0 con 0x55d4d9d39390 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.556+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/3017775003 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1d140254a0 con 0x55d4d9d39390 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.556+0000 7f1d38b8b740 10 client.16845 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.556+0000 7f1d38b8b740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.556+0000 7f1d38b8b740 20 client.16845 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.556+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9cedda0 0x55d4da10fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.557+0000 7f1d38b8b740 4 client.16845 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.557+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1d1400f8f0 con 0x55d4d9cedda0 2024-09-21T06:17:42.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.557+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9cedda0 0x55d4da10fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.557+0000 7f1d0b7fe640 10 client.16845 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.557+0000 7f1d30bb4640 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9cedda0 0x55d4da10fd60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cacaa1784f37ff2 server_cookie=962e0e25fa8fee93 in_seq=0 out_seq=0 2024-09-21T06:17:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d0b7fe640 1 -- 172.21.15.31:0/3017775003 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x7f1d1400f8f0 con 0x55d4d9cedda0 2024-09-21T06:17:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d0b7fe640 10 client.16845 handle_command_reply: tid=0 2024-09-21T06:17:42.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 client.16845 shutdown 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 msgr2=0x55d4da128a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da128a20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005270 tx=0x7f1d140052a0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 shutdown_connections 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d4d9cedda0 0x55d4da10fd60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4d9d5a070 0x55d4da109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4d9d39390 0x55d4da128a20 secure :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1d14005270 tx=0x7f1d140052a0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 --2- 172.21.15.31:0/3017775003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d4d9d311b0 0x55d4da128480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 >> 172.21.15.31:0/3017775003 conn(0x55d4d9d4e460 msgr2=0x55d4da103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.558+0000 7f1d0b7fe640 0 client.16845 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:42.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.559+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 shutdown_connections 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.559+0000 7f1d38b8b740 1 -- 172.21.15.31:0/3017775003 wait complete. 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.559+0000 7f1d38b8b740 20 client.16845 trim_cache size 0 max 16384 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:17:42.562 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.563 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.564 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.565 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.566 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:17:42.568 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:17:42.571 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:42.572 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:42.573 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1d180b6680 msgr2=0x7f1d180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1d180b6680 0x7f1d180b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55d4da1002d0 tx=0x7f1d2410bc60 comp rx=0 tx=0).stop 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 msgr2=0x7f1d2c074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c074e40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1d24006010 tx=0x7f1d240045e0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 shutdown_connections 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d2c109e30 0x7f1d2c06e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1d180b6680 0x7f1d180b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d2c1094e0 0x7f1d2c06e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 --2- 172.21.15.31:0/4126117056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d2c103ba0 0x7f1d2c074e40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:42.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 >> 172.21.15.31:0/4126117056 conn(0x7f1d2c0ff630 msgr2=0x7f1d2c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:42.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 shutdown_connections 2024-09-21T06:17:42.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:42.560+0000 7f1d32617640 1 -- 172.21.15.31:0/4126117056 wait complete. 2024-09-21T06:17:43.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:17:43.488 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:17:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:43 smithi031 ceph-mon[31237]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-09-21T06:17:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:43 smithi110 ceph-mon[34550]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-09-21T06:17:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:43 smithi107 ceph-mon[34380]: pgmap v503: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-09-21T06:17:43.910 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.499+0000 7f51ff18a640 1 -- 172.21.15.31:0/1671010374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 msgr2=0x7f51f8110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.499+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1671010374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 0x7f51f8110340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009ca0 tx=0x7f51f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 -- 172.21.15.31:0/1671010374 shutdown_connections 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1671010374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 0x7f51f8110340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1671010374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f81052b0 0x7f51f810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1671010374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51f8104880 0x7f51f8104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 -- 172.21.15.31:0/1671010374 >> 172.21.15.31:0/1671010374 conn(0x7f51f80797e0 msgr2=0x7f51f8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 -- 172.21.15.31:0/1671010374 shutdown_connections 2024-09-21T06:17:44.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.500+0000 7f51ff18a640 1 -- 172.21.15.31:0/1671010374 wait complete. 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 Processor -- start 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 -- start start 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 0x7f51f810d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51f81052b0 0x7f51f8107990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 0x7f51f8107f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51f806d6a0 con 0x7f51f810de30 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.501+0000 7f51ff18a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51f806d520 con 0x7f51f8104880 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51ff18a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51f806d820 con 0x7f51f81052b0 2024-09-21T06:17:44.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 0x7f51f810d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fd727640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 0x7f51f8107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 0x7f51f810d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46314/0 (socket says 172.21.15.31:46314) 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51f7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51f81052b0 0x7f51f8107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 -- 172.21.15.31:0/1655059230 learned_addr learned my addr 172.21.15.31:0/1655059230 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 -- 172.21.15.31:0/1655059230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51f81052b0 msgr2=0x7f51f8107990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51f81052b0 0x7f51f8107990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 -- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 msgr2=0x7f51f8107f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 0x7f51f8107f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 -- 172.21.15.31:0/1655059230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f8108750 con 0x7f51f8104880 2024-09-21T06:17:44.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.502+0000 7f51fcf26640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 0x7f51f810d6d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00e9c0 tx=0x7f51ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.503+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00cd00 con 0x7f51f8104880 2024-09-21T06:17:44.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.503+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec0035f0 con 0x7f51f8104880 2024-09-21T06:17:44.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.503+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec010690 con 0x7f51f8104880 2024-09-21T06:17:44.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.503+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f8102730 con 0x7f51f8104880 2024-09-21T06:17:44.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.503+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f8102bc0 con 0x7f51f8104880 2024-09-21T06:17:44.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.504+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f8104c90 con 0x7f51f8104880 2024-09-21T06:17:44.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.506+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f51ec0037a0 con 0x7f51f8104880 2024-09-21T06:17:44.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.507+0000 7f51f5ffb640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51e80b66b0 0x7f51e80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:44.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.507+0000 7f51f7fff640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51e80b66b0 0x7f51e80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:44.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.507+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f51ec0e7a50 con 0x7f51f8104880 2024-09-21T06:17:44.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.508+0000 7f51f7fff640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51e80b66b0 0x7f51e80b8bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51e40095f0 tx=0x7f51e4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:44.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.512+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51ec0adfe0 con 0x7f51f8104880 2024-09-21T06:17:44.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.872+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51f8108c90 con 0x7f51f8104880 2024-09-21T06:17:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.873+0000 7f51f5ffb640 1 -- 172.21.15.31:0/1655059230 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f51ec0b23c0 con 0x7f51f8104880 2024-09-21T06:17:44.875 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:17:44.876 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:17:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51e80b66b0 msgr2=0x7f51e80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51e80b66b0 0x7f51e80b8bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51e40095f0 tx=0x7f51e4005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 msgr2=0x7f51f810d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 0x7f51f810d6d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00e9c0 tx=0x7f51ec00ee80 comp rx=0 tx=0).stop 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 shutdown_connections 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51e80b66b0 0x7f51e80b8bc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f810de30 0x7f51f8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51f81052b0 0x7f51f8107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 --2- 172.21.15.31:0/1655059230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51f8104880 0x7f51f810d6d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 >> 172.21.15.31:0/1655059230 conn(0x7f51f80797e0 msgr2=0x7f51f80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 shutdown_connections 2024-09-21T06:17:44.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:44.877+0000 7f51ff18a640 1 -- 172.21.15.31:0/1655059230 wait complete. 2024-09-21T06:17:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 365.0 s, 2280.2 tps, lat 43.787 ms stddev 59.055 2024-09-21T06:17:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:45 smithi031 ceph-mon[31237]: pgmap v504: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2024-09-21T06:17:45.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:45 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1655059230' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:45 smithi110 ceph-mon[34550]: pgmap v504: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2024-09-21T06:17:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:45 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1655059230' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:45 smithi107 ceph-mon[34380]: pgmap v504: 97 pgs: 97 active+clean; 4.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.5 MiB/s wr, 659 op/s 2024-09-21T06:17:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:45 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1655059230' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:17:45.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:17:46.212 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 -- 172.21.15.31:0/579015328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c103b80 msgr2=0x7f179c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 --2- 172.21.15.31:0/579015328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c103b80 0x7f179c103f90 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f178c009900 tx=0x7f178c02a410 comp rx=0 tx=0).stop 2024-09-21T06:17:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 -- 172.21.15.31:0/579015328 shutdown_connections 2024-09-21T06:17:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 --2- 172.21.15.31:0/579015328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f179c109e10 0x7f179c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 --2- 172.21.15.31:0/579015328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c1094c0 0x7f179c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 --2- 172.21.15.31:0/579015328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c103b80 0x7f179c103f90 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 -- 172.21.15.31:0/579015328 >> 172.21.15.31:0/579015328 conn(0x7f179c0ff610 msgr2=0x7f179c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 -- 172.21.15.31:0/579015328 shutdown_connections 2024-09-21T06:17:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.807+0000 7f17a191d640 1 -- 172.21.15.31:0/579015328 wait complete. 2024-09-21T06:17:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.808+0000 7f17a191d640 1 Processor -- start 2024-09-21T06:17:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.808+0000 7f17a191d640 1 -- start start 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f17a191d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 0x7f179c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f17a191d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c1094c0 0x7f179c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f17a191d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f179c109e10 0x7f179c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f17a191d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f179c113ad0 con 0x7f179c1094c0 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f17a191d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f179c113950 con 0x7f179c103b80 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f17a191d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f179c113c50 con 0x7f179c109e10 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 0x7f179c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 0x7f179c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46330/0 (socket says 172.21.15.31:46330) 2024-09-21T06:17:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 -- 172.21.15.31:0/397485904 learned_addr learned my addr 172.21.15.31:0/397485904 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179a7fc640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c1094c0 0x7f179c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179b7fe640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f179c109e10 0x7f179c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 -- 172.21.15.31:0/397485904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f179c109e10 msgr2=0x7f179c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f179c109e10 0x7f179c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 -- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c1094c0 msgr2=0x7f179c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c1094c0 0x7f179c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.809+0000 7f179affd640 1 -- 172.21.15.31:0/397485904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c1a0430 con 0x7f179c103b80 2024-09-21T06:17:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.810+0000 7f179affd640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 0x7f179c19a4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f178c005940 tx=0x7f178c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.811+0000 7f17a091b640 1 -- 172.21.15.31:0/397485904 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c03f030 con 0x7f179c103b80 2024-09-21T06:17:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.811+0000 7f17a091b640 1 -- 172.21.15.31:0/397485904 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c0044c0 con 0x7f179c103b80 2024-09-21T06:17:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.811+0000 7f17a091b640 1 -- 172.21.15.31:0/397485904 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c035070 con 0x7f179c103b80 2024-09-21T06:17:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.811+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c1a06c0 con 0x7f179c103b80 2024-09-21T06:17:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.811+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1a0a90 con 0x7f179c103b80 2024-09-21T06:17:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.814+0000 7f17a091b640 1 -- 172.21.15.31:0/397485904 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f178c042030 con 0x7f179c103b80 2024-09-21T06:17:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.815+0000 7f17a7e91740 1 Processor -- start 2024-09-21T06:17:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.815+0000 7f17a7e91740 1 -- start start 2024-09-21T06:17:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.815+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff6596070 0x55eff693d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.815+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff693d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.815+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 0x55eff693dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.815+0000 7f17a091b640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6850 0x7f17840b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f17a7e91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eff68b3ea0 con 0x55eff6575390 2024-09-21T06:17:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f17a7e91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eff68a32d0 con 0x55eff656d1b0 2024-09-21T06:17:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f179affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff693d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f179affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff693d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46334/0 (socket says 172.21.15.31:46334) 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f179affd640 1 -- 172.21.15.31:0/2436033618 learned_addr learned my addr 172.21.15.31:0/2436033618 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f17a091b640 1 -- 172.21.15.31:0/397485904 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f178c107820 con 0x7f179c103b80 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f179b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff6596070 0x55eff693d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f17a7e91740 1 -- 172.21.15.31:0/2436033618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eff68a70e0 con 0x55eff6596070 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.816+0000 7f179a7fc640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6850 0x7f17840b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179affd640 1 -- 172.21.15.31:0/2436033618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff6596070 msgr2=0x55eff693d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179affd640 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff6596070 0x55eff693d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179affd640 1 -- 172.21.15.31:0/2436033618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 msgr2=0x55eff693dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179affd640 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 0x55eff693dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179affd640 1 -- 172.21.15.31:0/2436033618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff6601650 con 0x55eff656d1b0 2024-09-21T06:17:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179a7fc640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6850 0x7f17840b8d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f179c108e50 tx=0x7f1790005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:46.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f179affd640 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff693d800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f178c0caa50 tx=0x7f178c0d2130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca2cd228e0831ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f1781ffb640 1 -- 172.21.15.31:0/2436033618 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c049040 con 0x55eff656d1b0 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f1781ffb640 1 -- 172.21.15.31:0/2436033618 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c0cd0b0 con 0x55eff656d1b0 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.817+0000 7f1781ffb640 1 -- 172.21.15.31:0/2436033618 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0d62a0 con 0x55eff656d1b0 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 -- 172.21.15.31:0/2436033618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 msgr2=0x55eff693d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff693d800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f178c0caa50 tx=0x7f178c0d2130 comp rx=0 tx=0).stop 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 -- 172.21.15.31:0/2436033618 shutdown_connections 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 0x55eff693dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff693d800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2436033618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff6596070 0x55eff693d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 -- 172.21.15.31:0/2436033618 >> 172.21.15.31:0/2436033618 conn(0x55eff658a460 msgr2=0x55eff693b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 -- 172.21.15.31:0/2436033618 shutdown_connections 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.818+0000 7f17a7e91740 1 -- 172.21.15.31:0/2436033618 wait complete. 2024-09-21T06:17:46.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.819+0000 7f17a7e91740 1 -- start start 2024-09-21T06:17:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.819+0000 7f17a7e91740 1 Processor -- start 2024-09-21T06:17:46.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.820+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 0x55eff6948a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 0x55eff6948f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff6596070 0x55eff6954b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f17a7e91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55eff6704700 con 0x55eff6575390 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f17a7e91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eff683e160 con 0x55eff6596070 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f17a7e91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eff694b860 con 0x55eff656d1b0 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f17a7e91740 10 client.0 _ll_register_callbacks cb 0x55eff68579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 0x55eff6948a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 0x55eff6948a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47860/0 (socket says 172.21.15.31:47860) 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 -- 172.21.15.31:0/2734155919 learned_addr learned my addr 172.21.15.31:0/2734155919 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 -- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff6596070 msgr2=0x55eff6954b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff6596070 0x55eff6954b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 -- 172.21.15.31:0/2734155919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 msgr2=0x55eff6948f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 0x55eff6948f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.821+0000 7f179affd640 1 -- 172.21.15.31:0/2734155919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff686bc90 con 0x55eff656d1b0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.822+0000 7f179affd640 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 0x55eff6948a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f178c009a20 tx=0x7f178c0d6df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f580190d83f71c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.822+0000 7f177affd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.823+0000 7f177affd640 1 -- 172.21.15.31:0/2734155919 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0f4020 con 0x55eff656d1b0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.823+0000 7f177affd640 1 -- 172.21.15.31:0/2734155919 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f178c0ccbc0 con 0x55eff656d1b0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.823+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c0cd0b0 con 0x55eff656d1b0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.823+0000 7f177affd640 1 -- 172.21.15.31:0/2734155919 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f178c049070 con 0x55eff656d1b0 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.823+0000 7f17a7e91740 10 client.26644 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:46.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.823+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55eff6957560 con 0x55eff656d1b0 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f177affd640 1 -- 172.21.15.31:0/2734155919 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c0cd0b0 con 0x55eff656d1b0 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f177affd640 1 -- 172.21.15.31:0/2734155919 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f178c0d65b0 con 0x55eff656d1b0 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f17a7e91740 10 client.26644 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f17a7e91740 10 client.26644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f17a7e91740 20 client.26644 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff65549f0 0x55eff695a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f17a7e91740 4 client.26644 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eff68b5df0 con 0x55eff65549f0 2024-09-21T06:17:46.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.824+0000 7f179a7fc640 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff65549f0 0x55eff695a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.826+0000 7f177affd640 10 client.26644 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.826+0000 7f179a7fc640 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff65549f0 0x55eff695a730 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd1ef95a24a58d server_cookie=5fb26e6f7818637d in_seq=0 out_seq=0 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.826+0000 7f177affd640 1 -- 172.21.15.31:0/2734155919 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eff68b5df0 con 0x55eff65549f0 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.826+0000 7f177affd640 10 client.26644 handle_command_reply: tid=0 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.826+0000 7f17a7e91740 1 client.26644 shutdown 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 msgr2=0x55eff6948a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 0x55eff6948a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f178c009a20 tx=0x7f178c0d6df0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 shutdown_connections 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff65549f0 0x55eff695a730 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff6596070 0x55eff6954b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff6575390 0x55eff6948f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f177affd640 0 client.26644 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 --2- 172.21.15.31:0/2734155919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff656d1b0 0x55eff6948a10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 >> 172.21.15.31:0/2734155919 conn(0x55eff658a460 msgr2=0x55eff6940450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 shutdown_connections 2024-09-21T06:17:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 1 -- 172.21.15.31:0/2734155919 wait complete. 2024-09-21T06:17:46.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.827+0000 7f17a7e91740 20 client.26644 trim_cache size 0 max 16384 2024-09-21T06:17:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 Processor -- start 2024-09-21T06:17:46.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 -- start start 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff693b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff6954a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff6940360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f178800c7c0 con 0x55eff65549f0 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f178800f040 con 0x55eff656d1b0 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.846+0000 7f17a7e91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f178800ce30 con 0x55eff658a460 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.847+0000 7f179a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff6954a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.847+0000 7f179a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff6954a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57700/0 (socket says 172.21.15.31:57700) 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.847+0000 7f179a7fc640 1 -- 172.21.15.31:0/1184120231 learned_addr learned my addr 172.21.15.31:0/1184120231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:46.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.848+0000 7f179a7fc640 1 -- 172.21.15.31:0/1184120231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 msgr2=0x55eff693b200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.848+0000 7f179a7fc640 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff693b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.848+0000 7f179a7fc640 1 -- 172.21.15.31:0/1184120231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 msgr2=0x55eff6940360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.848+0000 7f179a7fc640 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff6940360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.848+0000 7f179a7fc640 1 -- 172.21.15.31:0/1184120231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff69436b0 con 0x55eff65549f0 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.848+0000 7f179a7fc640 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff6954a50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f179000e460 tx=0x7f1790014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d6935beb70ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.849+0000 7f17817fa640 1 -- 172.21.15.31:0/1184120231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790004230 con 0x55eff65549f0 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.849+0000 7f17817fa640 1 -- 172.21.15.31:0/1184120231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179000ee10 con 0x55eff65549f0 2024-09-21T06:17:46.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.849+0000 7f17817fa640 1 -- 172.21.15.31:0/1184120231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790026690 con 0x55eff65549f0 2024-09-21T06:17:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 -- 172.21.15.31:0/1184120231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 msgr2=0x55eff6954a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff6954a50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f179000e460 tx=0x7f1790014000 comp rx=0 tx=0).stop 2024-09-21T06:17:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 -- 172.21.15.31:0/1184120231 shutdown_connections 2024-09-21T06:17:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff6940360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff6954a50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 --2- 172.21.15.31:0/1184120231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff693b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 -- 172.21.15.31:0/1184120231 >> 172.21.15.31:0/1184120231 conn(0x55eff695aa60 msgr2=0x55eff693aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 -- 172.21.15.31:0/1184120231 shutdown_connections 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 -- 172.21.15.31:0/1184120231 wait complete. 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.850+0000 7f17a7e91740 1 -- start start 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 Processor -- start 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff694cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff694d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff69cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f178800d500 con 0x55eff65549f0 2024-09-21T06:17:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f178800e970 con 0x55eff656d1b0 2024-09-21T06:17:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f17a7e91740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f178800dbd0 con 0x55eff658a460 2024-09-21T06:17:46.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f179affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff694cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f179b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff69cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f179affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff694cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57702/0 (socket says 172.21.15.31:57702) 2024-09-21T06:17:46.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.851+0000 7f179affd640 1 -- 172.21.15.31:0/276553466 learned_addr learned my addr 172.21.15.31:0/276553466 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:46.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f179a7fc640 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff694d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f17a7e91740 10 client.0 _ll_register_callbacks cb 0x7f1794009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f179a7fc640 1 -- 172.21.15.31:0/276553466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 msgr2=0x55eff69cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f179a7fc640 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff69cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f179a7fc640 1 -- 172.21.15.31:0/276553466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 msgr2=0x55eff694cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f179a7fc640 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff694cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f179a7fc640 1 -- 172.21.15.31:0/276553466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff69cce80 con 0x55eff656d1b0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.852+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179000ee10 con 0x55eff656d1b0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f179a7fc640 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff694d470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1790003f30 tx=0x7f1790004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4361c27d00c97ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f177affd640 10 client.26594 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f177affd640 1 -- 172.21.15.31:0/276553466 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900037f0 con 0x55eff656d1b0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f177affd640 1 -- 172.21.15.31:0/276553466 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f1790003990 con 0x55eff656d1b0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f177affd640 1 -- 172.21.15.31:0/276553466 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1790003bd0 con 0x55eff656d1b0 2024-09-21T06:17:46.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f17a7e91740 10 client.26594 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:46.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.853+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55eff69cdee0 con 0x55eff656d1b0 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f177affd640 1 -- 172.21.15.31:0/276553466 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f179002e480 con 0x55eff656d1b0 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f17a7e91740 10 client.26594 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f17a7e91740 10 client.26594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f17a7e91740 20 client.26594 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f17a7e91740 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff6575390 0x55eff69d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f17a7e91740 4 client.26594 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eff68b18a0 con 0x55eff6575390 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f179affd640 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff6575390 0x55eff69d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f179affd640 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff6575390 0x55eff69d0490 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756fe859b640f8b9 server_cookie=2244ae4f0d0d7649 in_seq=0 out_seq=0 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.854+0000 7f177affd640 10 client.26594 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f177affd640 1 -- 172.21.15.31:0/276553466 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eff68b18a0 con 0x55eff6575390 2024-09-21T06:17:46.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f177affd640 10 client.26594 handle_command_reply: tid=0 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 client.26594 shutdown 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 msgr2=0x55eff694d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff694d470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1790003f30 tx=0x7f1790004180 comp rx=0 tx=0).stop 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 shutdown_connections 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55eff6575390 0x55eff69d0490 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eff658a460 0x55eff69cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eff656d1b0 0x55eff694d470 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f177affd640 0 client.26594 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 --2- 172.21.15.31:0/276553466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55eff65549f0 0x55eff694cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 >> 172.21.15.31:0/276553466 conn(0x55eff695aa60 msgr2=0x55eff693e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 shutdown_connections 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 1 -- 172.21.15.31:0/276553466 wait complete. 2024-09-21T06:17:46.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.855+0000 7f17a7e91740 20 client.26594 trim_cache size 0 max 16384 2024-09-21T06:17:46.861 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6850 msgr2=0x7f17840b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6850 0x7f17840b8d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f179c108e50 tx=0x7f1790005f90 comp rx=0 tx=0).stop 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 msgr2=0x7f179c19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 0x7f179c19a4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f178c005940 tx=0x7f178c0040c0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 shutdown_connections 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f179c109e10 0x7f179c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6850 0x7f17840b8d60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f179c1094c0 0x7f179c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 --2- 172.21.15.31:0/397485904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f179c103b80 0x7f179c19a4c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 >> 172.21.15.31:0/397485904 conn(0x7f179c0ff610 msgr2=0x7f179c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 shutdown_connections 2024-09-21T06:17:46.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:46.856+0000 7f17a191d640 1 -- 172.21.15.31:0/397485904 wait complete. 2024-09-21T06:17:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 -- 172.21.15.31:0/4174114211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc10a660 msgr2=0x7efbfc10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 --2- 172.21.15.31:0/4174114211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc10a660 0x7efbfc10aa70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009900 tx=0x7efbf802a410 comp rx=0 tx=0).stop 2024-09-21T06:17:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 -- 172.21.15.31:0/4174114211 shutdown_connections 2024-09-21T06:17:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 --2- 172.21.15.31:0/4174114211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc110350 0x7efbfc110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 --2- 172.21.15.31:0/4174114211 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efbfc10fa00 0x7efbfc10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 --2- 172.21.15.31:0/4174114211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc10a660 0x7efbfc10aa70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.407+0000 7efc04739640 1 -- 172.21.15.31:0/4174114211 >> 172.21.15.31:0/4174114211 conn(0x7efbfc070dd0 msgr2=0x7efbfc0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.409+0000 7efc04739640 1 -- 172.21.15.31:0/4174114211 shutdown_connections 2024-09-21T06:17:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.409+0000 7efc04739640 1 -- 172.21.15.31:0/4174114211 wait complete. 2024-09-21T06:17:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 Processor -- start 2024-09-21T06:17:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 -- start start 2024-09-21T06:17:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 0x7efbfc1a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efbfc10fa00 0x7efbfc1a9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc110350 0x7efbfc1aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1227f0 con 0x7efbfc10a660 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc122670 con 0x7efbfc110350 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.410+0000 7efc04739640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc122970 con 0x7efbfc10fa00 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 0x7efbfc1a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 0x7efbfc1a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57716/0 (socket says 172.21.15.31:57716) 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 -- 172.21.15.31:0/4278398722 learned_addr learned my addr 172.21.15.31:0/4278398722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 -- 172.21.15.31:0/4278398722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efbfc10fa00 msgr2=0x7efbfc1a9810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efbfc10fa00 0x7efbfc1a9810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 -- 172.21.15.31:0/4278398722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc110350 msgr2=0x7efbfc1aeaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc110350 0x7efbfc1aeaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.411+0000 7efc024d5640 1 -- 172.21.15.31:0/4278398722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc1af1a0 con 0x7efbfc10a660 2024-09-21T06:17:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.413+0000 7efc024d5640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 0x7efbfc1a9270 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005940 tx=0x7efbf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.413+0000 7efbf37fe640 1 -- 172.21.15.31:0/4278398722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf803f030 con 0x7efbfc10a660 2024-09-21T06:17:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.413+0000 7efbf37fe640 1 -- 172.21.15.31:0/4278398722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf8003cb0 con 0x7efbfc10a660 2024-09-21T06:17:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.413+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc1af430 con 0x7efbfc10a660 2024-09-21T06:17:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.415+0000 7efbf37fe640 1 -- 172.21.15.31:0/4278398722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8035070 con 0x7efbfc10a660 2024-09-21T06:17:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.415+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1b5c20 con 0x7efbfc10a660 2024-09-21T06:17:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.420+0000 7efc0b6a7740 1 Processor -- start 2024-09-21T06:17:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 -- start start 2024-09-21T06:17:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 0x55e2744df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e2744dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e2744e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e2744546f0 con 0x55e2741194d0 2024-09-21T06:17:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e274443b20 con 0x55e2741112f0 2024-09-21T06:17:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.421+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e274447930 con 0x55e27413a1b0 2024-09-21T06:17:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 0x55e2744df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 0x55e2744df9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47894/0 (socket says 172.21.15.31:47894) 2024-09-21T06:17:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 -- 172.21.15.31:0/733408049 learned_addr learned my addr 172.21.15.31:0/733408049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc01cd4640 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e2744e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efbf37fe640 1 -- 172.21.15.31:0/4278398722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efbf8042030 con 0x7efbfc10a660 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 -- 172.21.15.31:0/733408049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 msgr2=0x55e2744dfef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e2744dfef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 -- 172.21.15.31:0/733408049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 msgr2=0x55e2744e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e2744e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.422+0000 7efc02cd6640 1 -- 172.21.15.31:0/733408049 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e274409530 con 0x55e27413a1b0 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.425+0000 7efc02cd6640 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 0x55e2744df9b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009cb0 tx=0x7efbf400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b021de0dbe6dda42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.425+0000 7efbf0ff9640 1 -- 172.21.15.31:0/733408049 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4016070 con 0x55e27413a1b0 2024-09-21T06:17:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.425+0000 7efbf0ff9640 1 -- 172.21.15.31:0/733408049 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf4005020 con 0x55e27413a1b0 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.425+0000 7efbf37fe640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efbe80b67d0 0x7efbe80b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.426+0000 7efc024d5640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efbe80b67d0 0x7efbe80b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.426+0000 7efbf0ff9640 1 -- 172.21.15.31:0/733408049 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40116d0 con 0x55e27413a1b0 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.426+0000 7efbf37fe640 1 -- 172.21.15.31:0/4278398722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efbf81077f0 con 0x7efbfc10a660 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 -- 172.21.15.31:0/733408049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 msgr2=0x55e2744df9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 0x55e2744df9b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009cb0 tx=0x7efbf400e6c0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc024d5640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efbe80b67d0 0x7efbe80b8ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55e2744de630 tx=0x7efbf8032f60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 -- 172.21.15.31:0/733408049 shutdown_connections 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e2744e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e2744dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/733408049 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e27413a1b0 0x55e2744df9b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 -- 172.21.15.31:0/733408049 >> 172.21.15.31:0/733408049 conn(0x55e27412e300 msgr2=0x55e2744dd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 -- 172.21.15.31:0/733408049 shutdown_connections 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 -- 172.21.15.31:0/733408049 wait complete. 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 -- start start 2024-09-21T06:17:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.427+0000 7efc0b6a7740 1 Processor -- start 2024-09-21T06:17:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741112f0 0x55e2744eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e2744eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e274508530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e27437bab0 con 0x55e2741112f0 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2743e9730 con 0x55e27413a1b0 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e2744ff1e0 con 0x55e2741194d0 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc0b6a7740 10 client.0 _ll_register_callbacks cb 0x55e27443d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.428+0000 7efc01cd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e2744eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e2744eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47906/0 (socket says 172.21.15.31:47906) 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 -- 172.21.15.31:0/4028332436 learned_addr learned my addr 172.21.15.31:0/4028332436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 -- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 msgr2=0x55e274508530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e274508530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 -- 172.21.15.31:0/4028332436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741112f0 msgr2=0x55e2744eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741112f0 0x55e2744eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 -- 172.21.15.31:0/4028332436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2741a5790 con 0x55e2741194d0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efc01cd4640 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e2744eb530 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b930 tx=0x7efbec00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff74196e40af5be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.429+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4028332436 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec005020 con 0x55e2741194d0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf4005020 con 0x55e2741194d0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4028332436 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efbec0051c0 con 0x55e2741194d0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4028332436 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efbec0053c0 con 0x55e2741194d0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 10 client.26656 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e27421e620 con 0x55e2741194d0 2024-09-21T06:17:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4028332436 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf4005020 con 0x55e2741194d0 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4028332436 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efbec011710 con 0x55e2741194d0 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 10 client.26656 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 10 client.26656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 20 client.26656 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740f8d80 0x55e27450e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 4 client.26656 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.430+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e274456640 con 0x55e2740f8d80 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.431+0000 7efc02cd6640 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740f8d80 0x55e27450e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.431+0000 7efbe5ffb640 10 client.26656 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.431+0000 7efc02cd6640 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740f8d80 0x55e27450e370 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6181a3a40637e5ab server_cookie=666a310071c71cf1 in_seq=0 out_seq=0 2024-09-21T06:17:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4028332436 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e274456640 con 0x55e2740f8d80 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efbe5ffb640 10 client.26656 handle_command_reply: tid=0 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 client.26656 shutdown 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 msgr2=0x55e2744eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e2744eb530 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efbec00b930 tx=0x7efbec00bdf0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 shutdown_connections 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740f8d80 0x55e27450e370 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e274508530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e2744eb530 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efbe5ffb640 0 client.26656 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4028332436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741112f0 0x55e2744eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 >> 172.21.15.31:0/4028332436 conn(0x55e27412e300 msgr2=0x55e2744e09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 shutdown_connections 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4028332436 wait complete. 2024-09-21T06:17:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.432+0000 7efc0b6a7740 20 client.26656 trim_cache size 0 max 16384 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.453+0000 7efc0b6a7740 1 Processor -- start 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.453+0000 7efc0b6a7740 1 -- start start 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.453+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 0x55e2744ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e274508430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e2744e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1aa390 con 0x55e2741194d0 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1b63f0 con 0x55e27413a1b0 2024-09-21T06:17:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc11a120 con 0x55e2741112f0 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 0x55e2744ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 0x55e2744ecc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47910/0 (socket says 172.21.15.31:47910) 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 -- 172.21.15.31:0/263706517 learned_addr learned my addr 172.21.15.31:0/263706517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc01cd4640 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e274508430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e2744e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 -- 172.21.15.31:0/263706517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 msgr2=0x55e2744e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e2744e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 -- 172.21.15.31:0/263706517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 msgr2=0x55e274508430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e274508430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 -- 172.21.15.31:0/263706517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2744e2210 con 0x55e2741112f0 2024-09-21T06:17:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.454+0000 7efc02cd6640 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 0x55e2744ecc20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efbf40059c0 tx=0x7efbf40059f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e43795c86d9fab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.455+0000 7efbe7fff640 1 -- 172.21.15.31:0/263706517 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4029040 con 0x55e2741112f0 2024-09-21T06:17:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.455+0000 7efbe7fff640 1 -- 172.21.15.31:0/263706517 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf4004c60 con 0x55e2741112f0 2024-09-21T06:17:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.455+0000 7efbe7fff640 1 -- 172.21.15.31:0/263706517 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4016040 con 0x55e2741112f0 2024-09-21T06:17:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 -- 172.21.15.31:0/263706517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 msgr2=0x55e2744ecc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 0x55e2744ecc20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efbf40059c0 tx=0x7efbf40059f0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 -- 172.21.15.31:0/263706517 shutdown_connections 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e27413a1b0 0x55e2744e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e2741194d0 0x55e274508430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/263706517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741112f0 0x55e2744ecc20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 -- 172.21.15.31:0/263706517 >> 172.21.15.31:0/263706517 conn(0x55e27412e300 msgr2=0x55e2744ec510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 -- 172.21.15.31:0/263706517 shutdown_connections 2024-09-21T06:17:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 -- 172.21.15.31:0/263706517 wait complete. 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 -- start start 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.457+0000 7efc0b6a7740 1 Processor -- start 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e274501d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e274502300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e27413a1b0 0x55e274570d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc11a7f0 con 0x55e27413a1b0 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1b5df0 con 0x55e2741112f0 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc11aec0 con 0x55e2741194d0 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e274501d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc0b6a7740 10 client.0 _ll_register_callbacks cb 0x7efbec0043b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e274501d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46392/0 (socket says 172.21.15.31:46392) 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 -- 172.21.15.31:0/4170986855 learned_addr learned my addr 172.21.15.31:0/4170986855 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc01cd4640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e274502300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e27413a1b0 0x55e274570d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 -- 172.21.15.31:0/4170986855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 msgr2=0x55e274502300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e274502300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 -- 172.21.15.31:0/4170986855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e27413a1b0 msgr2=0x55e274570d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e27413a1b0 0x55e274570d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 -- 172.21.15.31:0/4170986855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e274571400 con 0x55e2741112f0 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efc02cd6640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e274501d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005b40 tx=0x7efbf40058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6374cda58527cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.458+0000 7efbe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4170986855 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4029070 con 0x55e2741112f0 2024-09-21T06:17:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4170986855 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7efbf4004a60 con 0x55e2741112f0 2024-09-21T06:17:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf4004c60 con 0x55e2741112f0 2024-09-21T06:17:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4170986855 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efbf4043050 con 0x55e2741112f0 2024-09-21T06:17:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efc0b6a7740 10 client.26603 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e274578730 con 0x55e2741112f0 2024-09-21T06:17:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4170986855 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf4004c60 con 0x55e2741112f0 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.459+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4170986855 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efbf402c070 con 0x55e2741112f0 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc0b6a7740 10 client.26603 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc0b6a7740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc0b6a7740 20 client.26603 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740ce1c0 0x55e27457abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc0b6a7740 4 client.26603 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2744520f0 con 0x55e2740ce1c0 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc01cd4640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740ce1c0 0x55e27457abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.460+0000 7efc01cd4640 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740ce1c0 0x55e27457abf0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5484b27b55b2277e server_cookie=df792a6519bba78f in_seq=0 out_seq=0 2024-09-21T06:17:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.461+0000 7efbe5ffb640 10 client.26603 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efbe5ffb640 1 -- 172.21.15.31:0/4170986855 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2744520f0 con 0x55e2740ce1c0 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efbe5ffb640 10 client.26603 handle_command_reply: tid=0 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 client.26603 shutdown 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 msgr2=0x55e274501d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e274501d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005b40 tx=0x7efbf40058b0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 shutdown_connections 2024-09-21T06:17:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e2740ce1c0 0x55e27457abf0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e27413a1b0 0x55e274570d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e2741194d0 0x55e274502300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efbe5ffb640 0 client.26603 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 --2- 172.21.15.31:0/4170986855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2741112f0 0x55e274501d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 >> 172.21.15.31:0/4170986855 conn(0x55e27412e300 msgr2=0x55e2744e09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 shutdown_connections 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.493+0000 7efc0b6a7740 1 -- 172.21.15.31:0/4170986855 wait complete. 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc0b6a7740 20 client.26603 trim_cache size 0 max 16384 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:17:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efbe80b67d0 msgr2=0x7efbe80b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efbe80b67d0 0x7efbe80b8ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55e2744de630 tx=0x7efbf8032f60 comp rx=0 tx=0).stop 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 msgr2=0x7efbfc1a9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 0x7efbfc1a9270 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efbf8005940 tx=0x7efbf80042f0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 shutdown_connections 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efbfc110350 0x7efbfc1aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efbfc10fa00 0x7efbfc1a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efbe80b67d0 0x7efbe80b8ce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 --2- 172.21.15.31:0/4278398722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbfc10a660 0x7efbfc1a9270 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 >> 172.21.15.31:0/4278398722 conn(0x7efbfc070dd0 msgr2=0x7efbfc073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 shutdown_connections 2024-09-21T06:17:47.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:17:47.494+0000 7efc04739640 1 -- 172.21.15.31:0/4278398722 wait complete. 2024-09-21T06:17:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:47 smithi031 ceph-mon[31237]: pgmap v505: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-09-21T06:17:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:47 smithi110 ceph-mon[34550]: pgmap v505: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-09-21T06:17:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:47 smithi107 ceph-mon[34380]: pgmap v505: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-09-21T06:17:48.680 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:17:48.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:17:49.072 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:49.578 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:49 smithi031 ceph-mon[31237]: pgmap v506: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.5 MiB/s wr, 735 op/s 2024-09-21T06:17:49.579 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:49 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:17:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.723+0000 7fe3325c7640 1 -- 172.21.15.31:0/80650320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 msgr2=0x7fe32c11f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.723+0000 7fe3325c7640 1 --2- 172.21.15.31:0/80650320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c11f140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009860 tx=0x7fe31c02a360 comp rx=0 tx=0).stop 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.727+0000 7fe3325c7640 1 -- 172.21.15.31:0/80650320 shutdown_connections 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.727+0000 7fe3325c7640 1 --2- 172.21.15.31:0/80650320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c11f140 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.727+0000 7fe3325c7640 1 --2- 172.21.15.31:0/80650320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe32c10a9e0 0x7fe32c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.727+0000 7fe3325c7640 1 --2- 172.21.15.31:0/80650320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c111f40 0x7fe32c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.727+0000 7fe3325c7640 1 -- 172.21.15.31:0/80650320 >> 172.21.15.31:0/80650320 conn(0x7fe32c070ec0 msgr2=0x7fe32c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.728+0000 7fe3325c7640 1 -- 172.21.15.31:0/80650320 shutdown_connections 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.728+0000 7fe3325c7640 1 -- 172.21.15.31:0/80650320 wait complete. 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 Processor -- start 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 -- start start 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10a9e0 0x7fe32c11c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe32c111f40 0x7fe32c118600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c118b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c114c10 con 0x7fe32c111f40 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c114a90 con 0x7fe32c11cc30 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.729+0000 7fe3325c7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c114d90 con 0x7fe32c10a9e0 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.731+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10a9e0 0x7fe32c11c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.731+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10a9e0 0x7fe32c11c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34074/0 (socket says 172.21.15.31:34074) 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.731+0000 7fe32bfff640 1 -- 172.21.15.31:0/4265340615 learned_addr learned my addr 172.21.15.31:0/4265340615 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:49.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.731+0000 7fe330b64640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c118b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe32c111f40 0x7fe32c118600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe330b64640 1 -- 172.21.15.31:0/4265340615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10a9e0 msgr2=0x7fe32c11c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe330b64640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10a9e0 0x7fe32c11c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe330b64640 1 -- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe32c111f40 msgr2=0x7fe32c118600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe330b64640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe32c111f40 0x7fe32c118600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe330b64640 1 -- 172.21.15.31:0/4265340615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c119290 con 0x7fe32c11cc30 2024-09-21T06:17:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.732+0000 7fe330b64640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c118b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe31c02a950 tx=0x7fe31c032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.733+0000 7fe3297fa640 1 -- 172.21.15.31:0/4265340615 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c037030 con 0x7fe32c11cc30 2024-09-21T06:17:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.733+0000 7fe3297fa640 1 -- 172.21.15.31:0/4265340615 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c038040 con 0x7fe32c11cc30 2024-09-21T06:17:49.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.734+0000 7fe3297fa640 1 -- 172.21.15.31:0/4265340615 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c004920 con 0x7fe32c11cc30 2024-09-21T06:17:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.738+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c1acdd0 con 0x7fe32c11cc30 2024-09-21T06:17:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.739+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1ad170 con 0x7fe32c11cc30 2024-09-21T06:17:49.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.742+0000 7fe338b3b740 1 Processor -- start 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 -- start start 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56357c123e60 con 0x56357bde5670 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56357c113290 con 0x56357be06350 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.743+0000 7fe338b3b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56357c1170a0 con 0x56357bddd490 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34082/0 (socket says 172.21.15.31:34082) 2024-09-21T06:17:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 -- 172.21.15.31:0/83982064 learned_addr learned my addr 172.21.15.31:0/83982064 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 -- 172.21.15.31:0/83982064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 msgr2=0x56357c1ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 -- 172.21.15.31:0/83982064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 msgr2=0x56357c1adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 -- 172.21.15.31:0/83982064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357be71930 con 0x56357bddd490 2024-09-21T06:17:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.744+0000 7fe32bfff640 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1ad970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe324011540 tx=0x7fe324011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427e2e854a48cc3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.745+0000 7fe3127fc640 1 -- 172.21.15.31:0/83982064 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324004b20 con 0x56357bddd490 2024-09-21T06:17:49.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.745+0000 7fe3127fc640 1 -- 172.21.15.31:0/83982064 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324004cc0 con 0x56357bddd490 2024-09-21T06:17:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.747+0000 7fe3297fa640 1 -- 172.21.15.31:0/4265340615 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe31c043030 con 0x7fe32c11cc30 2024-09-21T06:17:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.747+0000 7fe338b3b740 1 -- 172.21.15.31:0/83982064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 msgr2=0x56357c1ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.747+0000 7fe338b3b740 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1ad970 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe324011540 tx=0x7fe324011a00 comp rx=0 tx=0).stop 2024-09-21T06:17:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.748+0000 7fe3297fa640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3140b6880 0x7fe3140b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.749+0000 7fe3297fa640 1 -- 172.21.15.31:0/4265340615 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe31c106810 con 0x7fe32c11cc30 2024-09-21T06:17:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.749+0000 7fe338b3b740 1 -- 172.21.15.31:0/83982064 shutdown_connections 2024-09-21T06:17:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.749+0000 7fe338b3b740 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.749+0000 7fe338b3b740 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1ad970 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.749+0000 7fe338b3b740 1 --2- 172.21.15.31:0/83982064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.749+0000 7fe338b3b740 1 -- 172.21.15.31:0/83982064 >> 172.21.15.31:0/83982064 conn(0x56357bdfa740 msgr2=0x56357c1ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.752+0000 7fe338b3b740 1 -- 172.21.15.31:0/83982064 shutdown_connections 2024-09-21T06:17:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.755+0000 7fe338b3b740 1 -- 172.21.15.31:0/83982064 wait complete. 2024-09-21T06:17:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.756+0000 7fe338b3b740 1 -- start start 2024-09-21T06:17:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.756+0000 7fe338b3b740 1 Processor -- start 2024-09-21T06:17:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.756+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3140b6880 0x7fe3140b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56357bf74890 con 0x56357bde5670 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56357c0add40 con 0x56357be06350 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56357c1bbc60 con 0x56357bddd490 2024-09-21T06:17:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe338b3b740 10 client.0 _ll_register_callbacks cb 0x56357c111c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:49.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34096/0 (socket says 172.21.15.31:34096) 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.757+0000 7fe32bfff640 1 -- 172.21.15.31:0/4233271562 learned_addr learned my addr 172.21.15.31:0/4233271562 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3140b6880 0x7fe3140b8d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0714c0 tx=0x7fe320004730 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe330b64640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 -- 172.21.15.31:0/4233271562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 msgr2=0x56357c1b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 -- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 msgr2=0x56357c1c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.758+0000 7fe32b7fe640 1 -- 172.21.15.31:0/4233271562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357c1c5510 con 0x56357bde5670 2024-09-21T06:17:49.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.759+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1b9440 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe32c06c6e0 tx=0x7fe3200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3d40a581ccd2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:49.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.759+0000 7fe30b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:17:49.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.759+0000 7fe30b7fe640 1 -- 172.21.15.31:0/4233271562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32003a060 con 0x56357bde5670 2024-09-21T06:17:49.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.759+0000 7fe30b7fe640 1 -- 172.21.15.31:0/4233271562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe320019040 con 0x56357bde5670 2024-09-21T06:17:49.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.760+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe324004cc0 con 0x56357bde5670 2024-09-21T06:17:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.760+0000 7fe30b7fe640 1 -- 172.21.15.31:0/4233271562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe320020790 con 0x56357bde5670 2024-09-21T06:17:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.760+0000 7fe338b3b740 10 client.16887 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:49.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.760+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56357c1c7e80 con 0x56357bde5670 2024-09-21T06:17:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.761+0000 7fe30b7fe640 1 -- 172.21.15.31:0/4233271562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324004cc0 con 0x56357bde5670 2024-09-21T06:17:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe30b7fe640 1 -- 172.21.15.31:0/4233271562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe320029040 con 0x56357bde5670 2024-09-21T06:17:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe338b3b740 10 client.16887 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe338b3b740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe338b3b740 20 client.16887 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:49.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe338b3b740 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bdc4cd0 0x56357c1cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe338b3b740 4 client.16887 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357c125db0 con 0x56357bdc4cd0 2024-09-21T06:17:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.762+0000 7fe32bfff640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bdc4cd0 0x56357c1cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.763+0000 7fe32bfff640 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bdc4cd0 0x56357c1cb2a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fec79321c61fda0 server_cookie=41318f15b48ca62f in_seq=0 out_seq=0 2024-09-21T06:17:49.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.763+0000 7fe30b7fe640 10 client.16887 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:49.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.764+0000 7fe30b7fe640 1 -- 172.21.15.31:0/4233271562 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56357c125db0 con 0x56357bdc4cd0 2024-09-21T06:17:49.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.764+0000 7fe30b7fe640 10 client.16887 handle_command_reply: tid=0 2024-09-21T06:17:49.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.764+0000 7fe338b3b740 1 client.16887 shutdown 2024-09-21T06:17:49.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.764+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 msgr2=0x56357c1b9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.764+0000 7fe338b3b740 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1b9440 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe32c06c6e0 tx=0x7fe3200046c0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 shutdown_connections 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bdc4cd0 0x56357c1cb2a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357be06350 0x56357c1c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357bde5670 0x56357c1b9440 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 --2- 172.21.15.31:0/4233271562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 >> 172.21.15.31:0/4233271562 conn(0x56357bdfa740 msgr2=0x56357c1b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:49.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe30b7fe640 0 client.16887 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:49.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 shutdown_connections 2024-09-21T06:17:49.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.766+0000 7fe338b3b740 1 -- 172.21.15.31:0/4233271562 wait complete. 2024-09-21T06:17:49.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.768+0000 7fe338b3b740 20 client.16887 trim_cache size 0 max 16384 2024-09-21T06:17:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:49 smithi110 ceph-mon[34550]: pgmap v506: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.5 MiB/s wr, 735 op/s 2024-09-21T06:17:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:49 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:17:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:49 smithi107 ceph-mon[34380]: pgmap v506: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.5 MiB/s wr, 735 op/s 2024-09-21T06:17:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:49 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.795+0000 7fe338b3b740 1 Processor -- start 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.795+0000 7fe338b3b740 1 -- start start 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 0x56357c1aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c1b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe338b3b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b3c10 con 0x56357be06350 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe338b3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b6490 con 0x56357bde5670 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe338b3b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b4280 con 0x56357bddd490 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe330b64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c1b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 0x56357c1aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 0x56357c1aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43628/0 (socket says 172.21.15.31:43628) 2024-09-21T06:17:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe330b64640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c1b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52818/0 (socket says 172.21.15.31:52818) 2024-09-21T06:17:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 -- 172.21.15.31:0/2598458968 learned_addr learned my addr 172.21.15.31:0/2598458968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:49.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32bfff640 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34104/0 (socket says 172.21.15.31:34104) 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 -- 172.21.15.31:0/2598458968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 msgr2=0x56357c1b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 -- 172.21.15.31:0/2598458968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 msgr2=0x56357c1b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c1b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 -- 172.21.15.31:0/2598458968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357c1af720 con 0x56357bde5670 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.796+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 0x56357c1aeb70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe320027040 tx=0x7fe32001a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1323e7a867ec857e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.797+0000 7fe311ffb640 1 -- 172.21.15.31:0/2598458968 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320043080 con 0x56357bde5670 2024-09-21T06:17:49.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.797+0000 7fe311ffb640 1 -- 172.21.15.31:0/2598458968 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320020de0 con 0x56357bde5670 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.797+0000 7fe311ffb640 1 -- 172.21.15.31:0/2598458968 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32002a880 con 0x56357bde5670 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 -- 172.21.15.31:0/2598458968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 msgr2=0x56357c1aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 0x56357c1aeb70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe320027040 tx=0x7fe32001a400 comp rx=0 tx=0).stop 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 -- 172.21.15.31:0/2598458968 shutdown_connections 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c1b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bde5670 0x56357c1aeb70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 --2- 172.21.15.31:0/2598458968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bddd490 0x56357c1b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 -- 172.21.15.31:0/2598458968 >> 172.21.15.31:0/2598458968 conn(0x56357bdfa740 msgr2=0x56357c1ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 -- 172.21.15.31:0/2598458968 shutdown_connections 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 -- 172.21.15.31:0/2598458968 wait complete. 2024-09-21T06:17:49.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.798+0000 7fe338b3b740 1 -- start start 2024-09-21T06:17:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 Processor -- start 2024-09-21T06:17:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bddd490 0x56357c1bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bde5670 0x56357c1bd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c234c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b4950 con 0x56357be06350 2024-09-21T06:17:49.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b5dc0 con 0x56357bddd490 2024-09-21T06:17:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b5020 con 0x56357bde5670 2024-09-21T06:17:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bddd490 0x56357c1bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe32bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bddd490 0x56357c1bd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43630/0 (socket says 172.21.15.31:43630) 2024-09-21T06:17:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe32bfff640 1 -- 172.21.15.31:0/1222642699 learned_addr learned my addr 172.21.15.31:0/1222642699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe338b3b740 10 client.0 _ll_register_callbacks cb 0x7fe320022e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:49.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.799+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bde5670 0x56357c1bd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe330b64640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c234c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe32b7fe640 1 -- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bddd490 msgr2=0x56357c1bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bddd490 0x56357c1bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe32b7fe640 1 -- 172.21.15.31:0/1222642699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 msgr2=0x56357c234c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c234c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe32b7fe640 1 -- 172.21.15.31:0/1222642699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357c1bdea0 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe330b64640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c234c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe32b7fe640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bde5670 0x56357c1bd6b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe320027040 tx=0x7fe32002bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f579c71346165412 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.801+0000 7fe30b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe30b7fe640 1 -- 172.21.15.31:0/1222642699 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320043040 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe320020de0 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe30b7fe640 1 -- 172.21.15.31:0/1222642699 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7fe320020b40 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe30b7fe640 1 -- 172.21.15.31:0/1222642699 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe320055050 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe338b3b740 10 client.26680 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56357c236150 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe30b7fe640 1 -- 172.21.15.31:0/1222642699 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320020de0 con 0x56357bde5670 2024-09-21T06:17:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe30b7fe640 1 -- 172.21.15.31:0/1222642699 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe320029070 con 0x56357bde5670 2024-09-21T06:17:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe338b3b740 10 client.26680 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe338b3b740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.802+0000 7fe338b3b740 20 client.26680 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.803+0000 7fe338b3b740 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bd9a080 0x56357c2388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.803+0000 7fe338b3b740 4 client.26680 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:17:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.803+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56357c121860 con 0x56357bd9a080 2024-09-21T06:17:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.803+0000 7fe32bfff640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bd9a080 0x56357c2388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.803+0000 7fe30b7fe640 10 client.26680 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.803+0000 7fe32bfff640 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bd9a080 0x56357c2388c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60bab061fe3acdcb server_cookie=8caf9cdf080bd150 in_seq=0 out_seq=0 2024-09-21T06:17:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe30b7fe640 1 -- 172.21.15.31:0/1222642699 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56357c121860 con 0x56357bd9a080 2024-09-21T06:17:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe30b7fe640 10 client.26680 handle_command_reply: tid=0 2024-09-21T06:17:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 client.26680 shutdown 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bde5670 msgr2=0x56357c1bd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bde5670 0x56357c1bd6b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe320027040 tx=0x7fe32002bfd0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 shutdown_connections 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56357bd9a080 0x56357c2388c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe30b7fe640 0 client.26680 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56357be06350 0x56357c234c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56357bde5670 0x56357c1bd6b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 --2- 172.21.15.31:0/1222642699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56357bddd490 0x56357c1bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.805+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 >> 172.21.15.31:0/1222642699 conn(0x56357bdfa740 msgr2=0x56357c1b7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.806+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 shutdown_connections 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.806+0000 7fe338b3b740 1 -- 172.21.15.31:0/1222642699 wait complete. 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.806+0000 7fe338b3b740 20 client.26680 trim_cache size 0 max 16384 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "c43e959f-3ba1-4285-ae7f-d6acc8bd2145", 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:17:49.809 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.808+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3140b6880 msgr2=0x7fe3140b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.808+0000 7fe3325c7640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3140b6880 0x7fe3140b8d90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0714c0 tx=0x7fe320004730 comp rx=0 tx=0).stop 2024-09-21T06:17:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.808+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 msgr2=0x7fe32c118b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.808+0000 7fe3325c7640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c118b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe31c02a950 tx=0x7fe31c032cd0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 shutdown_connections 2024-09-21T06:17:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe32c11cc30 0x7fe32c118b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3140b6880 0x7fe3140b8d90 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0714c0 tx=0x7fe320004730 comp rx=0 tx=0).stop 2024-09-21T06:17:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe32c111f40 0x7fe32c118600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 --2- 172.21.15.31:0/4265340615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe32c10a9e0 0x7fe32c11c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:49.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 >> 172.21.15.31:0/4265340615 conn(0x7fe32c070ec0 msgr2=0x7fe32c073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:49.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 shutdown_connections 2024-09-21T06:17:49.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:49.809+0000 7fe3325c7640 1 -- 172.21.15.31:0/4265340615 wait complete. 2024-09-21T06:17:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 370.0 s, 2057.2 tps, lat 48.423 ms stddev 51.392 2024-09-21T06:17:50.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:50 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:17:50.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:50 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:50.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:50 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:17:50.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:17:50.601 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:17:50.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:17:50.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:17:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:50 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:17:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:50 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:50 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:17:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:17:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:17:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:17:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:50 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:17:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:50 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:50 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:17:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:17:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:17:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:17:51.014 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:17:51.437 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:51 smithi031 ceph-mon[31237]: pgmap v507: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.0 MiB/s wr, 715 op/s 2024-09-21T06:17:51.437 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:51 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:51.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.573+0000 7f9688237640 1 -- 172.21.15.31:0/4147943480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 msgr2=0x7f9680102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.573+0000 7f9688237640 1 --2- 172.21.15.31:0/4147943480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f9680102a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f967c009900 tx=0x7f967c02a480 comp rx=0 tx=0).stop 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 -- 172.21.15.31:0/4147943480 shutdown_connections 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 --2- 172.21.15.31:0/4147943480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9680110200 0x7f9680112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 --2- 172.21.15.31:0/4147943480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 0x7f968010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 --2- 172.21.15.31:0/4147943480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f9680102a60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 -- 172.21.15.31:0/4147943480 >> 172.21.15.31:0/4147943480 conn(0x7f9680079790 msgr2=0x7f9680079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:51.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 -- 172.21.15.31:0/4147943480 shutdown_connections 2024-09-21T06:17:51.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.574+0000 7f9688237640 1 -- 172.21.15.31:0/4147943480 wait complete. 2024-09-21T06:17:51.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.575+0000 7f9688237640 1 Processor -- start 2024-09-21T06:17:51.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.576+0000 7f9688237640 1 -- start start 2024-09-21T06:17:51.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9688237640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f968010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9688237640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 0x7f968010d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9688237640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9680110200 0x7f968010db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9688237640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9680115cf0 con 0x7f9680102650 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9688237640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9680115b70 con 0x7f9680102fa0 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9688237640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9680115e70 con 0x7f9680110200 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f968010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 0x7f968010d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f968010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52850/0 (socket says 172.21.15.31:52850) 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9685fd3640 1 -- 172.21.15.31:0/779429423 learned_addr learned my addr 172.21.15.31:0/779429423 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96867d4640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9680110200 0x7f968010db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 -- 172.21.15.31:0/779429423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9680110200 msgr2=0x7f968010db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9680110200 0x7f968010db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 -- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 msgr2=0x7f968010d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f968010d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 -- 172.21.15.31:0/779429423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96801acae0 con 0x7f9680102fa0 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f9685fd3640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f968010d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.577+0000 7f96857d2640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 0x7f968010d600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f967000e9f0 tx=0x7f967000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.578+0000 7f9676ffd640 1 -- 172.21.15.31:0/779429423 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967000ccc0 con 0x7f9680102fa0 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.578+0000 7f9676ffd640 1 -- 172.21.15.31:0/779429423 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96700108e0 con 0x7f9680102fa0 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.578+0000 7f9676ffd640 1 -- 172.21.15.31:0/779429423 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670018880 con 0x7f9680102fa0 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.578+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96801acdd0 con 0x7f9680102fa0 2024-09-21T06:17:51.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.578+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96801ad230 con 0x7f9680102fa0 2024-09-21T06:17:51.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.582+0000 7f968e7ab740 1 Processor -- start 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.582+0000 7f968e7ab740 1 -- start start 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad93c5070 0x556ad976c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 0x556ad976c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad976ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f968e7ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556ad96e2e50 con 0x556ad93a4390 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f968e7ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556ad96d2280 con 0x556ad939c1b0 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f968e7ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556ad96d6090 con 0x556ad93c5070 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f9676ffd640 1 -- 172.21.15.31:0/779429423 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9670010430 con 0x7f9680102fa0 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 0x556ad976c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 0x556ad976c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43646/0 (socket says 172.21.15.31:43646) 2024-09-21T06:17:51.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f9685fd3640 1 -- 172.21.15.31:0/3884463712 learned_addr learned my addr 172.21.15.31:0/3884463712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:51.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f96867d4640 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad93c5070 0x556ad976c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.583+0000 7f96857d2640 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad976ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f9685fd3640 1 -- 172.21.15.31:0/3884463712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad93c5070 msgr2=0x556ad976c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f9685fd3640 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad93c5070 0x556ad976c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f9685fd3640 1 -- 172.21.15.31:0/3884463712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 msgr2=0x556ad976ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f9685fd3640 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad976ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f9685fd3640 1 -- 172.21.15.31:0/3884463712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad9430650 con 0x556ad939c1b0 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f96857d2640 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad976ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.584+0000 7f9685fd3640 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 0x556ad976c7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f967c02aa70 tx=0x7f967c005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2517d19fe95dfbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.585+0000 7f966bfff640 1 -- 172.21.15.31:0/3884463712 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c040070 con 0x556ad939c1b0 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.585+0000 7f966bfff640 1 -- 172.21.15.31:0/3884463712 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c0047a0 con 0x556ad939c1b0 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.585+0000 7f9676ffd640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f966c0b6680 0x7f966c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.585+0000 7f966bfff640 1 -- 172.21.15.31:0/3884463712 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c035040 con 0x556ad939c1b0 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.585+0000 7f96867d4640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f966c0b6680 0x7f966c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.585+0000 7f9676ffd640 1 -- 172.21.15.31:0/779429423 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f96700e7a10 con 0x7f9680102fa0 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 -- 172.21.15.31:0/3884463712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 msgr2=0x556ad976c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f96867d4640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f966c0b6680 0x7f966c0b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f9678009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 0x556ad976c7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f967c02aa70 tx=0x7f967c005f30 comp rx=0 tx=0).stop 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 -- 172.21.15.31:0/3884463712 shutdown_connections 2024-09-21T06:17:51.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad976ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad939c1b0 0x556ad976c7b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 --2- 172.21.15.31:0/3884463712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad93c5070 0x556ad976c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 -- 172.21.15.31:0/3884463712 >> 172.21.15.31:0/3884463712 conn(0x556ad93b9460 msgr2=0x556ad976a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 -- 172.21.15.31:0/3884463712 shutdown_connections 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.586+0000 7f968e7ab740 1 -- 172.21.15.31:0/3884463712 wait complete. 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.587+0000 7f968e7ab740 1 -- start start 2024-09-21T06:17:51.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.587+0000 7f968e7ab740 1 Processor -- start 2024-09-21T06:17:51.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9777840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9777d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad97782c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556ad9533700 con 0x556ad93a4390 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556ad966d110 con 0x556ad93c5070 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556ad978baa0 con 0x556ad939c1b0 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f968e7ab740 10 client.0 _ll_register_callbacks cb 0x556ad9686990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f96867d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad97782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9777840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f96857d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9777d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f96867d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad97782c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43656/0 (socket says 172.21.15.31:43656) 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.588+0000 7f96867d4640 1 -- 172.21.15.31:0/1322489542 learned_addr learned my addr 172.21.15.31:0/1322489542 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96867d4640 1 -- 172.21.15.31:0/1322489542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 msgr2=0x556ad9777840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96867d4640 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9777840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96867d4640 1 -- 172.21.15.31:0/1322489542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 msgr2=0x556ad9777d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96867d4640 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9777d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96867d4640 1 -- 172.21.15.31:0/1322489542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad969ac40 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96867d4640 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad97782c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9678003fd0 tx=0x7f9678004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fdbd1bb11e69cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96697fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96697fa640 1 -- 172.21.15.31:0/1322489542 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967800f8b0 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96697fa640 1 -- 172.21.15.31:0/1322489542 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f967800fa50 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c0047a0 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f96697fa640 1 -- 172.21.15.31:0/1322489542 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f967800fc50 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.589+0000 7f968e7ab740 10 client.26630 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556ad97976d0 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f96697fa640 1 -- 172.21.15.31:0/1322489542 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c0047a0 con 0x556ad93c5070 2024-09-21T06:17:51.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f96697fa640 1 -- 172.21.15.31:0/1322489542 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f9678026070 con 0x556ad93c5070 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 10 client.26630 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 20 client.26630 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad93839f0 0x556ad979acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 4 client.26630 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad96e4da0 con 0x556ad93839f0 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.590+0000 7f9685fd3640 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad93839f0 0x556ad979acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.591+0000 7f9685fd3640 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad93839f0 0x556ad979acd0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7737cd9f045b8f7e server_cookie=be7b40b021bdd519 in_seq=0 out_seq=0 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.592+0000 7f96697fa640 10 client.26630 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.592+0000 7f96697fa640 1 -- 172.21.15.31:0/1322489542 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ad96e4da0 con 0x556ad93839f0 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.592+0000 7f96697fa640 10 client.26630 handle_command_reply: tid=0 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.592+0000 7f968e7ab740 1 client.26630 shutdown 2024-09-21T06:17:51.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.592+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 msgr2=0x556ad97782c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.592+0000 7f968e7ab740 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad97782c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9678003fd0 tx=0x7f9678004ab0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 shutdown_connections 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad93839f0 0x556ad979acd0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad97782c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9777d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 --2- 172.21.15.31:0/1322489542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9777840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 >> 172.21.15.31:0/1322489542 conn(0x556ad93b9460 msgr2=0x556ad976dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f96697fa640 0 client.26630 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 shutdown_connections 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 1 -- 172.21.15.31:0/1322489542 wait complete. 2024-09-21T06:17:51.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.593+0000 7f968e7ab740 20 client.26630 trim_cache size 0 max 16384 2024-09-21T06:17:51.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.617+0000 7f968e7ab740 1 Processor -- start 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.617+0000 7f968e7ab740 1 -- start start 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9779720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad97714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad976d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f968e7ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f968010e400 con 0x556ad93a4390 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f968e7ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96801adcd0 con 0x556ad93c5070 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f968e7ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96801a1ff0 con 0x556ad939c1b0 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f96857d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad97714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f96857d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad97714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52870/0 (socket says 172.21.15.31:52870) 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f96857d2640 1 -- 172.21.15.31:0/4185102029 learned_addr learned my addr 172.21.15.31:0/4185102029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f9685fd3640 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9779720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f96867d4640 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad976d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f9685fd3640 1 -- 172.21.15.31:0/4185102029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 msgr2=0x556ad976d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f9685fd3640 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad976d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f9685fd3640 1 -- 172.21.15.31:0/4185102029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 msgr2=0x556ad97714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f9685fd3640 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad97714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f9685fd3640 1 -- 172.21.15.31:0/4185102029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad97643f0 con 0x556ad939c1b0 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.618+0000 7f96857d2640 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad97714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.619+0000 7f9685fd3640 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9779720 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f967c000c00 tx=0x7f967c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe97c482c171ae27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.619+0000 7f966b7fe640 1 -- 172.21.15.31:0/4185102029 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c040040 con 0x556ad939c1b0 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.619+0000 7f966b7fe640 1 -- 172.21.15.31:0/4185102029 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c0314a0 con 0x556ad939c1b0 2024-09-21T06:17:51.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.619+0000 7f966b7fe640 1 -- 172.21.15.31:0/4185102029 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c035040 con 0x556ad939c1b0 2024-09-21T06:17:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.620+0000 7f968e7ab740 1 -- 172.21.15.31:0/4185102029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 msgr2=0x556ad9779720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.620+0000 7f968e7ab740 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9779720 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f967c000c00 tx=0x7f967c005fd0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 -- 172.21.15.31:0/4185102029 shutdown_connections 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad976d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad97714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 --2- 172.21.15.31:0/4185102029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9779720 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 -- 172.21.15.31:0/4185102029 >> 172.21.15.31:0/4185102029 conn(0x556ad93b9460 msgr2=0x556ad976d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 -- 172.21.15.31:0/4185102029 shutdown_connections 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.621+0000 7f968e7ab740 1 -- 172.21.15.31:0/4185102029 wait complete. 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 -- start start 2024-09-21T06:17:51.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 Processor -- start 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9790d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9791270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad978bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96801a26a0 con 0x556ad93a4390 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96801ad600 con 0x556ad93c5070 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96801a2d70 con 0x556ad939c1b0 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f968e7ab740 10 client.0 _ll_register_callbacks cb 0x7f9678009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9790d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f9685fd3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9790d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34184/0 (socket says 172.21.15.31:34184) 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f9685fd3640 1 -- 172.21.15.31:0/897550465 learned_addr learned my addr 172.21.15.31:0/897550465 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:17:51.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.622+0000 7f96857d2640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9791270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.623+0000 7f96867d4640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad978bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.623+0000 7f9685fd3640 1 -- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 msgr2=0x556ad978bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.623+0000 7f9685fd3640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad978bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.623+0000 7f9685fd3640 1 -- 172.21.15.31:0/897550465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 msgr2=0x556ad9791270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.623+0000 7f9685fd3640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9791270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.623+0000 7f9685fd3640 1 -- 172.21.15.31:0/897550465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad978c220 con 0x556ad939c1b0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f9685fd3640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9790d30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f967c031880 tx=0x7f967c005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4df9ce095edf74c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f96697fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f96697fa640 1 -- 172.21.15.31:0/897550465 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c040050 con 0x556ad939c1b0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f96697fa640 1 -- 172.21.15.31:0/897550465 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f967c0037f0 con 0x556ad939c1b0 2024-09-21T06:17:51.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c0314a0 con 0x556ad939c1b0 2024-09-21T06:17:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f96697fa640 1 -- 172.21.15.31:0/897550465 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f967c035040 con 0x556ad939c1b0 2024-09-21T06:17:51.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f968e7ab740 10 client.26698 fetch_fsmap learned FSMap version 19 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.624+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556ad9805d40 con 0x556ad939c1b0 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f96697fa640 1 -- 172.21.15.31:0/897550465 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c0314a0 con 0x556ad939c1b0 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f96697fa640 1 -- 172.21.15.31:0/897550465 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f967c04f900 con 0x556ad939c1b0 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f968e7ab740 10 client.26698 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f968e7ab740 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f968e7ab740 20 client.26698 populate_metadata read hostname 'smithi031' 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f968e7ab740 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad9358da0 0x556ad9807f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f968e7ab740 4 client.26698 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ad96e0850 con 0x556ad9358da0 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.625+0000 7f96857d2640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad9358da0 0x556ad9807f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.626+0000 7f96697fa640 10 client.26698 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:51.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.626+0000 7f96857d2640 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad9358da0 0x556ad9807f70 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf442c042c3dd82f server_cookie=fdfd9cda3296fd6 in_seq=0 out_seq=0 2024-09-21T06:17:51.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.627+0000 7f96697fa640 1 -- 172.21.15.31:0/897550465 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x556ad96e0850 con 0x556ad9358da0 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.627+0000 7f96697fa640 10 client.26698 handle_command_reply: tid=0 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.627+0000 7f968e7ab740 1 client.26698 shutdown 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 msgr2=0x556ad9790d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9790d30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f967c031880 tx=0x7f967c005f30 comp rx=0 tx=0).stop 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 shutdown_connections 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ad9358da0 0x556ad9807f70 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ad93c5070 0x556ad978bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ad93a4390 0x556ad9791270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 --2- 172.21.15.31:0/897550465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ad939c1b0 0x556ad9790d30 secure :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f967c031880 tx=0x7f967c005f30 comp rx=0 tx=0).stop 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 >> 172.21.15.31:0/897550465 conn(0x556ad93b9460 msgr2=0x556ad976dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f96697fa640 0 client.26698 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 shutdown_connections 2024-09-21T06:17:51.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 1 -- 172.21.15.31:0/897550465 wait complete. 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.628+0000 7f968e7ab740 20 client.26698 trim_cache size 0 max 16384 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "c43e959f-3ba1-4285-ae7f-d6acc8bd2145", 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.632 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.633 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:17:51.634 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.636 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:17:51.637 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:17:51.640 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:17:51.641 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:17:51.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f966c0b6680 msgr2=0x7f966c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f966c0b6680 0x7f966c0b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f9678009500 comp rx=0 tx=0).stop 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 msgr2=0x7f968010d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:17:51.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 0x7f968010d600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f967000e9f0 tx=0x7f967000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 shutdown_connections 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9680110200 0x7f968010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9680102fa0 0x7f968010d600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f966c0b6680 0x7f966c0b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 --2- 172.21.15.31:0/779429423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9680102650 0x7f968010d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 >> 172.21.15.31:0/779429423 conn(0x7f9680079790 msgr2=0x7f9680109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 shutdown_connections 2024-09-21T06:17:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:17:51.629+0000 7f9688237640 1 -- 172.21.15.31:0/779429423 wait complete. 2024-09-21T06:17:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:51 smithi110 ceph-mon[34550]: pgmap v507: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.0 MiB/s wr, 715 op/s 2024-09-21T06:17:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:51 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:51 smithi107 ceph-mon[34380]: pgmap v507: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 9.0 MiB/s wr, 715 op/s 2024-09-21T06:17:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:51 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:17:52.457 INFO:tasks.cephfs.filesystem:scrub status for tag:c43e959f-3ba1-4285-ae7f-d6acc8bd2145 - {'path': '/', 'tag': 'c43e959f-3ba1-4285-ae7f-d6acc8bd2145', 'options': 'recursive,force'} 2024-09-21T06:17:52.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:52 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:17:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:52 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:17:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:52 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:17:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:53 smithi031 ceph-mon[31237]: pgmap v508: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-21T06:17:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:53 smithi110 ceph-mon[34550]: pgmap v508: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-21T06:17:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:53 smithi107 ceph-mon[34380]: pgmap v508: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-21T06:17:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 375.0 s, 2749.2 tps, lat 36.545 ms stddev 45.193 2024-09-21T06:17:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:55 smithi110 ceph-mon[34550]: pgmap v509: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.9 MiB/s wr, 725 op/s 2024-09-21T06:17:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:55 smithi107 ceph-mon[34380]: pgmap v509: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.9 MiB/s wr, 725 op/s 2024-09-21T06:17:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:55 smithi031 ceph-mon[31237]: pgmap v509: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.9 MiB/s wr, 725 op/s 2024-09-21T06:17:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:57 smithi110 ceph-mon[34550]: pgmap v510: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-21T06:17:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:57 smithi107 ceph-mon[34380]: pgmap v510: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-21T06:17:57.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:57 smithi031 ceph-mon[31237]: pgmap v510: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 11 MiB/s wr, 944 op/s 2024-09-21T06:17:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:17:59 smithi110 ceph-mon[34550]: pgmap v511: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 8.7 MiB/s wr, 732 op/s 2024-09-21T06:17:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:17:59 smithi107 ceph-mon[34380]: pgmap v511: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 8.7 MiB/s wr, 732 op/s 2024-09-21T06:17:59.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:17:59 smithi031 ceph-mon[31237]: pgmap v511: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 8.7 MiB/s wr, 732 op/s 2024-09-21T06:18:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 380.0 s, 2406.4 tps, lat 41.210 ms stddev 45.557 2024-09-21T06:18:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 -- 172.21.15.31:0/94686775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 msgr2=0x7f7ab4110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 --2- 172.21.15.31:0/94686775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab4110580 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009ca0 tx=0x7f7aac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 -- 172.21.15.31:0/94686775 shutdown_connections 2024-09-21T06:18:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 --2- 172.21.15.31:0/94686775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab4110580 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 --2- 172.21.15.31:0/94686775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 --2- 172.21.15.31:0/94686775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 -- 172.21.15.31:0/94686775 >> 172.21.15.31:0/94686775 conn(0x7f7ab40ff510 msgr2=0x7f7ab4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 -- 172.21.15.31:0/94686775 shutdown_connections 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.642+0000 7f7ababc2640 1 -- 172.21.15.31:0/94686775 wait complete. 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 Processor -- start 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 -- start start 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab406ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab406f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab406d6a0 con 0x7f7ab4109400 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab406d520 con 0x7f7ab4103ad0 2024-09-21T06:18:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.643+0000 7f7ababc2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab406d820 con 0x7f7ab4109d50 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab406ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab895e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab915f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab406f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab406ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34952/0 (socket says 172.21.15.31:34952) 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 -- 172.21.15.31:0/2906526803 learned_addr learned my addr 172.21.15.31:0/2906526803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab406f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47252/0 (socket says 172.21.15.31:47252) 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab895e640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4074cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:51468/0 (socket says 172.21.15.31:51468) 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 -- 172.21.15.31:0/2906526803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 msgr2=0x7f7ab406f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab406f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 -- 172.21.15.31:0/2906526803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 msgr2=0x7f7ab4074cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4074cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 -- 172.21.15.31:0/2906526803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab406fb50 con 0x7f7ab4109400 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab895e640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.644+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab406ef70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e970 tx=0x7f7aa000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.645+0000 7f7ab1ffb640 1 -- 172.21.15.31:0/2906526803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000cd80 con 0x7f7ab4109400 2024-09-21T06:18:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.645+0000 7f7ab1ffb640 1 -- 172.21.15.31:0/2906526803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0005020 con 0x7f7ab4109400 2024-09-21T06:18:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.645+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab40755f0 con 0x7f7ab4109400 2024-09-21T06:18:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.645+0000 7f7ab1ffb640 1 -- 172.21.15.31:0/2906526803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0004a00 con 0x7f7ab4109400 2024-09-21T06:18:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.645+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab4075b40 con 0x7f7ab4109400 2024-09-21T06:18:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.648+0000 7f7ac1c08740 1 Processor -- start 2024-09-21T06:18:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 -- start start 2024-09-21T06:18:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab1ffb640 1 -- 172.21.15.31:0/2906526803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7aa00051d0 con 0x7f7ab4109400 2024-09-21T06:18:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb3fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 0x5561bb3feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0384d0 0x5561bb3ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5561bb3736f0 con 0x5561bb0302f0 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5561bb362b20 con 0x5561bb0384d0 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5561bb366930 con 0x5561bb0591b0 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 0x5561bb3feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 0x5561bb3feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34964/0 (socket says 172.21.15.31:34964) 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 -- 172.21.15.31:0/781019365 learned_addr learned my addr 172.21.15.31:0/781019365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab915f640 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb3fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0384d0 0x5561bb3ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 -- 172.21.15.31:0/781019365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 msgr2=0x5561bb3fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb3fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 -- 172.21.15.31:0/781019365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0384d0 msgr2=0x5561bb3ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0384d0 0x5561bb3ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.649+0000 7f7ab895e640 1 -- 172.21.15.31:0/781019365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561bb328530 con 0x5561bb0302f0 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7ab895e640 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 0x5561bb3feef0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80114f0 tx=0x7f7aa80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e100d1d4759f175 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7a9effd640 1 -- 172.21.15.31:0/781019365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa801e070 con 0x5561bb0302f0 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7ab1ffb640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7aa40b6680 0x7f7aa40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7a9effd640 1 -- 172.21.15.31:0/781019365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa80035f0 con 0x5561bb0302f0 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7a9effd640 1 -- 172.21.15.31:0/781019365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800f820 con 0x5561bb0302f0 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7aa40b6680 0x7f7aa40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.650+0000 7f7ab1ffb640 1 -- 172.21.15.31:0/2906526803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7aa00e75b0 con 0x7f7ab4109400 2024-09-21T06:18:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 -- 172.21.15.31:0/781019365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 msgr2=0x5561bb3feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7aa40b6680 0x7f7aa40b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009620 tx=0x7f7aac03f040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 0x5561bb3feef0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80114f0 tx=0x7f7aa80119b0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 -- 172.21.15.31:0/781019365 shutdown_connections 2024-09-21T06:18:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0384d0 0x5561bb3ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0302f0 0x5561bb3feef0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/781019365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb3fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.651+0000 7f7ac1c08740 1 -- 172.21.15.31:0/781019365 >> 172.21.15.31:0/781019365 conn(0x5561bb04d300 msgr2=0x5561bb3fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.652+0000 7f7ac1c08740 1 -- 172.21.15.31:0/781019365 shutdown_connections 2024-09-21T06:18:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.653+0000 7f7ac1c08740 1 -- 172.21.15.31:0/781019365 wait complete. 2024-09-21T06:18:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.654+0000 7f7ac1c08740 1 -- start start 2024-09-21T06:18:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.654+0000 7f7ac1c08740 1 Processor -- start 2024-09-21T06:18:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0302f0 0x5561bb40a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb40a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb416450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5561bb29aab0 con 0x5561bb0384d0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5561bb308730 con 0x5561bb0302f0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5561bb40d130 con 0x5561bb0591b0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb416450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb40a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb416450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47258/0 (socket says 172.21.15.31:47258) 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 -- 172.21.15.31:0/2953503061 learned_addr learned my addr 172.21.15.31:0/2953503061 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab895e640 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0302f0 0x5561bb40a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ac1c08740 10 client.0 _ll_register_callbacks cb 0x5561bb35c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 -- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0302f0 msgr2=0x5561bb40a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0302f0 0x5561bb40a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 -- 172.21.15.31:0/2953503061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 msgr2=0x5561bb40a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb40a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.655+0000 7f7ab915f640 1 -- 172.21.15.31:0/2953503061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bb0c4790 con 0x5561bb0591b0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa80035f0 con 0x5561bb0591b0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb416450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0011d0 tx=0x7f7aac02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d314a5fa9859e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7a93fff640 10 client.26710 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7a93fff640 1 -- 172.21.15.31:0/2953503061 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac03cb30 con 0x5561bb0591b0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7a93fff640 1 -- 172.21.15.31:0/2953503061 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7aac03ccd0 con 0x5561bb0591b0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7a93fff640 1 -- 172.21.15.31:0/2953503061 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7aac003fc0 con 0x5561bb0591b0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7ac1c08740 10 client.26710 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.656+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5561bb13d620 con 0x5561bb0591b0 2024-09-21T06:18:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7a93fff640 1 -- 172.21.15.31:0/2953503061 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f7aac05d030 con 0x5561bb0591b0 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ac1c08740 10 client.26710 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ac1c08740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ac1c08740 20 client.26710 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb017d80 0x5561bb41c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ac1c08740 4 client.26710 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561bb375640 con 0x5561bb017d80 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.657+0000 7f7ab895e640 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb017d80 0x5561bb41c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.658+0000 7f7ab895e640 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb017d80 0x5561bb41c020 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25935919c33077b6 server_cookie=9956b26cb05fc357 in_seq=0 out_seq=0 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.658+0000 7f7a93fff640 10 client.26710 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7a93fff640 1 -- 172.21.15.31:0/2953503061 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561bb375640 con 0x5561bb017d80 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7a93fff640 10 client.26710 handle_command_reply: tid=0 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 client.26710 shutdown 2024-09-21T06:18:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 msgr2=0x5561bb416450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb416450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0011d0 tx=0x7f7aac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 shutdown_connections 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb017d80 0x5561bb41c020 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0591b0 0x5561bb416450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb40a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2953503061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb0302f0 0x5561bb40a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7a93fff640 0 client.26710 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.659+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 >> 172.21.15.31:0/2953503061 conn(0x5561bb04d300 msgr2=0x5561bb3ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.660+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 shutdown_connections 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.660+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2953503061 wait complete. 2024-09-21T06:18:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.660+0000 7f7ac1c08740 20 client.26710 trim_cache size 0 max 16384 2024-09-21T06:18:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.685+0000 7f7ac1c08740 1 Processor -- start 2024-09-21T06:18:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.685+0000 7f7ac1c08740 1 -- start start 2024-09-21T06:18:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb4000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb416350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb4071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b3b90 con 0x5561bb0384d0 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b6410 con 0x5561bb017d80 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b4200 con 0x5561bb0302f0 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb4071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb4071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34982/0 (socket says 172.21.15.31:34982) 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 -- 172.21.15.31:0/2793343842 learned_addr learned my addr 172.21.15.31:0/2793343842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb416350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab895e640 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb4000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 -- 172.21.15.31:0/2793343842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 msgr2=0x5561bb416350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb416350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 -- 172.21.15.31:0/2793343842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 msgr2=0x5561bb4000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb4000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.686+0000 7f7ab915f640 1 -- 172.21.15.31:0/2793343842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561bb4084e0 con 0x5561bb0384d0 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.687+0000 7f7ab915f640 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb4071d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7aac02a900 tx=0x7f7aac053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dff1b2f96a880d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.687+0000 7f7a9e7fc640 1 -- 172.21.15.31:0/2793343842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac068020 con 0x5561bb0384d0 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.687+0000 7f7a9e7fc640 1 -- 172.21.15.31:0/2793343842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa80035f0 con 0x5561bb0384d0 2024-09-21T06:18:00.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.687+0000 7f7a9e7fc640 1 -- 172.21.15.31:0/2793343842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac02fb10 con 0x5561bb0384d0 2024-09-21T06:18:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.688+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2793343842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 msgr2=0x5561bb4071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.688+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb4071d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7aac02a900 tx=0x7f7aac053000 comp rx=0 tx=0).stop 2024-09-21T06:18:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2793343842 shutdown_connections 2024-09-21T06:18:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb4071d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb416350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/2793343842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb4000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2793343842 >> 172.21.15.31:0/2793343842 conn(0x5561bb04d300 msgr2=0x5561bb3ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2793343842 shutdown_connections 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.689+0000 7f7ac1c08740 1 -- 172.21.15.31:0/2793343842 wait complete. 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.690+0000 7f7ac1c08740 1 -- start start 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.690+0000 7f7ac1c08740 1 Processor -- start 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb412c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb4100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb410690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b48d0 con 0x5561bb0384d0 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b5d40 con 0x5561bb017d80 2024-09-21T06:18:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41b4fa0 con 0x5561bb0302f0 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ac1c08740 10 client.0 _ll_register_callbacks cb 0x7f7aac009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb4100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab915f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb410690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb4100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47276/0 (socket says 172.21.15.31:47276) 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 -- 172.21.15.31:0/434414595 learned_addr learned my addr 172.21.15.31:0/434414595 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab895e640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb412c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 -- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 msgr2=0x5561bb412c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb412c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 -- 172.21.15.31:0/434414595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 msgr2=0x5561bb410690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb410690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 -- 172.21.15.31:0/434414595 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bb410e20 con 0x5561bb0302f0 2024-09-21T06:18:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab895e640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb412c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7ab3fff640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb4100d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0004770 tx=0x7f7aa00ace50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9618ee612532f08b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.691+0000 7f7a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7a93fff640 1 -- 172.21.15.31:0/434414595 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa001c040 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7a93fff640 1 -- 172.21.15.31:0/434414595 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8455+0+0 (secure 0 0 0) 0x7f7aa0004020 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa80035f0 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7a93fff640 1 -- 172.21.15.31:0/434414595 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7aa00b1f20 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 10 client.26716 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5561bb486a00 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7a93fff640 1 -- 172.21.15.31:0/434414595 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa00b6e60 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7a93fff640 1 -- 172.21.15.31:0/434414595 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f7aa00ac6a0 con 0x5561bb0302f0 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 10 client.26716 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 10 client.26716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 20 client.26716 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb0591b0 0x5561bb489180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 4 client.26716 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.692+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7aa80035f0 con 0x5561bb0591b0 2024-09-21T06:18:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.693+0000 7f7ab895e640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb0591b0 0x5561bb489180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.694+0000 7f7a93fff640 10 client.26716 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.695+0000 7f7ab895e640 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb0591b0 0x5561bb489180 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d81a43aae83b7e96 server_cookie=2ca966341844d824 in_seq=0 out_seq=0 2024-09-21T06:18:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7a93fff640 1 -- 172.21.15.31:0/434414595 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7aa80035f0 con 0x5561bb0591b0 2024-09-21T06:18:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7a93fff640 10 client.26716 handle_command_reply: tid=0 2024-09-21T06:18:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 client.26716 shutdown 2024-09-21T06:18:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 msgr2=0x5561bb4100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb4100d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0004770 tx=0x7f7aa00ace50 comp rx=0 tx=0).stop 2024-09-21T06:18:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 shutdown_connections 2024-09-21T06:18:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5561bb0591b0 0x5561bb489180 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5561bb0384d0 0x5561bb410690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7a93fff640 0 client.26716 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5561bb0302f0 0x5561bb4100d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 --2- 172.21.15.31:0/434414595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5561bb017d80 0x5561bb412c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.716+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 >> 172.21.15.31:0/434414595 conn(0x5561bb04d300 msgr2=0x5561bb3faea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.719+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 shutdown_connections 2024-09-21T06:18:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.719+0000 7f7ac1c08740 1 -- 172.21.15.31:0/434414595 wait complete. 2024-09-21T06:18:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:18:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:18:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:18:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.719+0000 7f7ac1c08740 20 client.26716 trim_cache size 0 max 16384 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7aa40b6680 msgr2=0x7f7aa40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7aa40b6680 0x7f7aa40b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009620 tx=0x7f7aac03f040 comp rx=0 tx=0).stop 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 msgr2=0x7f7ab406ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab406ef70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e970 tx=0x7f7aa000ee30 comp rx=0 tx=0).stop 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 shutdown_connections 2024-09-21T06:18:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ab4109d50 0x7f7ab406f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7aa40b6680 0x7f7aa40b8b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ab4109400 0x7f7ab406ef70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 --2- 172.21.15.31:0/2906526803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ab4103ad0 0x7f7ab4074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 >> 172.21.15.31:0/2906526803 conn(0x7f7ab40ff510 msgr2=0x7f7ab4107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 shutdown_connections 2024-09-21T06:18:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:00.720+0000 7f7ababc2640 1 -- 172.21.15.31:0/2906526803 wait complete. 2024-09-21T06:18:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:01 smithi110 ceph-mon[34550]: pgmap v512: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 8.5 MiB/s wr, 724 op/s 2024-09-21T06:18:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:01 smithi107 ceph-mon[34380]: pgmap v512: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 8.5 MiB/s wr, 724 op/s 2024-09-21T06:18:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:01 smithi031 ceph-mon[31237]: pgmap v512: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 8.5 MiB/s wr, 724 op/s 2024-09-21T06:18:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:03 smithi110 ceph-mon[34550]: pgmap v513: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 10 MiB/s wr, 885 op/s 2024-09-21T06:18:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:03 smithi107 ceph-mon[34380]: pgmap v513: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 10 MiB/s wr, 885 op/s 2024-09-21T06:18:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:03 smithi031 ceph-mon[31237]: pgmap v513: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 10 MiB/s wr, 885 op/s 2024-09-21T06:18:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 385.0 s, 459.0 tps, lat 206.087 ms stddev 218.788 2024-09-21T06:18:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:05 smithi110 ceph-mon[34550]: pgmap v514: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 621 op/s 2024-09-21T06:18:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:05 smithi107 ceph-mon[34380]: pgmap v514: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 621 op/s 2024-09-21T06:18:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:05 smithi031 ceph-mon[31237]: pgmap v514: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 6.4 MiB/s wr, 621 op/s 2024-09-21T06:18:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:07 smithi110 ceph-mon[34550]: pgmap v515: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.2 MiB/s wr, 747 op/s 2024-09-21T06:18:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:07 smithi107 ceph-mon[34380]: pgmap v515: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.2 MiB/s wr, 747 op/s 2024-09-21T06:18:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:07 smithi031 ceph-mon[31237]: pgmap v515: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.2 MiB/s wr, 747 op/s 2024-09-21T06:18:08.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:08 smithi110 ceph-mon[34550]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:18:08.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:08 smithi107 ceph-mon[34380]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:18:08.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:08 smithi031 ceph-mon[31237]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:18:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:09 smithi110 ceph-mon[34550]: pgmap v517: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-09-21T06:18:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:09 smithi107 ceph-mon[34380]: pgmap v517: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-09-21T06:18:09.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:09 smithi031 ceph-mon[31237]: pgmap v517: 97 pgs: 97 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.9 MiB/s wr, 560 op/s 2024-09-21T06:18:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 390.0 s, 199.0 tps, lat 521.854 ms stddev 822.145 2024-09-21T06:18:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:11 smithi110 ceph-mon[34550]: pgmap v518: 97 pgs: 10 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 81 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.8 MiB/s wr, 716 op/s 2024-09-21T06:18:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:11 smithi107 ceph-mon[34380]: pgmap v518: 97 pgs: 10 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 81 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.8 MiB/s wr, 716 op/s 2024-09-21T06:18:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:11 smithi031 ceph-mon[31237]: pgmap v518: 97 pgs: 10 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 81 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.8 MiB/s wr, 716 op/s 2024-09-21T06:18:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:13 smithi110 ceph-mon[34550]: pgmap v519: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.55k op/s 2024-09-21T06:18:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:13 smithi107 ceph-mon[34380]: pgmap v519: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.55k op/s 2024-09-21T06:18:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 -- 172.21.15.31:0/3547062502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 msgr2=0x7f5bb0110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 --2- 172.21.15.31:0/3547062502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb0110270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009ca0 tx=0x7f5ba802a950 comp rx=0 tx=0).stop 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 -- 172.21.15.31:0/3547062502 shutdown_connections 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 --2- 172.21.15.31:0/3547062502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb0110270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 --2- 172.21.15.31:0/3547062502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 --2- 172.21.15.31:0/3547062502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.879+0000 7f5bb7694640 1 -- 172.21.15.31:0/3547062502 >> 172.21.15.31:0/3547062502 conn(0x7f5bb007a6f0 msgr2=0x7f5bb007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.880+0000 7f5bb7694640 1 -- 172.21.15.31:0/3547062502 shutdown_connections 2024-09-21T06:18:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.880+0000 7f5bb7694640 1 -- 172.21.15.31:0/3547062502 wait complete. 2024-09-21T06:18:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.881+0000 7f5bb7694640 1 Processor -- start 2024-09-21T06:18:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 -- start start 2024-09-21T06:18:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb019f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01139f0 con 0x7f5bb0105130 2024-09-21T06:18:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0113870 con 0x7f5bb01047e0 2024-09-21T06:18:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb7694640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb0113b70 con 0x7f5bb010dd90 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.882+0000 7f5bb5430640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb5430640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb019a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:54030/0 (socket says 172.21.15.31:54030) 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb019f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb4c2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb4c2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35958/0 (socket says 172.21.15.31:35958) 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb4c2f640 1 -- 172.21.15.31:0/2487688109 learned_addr learned my addr 172.21.15.31:0/2487688109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb5430640 1 -- 172.21.15.31:0/2487688109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 msgr2=0x7f5bb019f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb5430640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb019f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.883+0000 7f5bb5430640 1 -- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 msgr2=0x7f5bb019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.884+0000 7f5bb5430640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.884+0000 7f5bb5430640 1 -- 172.21.15.31:0/2487688109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb01a00d0 con 0x7f5bb01047e0 2024-09-21T06:18:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.884+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.884+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb019f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.884+0000 7f5bb5430640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb019a1a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00e9f0 tx=0x7f5bac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:13.888 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:13 smithi031 ceph-mon[31237]: pgmap v519: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.55k op/s 2024-09-21T06:18:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.885+0000 7f5ba67fc640 1 -- 172.21.15.31:0/2487688109 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00ccc0 con 0x7f5bb01047e0 2024-09-21T06:18:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.885+0000 7f5ba67fc640 1 -- 172.21.15.31:0/2487688109 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bac0108e0 con 0x7f5bb01047e0 2024-09-21T06:18:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.885+0000 7f5ba67fc640 1 -- 172.21.15.31:0/2487688109 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac018880 con 0x7f5bb01047e0 2024-09-21T06:18:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.885+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb01a03c0 con 0x7f5bb01047e0 2024-09-21T06:18:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.885+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb01a6b50 con 0x7f5bb01047e0 2024-09-21T06:18:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.889+0000 7f5bbe604740 1 Processor -- start 2024-09-21T06:18:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 -- start start 2024-09-21T06:18:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa22889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2288ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2289430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55afa21fd6f0 con 0x55afa1ee31b0 2024-09-21T06:18:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55afa21ecb20 con 0x55afa1ec24d0 2024-09-21T06:18:13.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.890+0000 7f5bbe604740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55afa21f0930 con 0x55afa1eba2f0 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa22889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa22889b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35960/0 (socket says 172.21.15.31:35960) 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1193574024 learned_addr learned my addr 172.21.15.31:0/1193574024 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2289430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1193574024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 msgr2=0x55afa2288ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2288ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1193574024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 msgr2=0x55afa2289430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2289430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1193574024 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa21b2530 con 0x55afa1ee31b0 2024-09-21T06:18:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.891+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2289430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.892+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa22889b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009ca0 tx=0x7f5ba8009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd11e6095018f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.892+0000 7f5ba67fc640 1 -- 172.21.15.31:0/2487688109 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5bac010430 con 0x7f5bb01047e0 2024-09-21T06:18:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.893+0000 7f5b9b7fe640 1 -- 172.21.15.31:0/1193574024 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8048070 con 0x55afa1ee31b0 2024-09-21T06:18:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.893+0000 7f5b9b7fe640 1 -- 172.21.15.31:0/1193574024 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba802fd20 con 0x55afa1ee31b0 2024-09-21T06:18:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5ba67fc640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b9c0b67d0 0x7f5b9c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bb5430640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b9c0b67d0 0x7f5b9c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5ba67fc640 1 -- 172.21.15.31:0/2487688109 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8543+0+0 (secure 0 0 0) 0x7f5bac0e7950 con 0x7f5bb01047e0 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 -- 172.21.15.31:0/1193574024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 msgr2=0x55afa22889b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa22889b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009ca0 tx=0x7f5ba8009490 comp rx=0 tx=0).stop 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 -- 172.21.15.31:0/1193574024 shutdown_connections 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2289430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2288ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1193574024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa22889b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.894+0000 7f5bbe604740 1 -- 172.21.15.31:0/1193574024 >> 172.21.15.31:0/1193574024 conn(0x55afa1ed7300 msgr2=0x55afa22868a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.895+0000 7f5bbe604740 1 -- 172.21.15.31:0/1193574024 shutdown_connections 2024-09-21T06:18:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.895+0000 7f5bb5430640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b9c0b67d0 0x7f5b9c0b8ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55afa2287630 tx=0x7f5bac0a96e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.895+0000 7f5bbe604740 1 -- 172.21.15.31:0/1193574024 wait complete. 2024-09-21T06:18:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.895+0000 7f5bbe604740 1 -- start start 2024-09-21T06:18:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 Processor -- start 2024-09-21T06:18:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2290000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2290540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa2290a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55afa2124ab0 con 0x55afa1ee31b0 2024-09-21T06:18:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55afa2192730 con 0x55afa1ec24d0 2024-09-21T06:18:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55afa229f980 con 0x55afa1eba2f0 2024-09-21T06:18:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.896+0000 7f5bbe604740 10 client.0 _ll_register_callbacks cb 0x55afa21e66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.897+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2290000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.897+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2290000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49510/0 (socket says 172.21.15.31:49510) 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.897+0000 7f5bb5c31640 1 -- 172.21.15.31:0/981253316 learned_addr learned my addr 172.21.15.31:0/981253316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.897+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa2290a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.897+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2290540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5bb5c31640 1 -- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 msgr2=0x55afa2290540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2290540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5bb5c31640 1 -- 172.21.15.31:0/981253316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 msgr2=0x55afa2290a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa2290a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5bb5c31640 1 -- 172.21.15.31:0/981253316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa1f4e790 con 0x55afa1eba2f0 2024-09-21T06:18:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2290000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8004140 tx=0x7f5ba8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26d96f687f07a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5b98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.898+0000 7f5b98ff9640 1 -- 172.21.15.31:0/981253316 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8048040 con 0x55afa1eba2f0 2024-09-21T06:18:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.899+0000 7f5b98ff9640 1 -- 172.21.15.31:0/981253316 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8543+0+0 (secure 0 0 0) 0x7f5ba802fa80 con 0x55afa1eba2f0 2024-09-21T06:18:13.902 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.899+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba802fd20 con 0x55afa1eba2f0 2024-09-21T06:18:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.900+0000 7f5b98ff9640 1 -- 172.21.15.31:0/981253316 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5ba8056050 con 0x55afa1eba2f0 2024-09-21T06:18:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.900+0000 7f5bbe604740 10 client.26725 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.900+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55afa1fc7620 con 0x55afa1eba2f0 2024-09-21T06:18:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5b98ff9640 1 -- 172.21.15.31:0/981253316 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba802fd20 con 0x55afa1eba2f0 2024-09-21T06:18:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5b98ff9640 1 -- 172.21.15.31:0/981253316 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f5ba8043410 con 0x55afa1eba2f0 2024-09-21T06:18:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5bbe604740 10 client.26725 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5bbe604740 10 client.26725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5bbe604740 20 client.26725 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5bbe604740 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1ea1d80 0x55afa2298d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5bbe604740 4 client.26725 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:13.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.901+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afa21ff640 con 0x55afa1ea1d80 2024-09-21T06:18:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.902+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1ea1d80 0x55afa2298d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.903+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1ea1d80 0x55afa2298d90 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec62395397be113e server_cookie=fde5620cc57ed554 in_seq=0 out_seq=0 2024-09-21T06:18:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.903+0000 7f5b98ff9640 10 client.26725 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.904+0000 7f5b98ff9640 1 -- 172.21.15.31:0/981253316 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55afa21ff640 con 0x55afa1ea1d80 2024-09-21T06:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.904+0000 7f5b98ff9640 10 client.26725 handle_command_reply: tid=0 2024-09-21T06:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.904+0000 7f5bbe604740 1 client.26725 shutdown 2024-09-21T06:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.905+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 msgr2=0x55afa2290000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.905+0000 7f5bbe604740 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2290000 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8004140 tx=0x7f5ba8005fd0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.905+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 shutdown_connections 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.905+0000 7f5bbe604740 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1ea1d80 0x55afa2298d90 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.905+0000 7f5bbe604740 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ee31b0 0x55afa2290a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5bbe604740 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ec24d0 0x55afa2290540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5bbe604740 1 --2- 172.21.15.31:0/981253316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa2290000 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 >> 172.21.15.31:0/981253316 conn(0x55afa1ed7300 msgr2=0x55afa2284ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5b98ff9640 0 client.26725 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 shutdown_connections 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5bbe604740 1 -- 172.21.15.31:0/981253316 wait complete. 2024-09-21T06:18:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.906+0000 7f5bbe604740 20 client.26725 trim_cache size 0 max 16384 2024-09-21T06:18:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.930+0000 7f5bbe604740 1 Processor -- start 2024-09-21T06:18:13.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.930+0000 7f5bbe604740 1 -- start start 2024-09-21T06:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 0x55afa2286740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 0x55afa2286740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 0x55afa2286740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35980/0 (socket says 172.21.15.31:35980) 2024-09-21T06:18:13.934 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1087508699 learned_addr learned my addr 172.21.15.31:0/1087508699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1ec24d0 0x55afa22904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa2290a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01ab530 con 0x55afa1eba2f0 2024-09-21T06:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01add90 con 0x55afa1ee31b0 2024-09-21T06:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.931+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01abb80 con 0x55afa1ec24d0 2024-09-21T06:18:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1ec24d0 0x55afa22904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa2290a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1087508699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1ec24d0 msgr2=0x55afa22904e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1ec24d0 0x55afa22904e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1087508699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 msgr2=0x55afa2290a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa2290a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.932+0000 7f5bb5c31640 1 -- 172.21.15.31:0/1087508699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa2280b40 con 0x55afa1eba2f0 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.933+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 0x55afa2286740 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ba802ae70 tx=0x7f5ba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9ef07c31024c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:13.936 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.933+0000 7f5b9affd640 1 -- 172.21.15.31:0/1087508699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8048070 con 0x55afa1eba2f0 2024-09-21T06:18:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.933+0000 7f5b9affd640 1 -- 172.21.15.31:0/1087508699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba8061070 con 0x55afa1eba2f0 2024-09-21T06:18:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.934+0000 7f5b9affd640 1 -- 172.21.15.31:0/1087508699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80439d0 con 0x55afa1eba2f0 2024-09-21T06:18:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.934+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 msgr2=0x55afa2286740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.935+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 0x55afa2286740 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5ba802ae70 tx=0x7f5ba8004a40 comp rx=0 tx=0).stop 2024-09-21T06:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.936+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 shutdown_connections 2024-09-21T06:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.936+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa2290a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.936+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1ec24d0 0x55afa22904e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.936+0000 7f5bbe604740 1 --2- 172.21.15.31:0/1087508699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1eba2f0 0x55afa2286740 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.936+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 >> 172.21.15.31:0/1087508699 conn(0x55afa1ed7300 msgr2=0x55afa2286030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.937+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 shutdown_connections 2024-09-21T06:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.937+0000 7f5bbe604740 1 -- 172.21.15.31:0/1087508699 wait complete. 2024-09-21T06:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.938+0000 7f5bbe604740 1 -- start start 2024-09-21T06:18:13.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.938+0000 7f5bbe604740 1 Processor -- start 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa22a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ec24d0 0x55afa229fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bbe604740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa22a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa22a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bb5c31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa22a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49524/0 (socket says 172.21.15.31:49524) 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bb5c31640 1 -- 172.21.15.31:0/2308388635 learned_addr learned my addr 172.21.15.31:0/2308388635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01ac250 con 0x55afa1ec24d0 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.939+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ec24d0 0x55afa229fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01ad6c0 con 0x55afa1ee31b0 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb01ac920 con 0x55afa1eba2f0 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bbe604740 10 client.0 _ll_register_callbacks cb 0x7f5ba8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa22a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bb4c2f640 1 -- 172.21.15.31:0/2308388635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 msgr2=0x55afa22a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa22a52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bb4c2f640 1 -- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 msgr2=0x55afa22a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa22a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.940+0000 7f5bb4c2f640 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa22a0750 con 0x55afa1ec24d0 2024-09-21T06:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.941+0000 7f5bb4c2f640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ec24d0 0x55afa229fc50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55afa2299460 tx=0x7f5ba0012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996abcb2776c4c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.941+0000 7f5b98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.942+0000 7f5b98ff9640 1 -- 172.21.15.31:0/2308388635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0011870 con 0x55afa1ec24d0 2024-09-21T06:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.942+0000 7f5b98ff9640 1 -- 172.21.15.31:0/2308388635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8543+0+0 (secure 0 0 0) 0x7f5ba0011a10 con 0x55afa1ec24d0 2024-09-21T06:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.942+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba8061070 con 0x55afa1ec24d0 2024-09-21T06:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.942+0000 7f5b98ff9640 1 -- 172.21.15.31:0/2308388635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5ba0011c10 con 0x55afa1ec24d0 2024-09-21T06:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.942+0000 7f5bbe604740 10 client.16944 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.942+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55afa2320dc0 con 0x55afa1ec24d0 2024-09-21T06:18:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5b98ff9640 1 -- 172.21.15.31:0/2308388635 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba0011db0 con 0x55afa1ec24d0 2024-09-21T06:18:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5b98ff9640 1 -- 172.21.15.31:0/2308388635 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f5ba0023510 con 0x55afa1ec24d0 2024-09-21T06:18:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bbe604740 10 client.16944 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bbe604740 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bbe604740 20 client.16944 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bbe604740 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1e771c0 0x55afa23234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bbe604740 4 client.16944 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5ba8061070 con 0x55afa1e771c0 2024-09-21T06:18:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.943+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1e771c0 0x55afa23234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.944+0000 7f5b98ff9640 10 client.16944 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.944+0000 7f5bb5c31640 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1e771c0 0x55afa23234e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e879ffb8a123156e server_cookie=20666e09c7eb4fc7 in_seq=0 out_seq=0 2024-09-21T06:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.966+0000 7f5b98ff9640 1 -- 172.21.15.31:0/2308388635 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5ba8061070 con 0x55afa1e771c0 2024-09-21T06:18:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.966+0000 7f5b98ff9640 10 client.16944 handle_command_reply: tid=0 2024-09-21T06:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.966+0000 7f5bbe604740 1 client.16944 shutdown 2024-09-21T06:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ec24d0 msgr2=0x55afa229fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ec24d0 0x55afa229fc50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55afa2299460 tx=0x7f5ba0012b40 comp rx=0 tx=0).stop 2024-09-21T06:18:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 shutdown_connections 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55afa1e771c0 0x55afa23234e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afa1ee31b0 0x55afa22a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5b98ff9640 0 client.16944 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55afa1ec24d0 0x55afa229fc50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 --2- 172.21.15.31:0/2308388635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55afa1eba2f0 0x55afa22a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.967+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 >> 172.21.15.31:0/2308388635 conn(0x55afa1ed7300 msgr2=0x55afa22899c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.968+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 shutdown_connections 2024-09-21T06:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.969+0000 7f5bbe604740 1 -- 172.21.15.31:0/2308388635 wait complete. 2024-09-21T06:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.969+0000 7f5bbe604740 20 client.16944 trim_cache size 0 max 16384 2024-09-21T06:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:18:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:18:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.970+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b9c0b67d0 msgr2=0x7f5b9c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.970+0000 7f5bb7694640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b9c0b67d0 0x7f5b9c0b8ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55afa2287630 tx=0x7f5bac0a96e0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.970+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 msgr2=0x7f5bb019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.970+0000 7f5bb7694640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb019a1a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00e9f0 tx=0x7f5bac00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 shutdown_connections 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5bb010dd90 0x7f5bb019f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5b9c0b67d0 0x7f5b9c0b8ce0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bb0105130 0x7f5bb019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 --2- 172.21.15.31:0/2487688109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb01047e0 0x7f5bb019a1a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 >> 172.21.15.31:0/2487688109 conn(0x7f5bb007a6f0 msgr2=0x7f5bb01057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 shutdown_connections 2024-09-21T06:18:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:13.971+0000 7f5bb7694640 1 -- 172.21.15.31:0/2487688109 wait complete. 2024-09-21T06:18:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 395.0 s, 3014.4 tps, lat 33.932 ms stddev 60.663 2024-09-21T06:18:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:15 smithi110 ceph-mon[34550]: pgmap v520: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-09-21T06:18:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:15 smithi107 ceph-mon[34380]: pgmap v520: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-09-21T06:18:15.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:15 smithi031 ceph-mon[31237]: pgmap v520: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 4.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 28 MiB/s wr, 1.53k op/s 2024-09-21T06:18:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:17 smithi110 ceph-mon[34550]: pgmap v521: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2024-09-21T06:18:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:17 smithi107 ceph-mon[34380]: pgmap v521: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2024-09-21T06:18:17.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:17 smithi031 ceph-mon[31237]: pgmap v521: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 32 MiB/s wr, 1.80k op/s 2024-09-21T06:18:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:19 smithi110 ceph-mon[34550]: pgmap v522: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-09-21T06:18:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:19 smithi107 ceph-mon[34380]: pgmap v522: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-09-21T06:18:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:19 smithi031 ceph-mon[31237]: pgmap v522: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 1.79k op/s 2024-09-21T06:18:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 400.0 s, 2584.8 tps, lat 38.624 ms stddev 41.478 2024-09-21T06:18:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:21 smithi110 ceph-mon[34550]: pgmap v523: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2024-09-21T06:18:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:21 smithi107 ceph-mon[34380]: pgmap v523: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2024-09-21T06:18:21.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:21 smithi031 ceph-mon[31237]: pgmap v523: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 28 MiB/s wr, 1.60k op/s 2024-09-21T06:18:22.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:18:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 -- 172.21.15.31:0/146908832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f519810fb40 msgr2=0x7f519810ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 --2- 172.21.15.31:0/146908832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f519810fb40 0x7f519810ff50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5188009860 tx=0x7f518802a340 comp rx=0 tx=0).stop 2024-09-21T06:18:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 -- 172.21.15.31:0/146908832 shutdown_connections 2024-09-21T06:18:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 --2- 172.21.15.31:0/146908832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5198110490 0x7f5198075660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 --2- 172.21.15.31:0/146908832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f519810fb40 0x7f519810ff50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 --2- 172.21.15.31:0/146908832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f519810baf0 0x7f519810bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.574+0000 7f51a03db640 1 -- 172.21.15.31:0/146908832 >> 172.21.15.31:0/146908832 conn(0x7f5198070ec0 msgr2=0x7f51980712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:23.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.575+0000 7f51a03db640 1 -- 172.21.15.31:0/146908832 shutdown_connections 2024-09-21T06:18:23.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.575+0000 7f51a03db640 1 -- 172.21.15.31:0/146908832 wait complete. 2024-09-21T06:18:23.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.576+0000 7f51a03db640 1 Processor -- start 2024-09-21T06:18:23.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.576+0000 7f51a03db640 1 -- start start 2024-09-21T06:18:23.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.577+0000 7f51a03db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f519810baf0 0x7f51980806f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.577+0000 7f51a03db640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f519810fb40 0x7f5198080c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.577+0000 7f51a03db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 0x7f51981c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.577+0000 7f51a03db640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f519807ba90 con 0x7f519810baf0 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.577+0000 7f51a03db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f519807b910 con 0x7f5198110490 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.577+0000 7f51a03db640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f519807bc10 con 0x7f519810fb40 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 0x7f51981c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e177640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f519810baf0 0x7f51980806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 0x7f51981c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49898/0 (socket says 172.21.15.31:49898) 2024-09-21T06:18:23.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e177640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f519810baf0 0x7f51980806f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35292/0 (socket says 172.21.15.31:35292) 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 -- 172.21.15.31:0/3243890038 learned_addr learned my addr 172.21.15.31:0/3243890038 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519d976640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f519810fb40 0x7f5198080c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 -- 172.21.15.31:0/3243890038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f519810fb40 msgr2=0x7f5198080c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f519810fb40 0x7f5198080c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 -- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f519810baf0 msgr2=0x7f51980806f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f519810baf0 0x7f51980806f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.578+0000 7f519e978640 1 -- 172.21.15.31:0/3243890038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51981c66e0 con 0x7f5198110490 2024-09-21T06:18:23.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.579+0000 7f519e978640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 0x7f51981c61a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f519000b970 tx=0x7f519000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.579+0000 7f518f7fe640 1 -- 172.21.15.31:0/3243890038 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190019070 con 0x7f5198110490 2024-09-21T06:18:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.579+0000 7f518f7fe640 1 -- 172.21.15.31:0/3243890038 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5190014440 con 0x7f5198110490 2024-09-21T06:18:23.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.579+0000 7f518f7fe640 1 -- 172.21.15.31:0/3243890038 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51900075d0 con 0x7f5198110490 2024-09-21T06:18:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.579+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51981c68b0 con 0x7f5198110490 2024-09-21T06:18:23.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.579+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51981c6d70 con 0x7f5198110490 2024-09-21T06:18:23.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.583+0000 7f51a694f740 1 Processor -- start 2024-09-21T06:18:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.585+0000 7f51a694f740 1 -- start start 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f51a694f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95513b070 0x55b9554e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f51a694f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f51a694f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f51a694f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b955458e50 con 0x55b95513b070 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f51a694f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b955448280 con 0x55b9551121b0 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f51a694f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b95544c090 con 0x55b95511a390 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.586+0000 7f518f7fe640 1 -- 172.21.15.31:0/3243890038 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f519001c070 con 0x7f5198110490 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.587+0000 7f518f7fe640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51840b67f0 0x7f51840b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.587+0000 7f519e978640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51840b67f0 0x7f51840b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.588+0000 7f519e177640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.588+0000 7f518f7fe640 1 -- 172.21.15.31:0/3243890038 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f51900e6980 con 0x7f5198110490 2024-09-21T06:18:23.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.588+0000 7f519e177640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554e2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49910/0 (socket says 172.21.15.31:49910) 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.588+0000 7f519e177640 1 -- 172.21.15.31:0/2547028120 learned_addr learned my addr 172.21.15.31:0/2547028120 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.588+0000 7f519e978640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51840b67f0 0x7f51840b8d00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51900a8e00 tx=0x7f51900dd220 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.589+0000 7f519e177640 1 -- 172.21.15.31:0/2547028120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 msgr2=0x55b9554e30d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.589+0000 7f519e177640 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554e30d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.589+0000 7f519e177640 1 -- 172.21.15.31:0/2547028120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95513b070 msgr2=0x55b9554e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.589+0000 7f519e177640 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95513b070 0x55b9554e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.589+0000 7f519e177640 1 -- 172.21.15.31:0/2547028120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9551a6650 con 0x55b9551121b0 2024-09-21T06:18:23.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.589+0000 7f519e177640 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554e2b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f519400dc80 tx=0x7f519400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911a077bc35cfe3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.590+0000 7f518cd9a640 1 -- 172.21.15.31:0/2547028120 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194011440 con 0x55b9551121b0 2024-09-21T06:18:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.590+0000 7f518cd9a640 1 -- 172.21.15.31:0/2547028120 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194011a90 con 0x55b9551121b0 2024-09-21T06:18:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.591+0000 7f51a694f740 1 -- 172.21.15.31:0/2547028120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 msgr2=0x55b9554e2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.591+0000 7f518cd9a640 1 -- 172.21.15.31:0/2547028120 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194011440 con 0x55b9551121b0 2024-09-21T06:18:23.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.591+0000 7f51a694f740 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554e2b90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f519400dc80 tx=0x7f519400f730 comp rx=0 tx=0).stop 2024-09-21T06:18:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 -- 172.21.15.31:0/2547028120 shutdown_connections 2024-09-21T06:18:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554e30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554e2b90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 --2- 172.21.15.31:0/2547028120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95513b070 0x55b9554e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 -- 172.21.15.31:0/2547028120 >> 172.21.15.31:0/2547028120 conn(0x55b95512f460 msgr2=0x55b9554e0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:23.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 -- 172.21.15.31:0/2547028120 shutdown_connections 2024-09-21T06:18:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.592+0000 7f51a694f740 1 -- 172.21.15.31:0/2547028120 wait complete. 2024-09-21T06:18:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.593+0000 7f51a694f740 1 -- start start 2024-09-21T06:18:23.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.593+0000 7f51a694f740 1 Processor -- start 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.593+0000 7f51a694f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9551121b0 0x55b9554edcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.593+0000 7f51a694f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f51a694f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b95513b070 0x55b9554ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f51a694f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b9552a9700 con 0x55b9551121b0 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f51a694f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9553e3110 con 0x55b95513b070 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f51a694f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b955501e90 con 0x55b95511a390 2024-09-21T06:18:23.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f51a694f740 10 client.0 _ll_register_callbacks cb 0x55b9553fc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f519e177640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f519e177640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554ee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34200/0 (socket says 172.21.15.31:34200) 2024-09-21T06:18:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f519e177640 1 -- 172.21.15.31:0/2229037700 learned_addr learned my addr 172.21.15.31:0/2229037700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.594+0000 7f519d976640 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b95513b070 0x55b9554ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.595+0000 7f519e177640 1 -- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b95513b070 msgr2=0x55b9554ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:23.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.595+0000 7f519e177640 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b95513b070 0x55b9554ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.595+0000 7f519e177640 1 -- 172.21.15.31:0/2229037700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9551121b0 msgr2=0x55b9554edcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:23.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.595+0000 7f519e177640 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9551121b0 0x55b9554edcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.595+0000 7f519e177640 1 -- 172.21.15.31:0/2229037700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b955410c40 con 0x55b95511a390 2024-09-21T06:18:23.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.596+0000 7f519e177640 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554ee230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f519400dfa0 tx=0x7f519400dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857ae576f8f48ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.596+0000 7f5181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.596+0000 7f5181ffb640 1 -- 172.21.15.31:0/2229037700 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194004450 con 0x55b95511a390 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.596+0000 7f5181ffb640 1 -- 172.21.15.31:0/2229037700 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f5194011440 con 0x55b95511a390 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.596+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5194011a90 con 0x55b95511a390 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f5181ffb640 1 -- 172.21.15.31:0/2229037700 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5194011640 con 0x55b95511a390 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f51a694f740 10 client.26740 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:23.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b95550dc10 con 0x55b95511a390 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f5181ffb640 1 -- 172.21.15.31:0/2229037700 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194011a90 con 0x55b95511a390 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f5181ffb640 1 -- 172.21.15.31:0/2229037700 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f519402e030 con 0x55b95511a390 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f51a694f740 10 client.26740 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f51a694f740 10 client.26740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.597+0000 7f51a694f740 20 client.26740 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.598+0000 7f51a694f740 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550f99f0 0x55b955511020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.598+0000 7f51a694f740 4 client.26740 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:23.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.598+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b95545ada0 con 0x55b9550f99f0 2024-09-21T06:18:23.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.598+0000 7f519d976640 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550f99f0 0x55b955511020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.599+0000 7f5181ffb640 10 client.26740 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:23.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.599+0000 7f519d976640 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550f99f0 0x55b955511020 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ffffd8a9ec73a4 server_cookie=babf259acb81fc8d in_seq=0 out_seq=0 2024-09-21T06:18:23.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.600+0000 7f5181ffb640 1 -- 172.21.15.31:0/2229037700 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b95545ada0 con 0x55b9550f99f0 2024-09-21T06:18:23.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.600+0000 7f5181ffb640 10 client.26740 handle_command_reply: tid=0 2024-09-21T06:18:23.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.600+0000 7f51a694f740 1 client.26740 shutdown 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 msgr2=0x55b9554ee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554ee230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f519400dfa0 tx=0x7f519400dfd0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 shutdown_connections 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550f99f0 0x55b955511020 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b95513b070 0x55b9554ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95511a390 0x55b9554ee230 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 --2- 172.21.15.31:0/2229037700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9551121b0 0x55b9554edcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 >> 172.21.15.31:0/2229037700 conn(0x55b95512f460 msgr2=0x55b9554e5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:23.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f5181ffb640 0 client.26740 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:23.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.603+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 shutdown_connections 2024-09-21T06:18:23.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.604+0000 7f51a694f740 1 -- 172.21.15.31:0/2229037700 wait complete. 2024-09-21T06:18:23.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.604+0000 7f51a694f740 20 client.26740 trim_cache size 0 max 16384 2024-09-21T06:18:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.637+0000 7f51a694f740 1 Processor -- start 2024-09-21T06:18:23.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:23 smithi031 ceph-mon[31237]: pgmap v524: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.65k op/s 2024-09-21T06:18:23.639 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:23 smithi031 ceph-mon[31237]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:18:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.637+0000 7f51a694f740 1 -- start start 2024-09-21T06:18:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f51a694f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554ef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f51a694f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b95550b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f51a694f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b9554dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f519d976640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f519d976640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554ef210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49924/0 (socket says 172.21.15.31:49924) 2024-09-21T06:18:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f519d976640 1 -- 172.21.15.31:0/390234879 learned_addr learned my addr 172.21.15.31:0/390234879 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00c840 con 0x55b95511a390 2024-09-21T06:18:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.638+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00f0a0 con 0x55b9551121b0 2024-09-21T06:18:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00ce90 con 0x55b95513b070 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519e177640 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b95550b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519d976640 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b9554dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519d976640 1 -- 172.21.15.31:0/390234879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 msgr2=0x55b9554dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519d976640 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b9554dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519d976640 1 -- 172.21.15.31:0/390234879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 msgr2=0x55b95550b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519d976640 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b95550b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.639+0000 7f519d976640 1 -- 172.21.15.31:0/390234879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9554f00c0 con 0x55b9551121b0 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.640+0000 7f519d976640 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554ef210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51880092e0 tx=0x7f5188030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e4b37919f62a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.640+0000 7f5183fff640 1 -- 172.21.15.31:0/390234879 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188037040 con 0x55b9551121b0 2024-09-21T06:18:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.640+0000 7f5183fff640 1 -- 172.21.15.31:0/390234879 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b95545ada0 con 0x55b9551121b0 2024-09-21T06:18:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.640+0000 7f5183fff640 1 -- 172.21.15.31:0/390234879 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188004c80 con 0x55b9551121b0 2024-09-21T06:18:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.642+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 msgr2=0x55b9554ef210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.642+0000 7f51a694f740 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554ef210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f51880092e0 tx=0x7f5188030b50 comp rx=0 tx=0).stop 2024-09-21T06:18:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.642+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 shutdown_connections 2024-09-21T06:18:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.642+0000 7f51a694f740 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b9554dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.642+0000 7f51a694f740 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b95550b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.643+0000 7f51a694f740 1 --2- 172.21.15.31:0/390234879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b9554ef210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.643+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 >> 172.21.15.31:0/390234879 conn(0x55b95512f460 msgr2=0x55b9554e4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:23.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.643+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 shutdown_connections 2024-09-21T06:18:23.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.643+0000 7f51a694f740 1 -- 172.21.15.31:0/390234879 wait complete. 2024-09-21T06:18:23.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.644+0000 7f51a694f740 1 -- start start 2024-09-21T06:18:23.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.644+0000 7f51a694f740 1 Processor -- start 2024-09-21T06:18:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.645+0000 7f51a694f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b955514930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.645+0000 7f51a694f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b955511e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f519d976640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b955514930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f519d976640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b955514930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49934/0 (socket says 172.21.15.31:49934) 2024-09-21T06:18:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f51a694f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b955512430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f51a694f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00d560 con 0x55b95511a390 2024-09-21T06:18:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f51a694f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00e9d0 con 0x55b9551121b0 2024-09-21T06:18:23.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f51a694f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f517c00dc30 con 0x55b95513b070 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f51a694f740 10 client.0 _ll_register_callbacks cb 0x7f5194015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f519e177640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b955511e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f519e177640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b955511e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35334/0 (socket says 172.21.15.31:35334) 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.646+0000 7f519e177640 1 -- 172.21.15.31:0/1456180545 learned_addr learned my addr 172.21.15.31:0/1456180545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.647+0000 7f519e177640 1 -- 172.21.15.31:0/1456180545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 msgr2=0x55b955512430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.647+0000 7f519e177640 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b955512430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.647+0000 7f519e177640 1 -- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 msgr2=0x55b955514930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:23.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.647+0000 7f519e177640 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b955514930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.647+0000 7f519e177640 1 -- 172.21.15.31:0/1456180545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b955512bc0 con 0x55b95511a390 2024-09-21T06:18:23.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.649+0000 7f519e177640 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b955511e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f519400f770 tx=0x7f519400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143f3756339798ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:23.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.649+0000 7f5181ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:18:23.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.650+0000 7f5181ffb640 1 -- 172.21.15.31:0/1456180545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519403c070 con 0x55b95511a390 2024-09-21T06:18:23.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.650+0000 7f5181ffb640 1 -- 172.21.15.31:0/1456180545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f519401b600 con 0x55b95511a390 2024-09-21T06:18:23.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.650+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b95545ada0 con 0x55b95511a390 2024-09-21T06:18:23.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f5181ffb640 1 -- 172.21.15.31:0/1456180545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f51940117e0 con 0x55b95511a390 2024-09-21T06:18:23.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f51a694f740 10 client.16953 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:23.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b95557b7f0 con 0x55b95511a390 2024-09-21T06:18:23.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f5181ffb640 1 -- 172.21.15.31:0/1456180545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194011980 con 0x55b95511a390 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f5181ffb640 1 -- 172.21.15.31:0/1456180545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f519402e040 con 0x55b95511a390 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f51a694f740 10 client.16953 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f51a694f740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f51a694f740 20 client.16953 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.651+0000 7f51a694f740 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550ceda0 0x55b95557ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.652+0000 7f51a694f740 4 client.16953 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.652+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b95545ada0 con 0x55b9550ceda0 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.652+0000 7f519d976640 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550ceda0 0x55b95557ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:23.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.654+0000 7f5181ffb640 10 client.16953 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:23.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.654+0000 7f519d976640 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550ceda0 0x55b95557ded0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b70c7d4a4de2d8f server_cookie=4b120d71f01a8c8e in_seq=0 out_seq=0 2024-09-21T06:18:23.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.655+0000 7f5181ffb640 1 -- 172.21.15.31:0/1456180545 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x55b95545ada0 con 0x55b9550ceda0 2024-09-21T06:18:23.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.655+0000 7f5181ffb640 10 client.16953 handle_command_reply: tid=0 2024-09-21T06:18:23.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.655+0000 7f51a694f740 1 client.16953 shutdown 2024-09-21T06:18:23.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 msgr2=0x55b955511e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b955511e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f519400f770 tx=0x7f519400dc40 comp rx=0 tx=0).stop 2024-09-21T06:18:23.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 shutdown_connections 2024-09-21T06:18:23.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9550ceda0 0x55b95557ded0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f5181ffb640 0 client.16953 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:23.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b95513b070 0x55b955512430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b95511a390 0x55b955511e00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 --2- 172.21.15.31:0/1456180545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9551121b0 0x55b955514930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 >> 172.21.15.31:0/1456180545 conn(0x55b95512f460 msgr2=0x55b9554e6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 shutdown_connections 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 1 -- 172.21.15.31:0/1456180545 wait complete. 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.656+0000 7f51a694f740 20 client.16953 trim_cache size 0 max 16384 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:18:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.660 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:18:23.661 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.662 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:18:23.664 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:18:23.665 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:18:23.668 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.669 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:23.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.657+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51840b67f0 msgr2=0x7f51840b8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.657+0000 7f51a03db640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51840b67f0 0x7f51840b8d00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f51900a8e00 tx=0x7f51900dd220 comp rx=0 tx=0).stop 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.657+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 msgr2=0x7f51981c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.657+0000 7f51a03db640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 0x7f51981c61a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f519000b970 tx=0x7f519000be30 comp rx=0 tx=0).stop 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 shutdown_connections 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5198110490 0x7f51981c61a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f519810fb40 0x7f5198080c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51840b67f0 0x7f51840b8d00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 --2- 172.21.15.31:0/3243890038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f519810baf0 0x7f51980806f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 >> 172.21.15.31:0/3243890038 conn(0x7f5198070ec0 msgr2=0x7f519807cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 shutdown_connections 2024-09-21T06:18:23.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:23.658+0000 7f51a03db640 1 -- 172.21.15.31:0/3243890038 wait complete. 2024-09-21T06:18:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:23 smithi110 ceph-mon[34550]: pgmap v524: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.65k op/s 2024-09-21T06:18:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:23 smithi110 ceph-mon[34550]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:18:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:23 smithi107 ceph-mon[34380]: pgmap v524: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.65k op/s 2024-09-21T06:18:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:23 smithi107 ceph-mon[34380]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:18:24.528 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:18:24.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:18:24.965 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 405.0 s, 4166.2 tps, lat 23.909 ms stddev 27.991 2024-09-21T06:18:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.583+0000 7fbc76243640 1 -- 172.21.15.31:0/1448527305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70108cf0 msgr2=0x7fbc7010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.583+0000 7fbc76243640 1 --2- 172.21.15.31:0/1448527305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70108cf0 0x7fbc7010f520 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009900 tx=0x7fbc6002a410 comp rx=0 tx=0).stop 2024-09-21T06:18:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.584+0000 7fbc76243640 1 -- 172.21.15.31:0/1448527305 shutdown_connections 2024-09-21T06:18:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.584+0000 7fbc76243640 1 --2- 172.21.15.31:0/1448527305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70108cf0 0x7fbc7010f520 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.584+0000 7fbc76243640 1 --2- 172.21.15.31:0/1448527305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.584+0000 7fbc76243640 1 --2- 172.21.15.31:0/1448527305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc70102a60 0x7fbc70102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.584+0000 7fbc76243640 1 -- 172.21.15.31:0/1448527305 >> 172.21.15.31:0/1448527305 conn(0x7fbc70079730 msgr2=0x7fbc70079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.584+0000 7fbc76243640 1 -- 172.21.15.31:0/1448527305 shutdown_connections 2024-09-21T06:18:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.585+0000 7fbc76243640 1 -- 172.21.15.31:0/1448527305 wait complete. 2024-09-21T06:18:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.586+0000 7fbc76243640 1 Processor -- start 2024-09-21T06:18:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.586+0000 7fbc76243640 1 -- start start 2024-09-21T06:18:25.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc76243640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70102a60 0x7fbc70113560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:25.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc76243640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc70113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:25.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc76243640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc70108cf0 0x7fbc7011df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc76243640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701118a0 con 0x7fbc70108cf0 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc76243640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70111720 con 0x7fbc70102a60 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc76243640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70111a20 con 0x7fbc701083a0 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc70113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc70108cf0 0x7fbc7011df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc70113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34242/0 (socket says 172.21.15.31:34242) 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 -- 172.21.15.31:0/2153159326 learned_addr learned my addr 172.21.15.31:0/2153159326 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6f7fe640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70102a60 0x7fbc70113560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 -- 172.21.15.31:0/2153159326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70102a60 msgr2=0x7fbc70113560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70102a60 0x7fbc70113560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 -- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc70108cf0 msgr2=0x7fbc7011df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc70108cf0 0x7fbc7011df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6effd640 1 -- 172.21.15.31:0/2153159326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7011e660 con 0x7fbc701083a0 2024-09-21T06:18:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.587+0000 7fbc6f7fe640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70102a60 0x7fbc70113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.588+0000 7fbc6effd640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc70113ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400e9c0 tx=0x7fbc6400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.588+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400cd80 con 0x7fbc701083a0 2024-09-21T06:18:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.588+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc640054d0 con 0x7fbc701083a0 2024-09-21T06:18:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.588+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64010430 con 0x7fbc701083a0 2024-09-21T06:18:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.588+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc70118e00 con 0x7fbc701083a0 2024-09-21T06:18:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.588+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc701192f0 con 0x7fbc701083a0 2024-09-21T06:18:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.590+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c005290 con 0x7fbc701083a0 2024-09-21T06:18:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.591+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbc640105d0 con 0x7fbc701083a0 2024-09-21T06:18:25.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.593+0000 7fbc6cff9640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc580b67d0 0x7fbc580b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.594+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fbc640e78f0 con 0x7fbc701083a0 2024-09-21T06:18:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.594+0000 7fbc6f7fe640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc580b67d0 0x7fbc580b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.595+0000 7fbc6f7fe640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc580b67d0 0x7fbc580b8ce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbc70119e30 tx=0x7fbc68009990 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.597+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbc640addf0 con 0x7fbc701083a0 2024-09-21T06:18:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:25 smithi110 ceph-mon[34550]: pgmap v526: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 909 op/s 2024-09-21T06:18:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:25 smithi107 ceph-mon[34380]: pgmap v526: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 909 op/s 2024-09-21T06:18:25.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:25 smithi031 ceph-mon[31237]: pgmap v526: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 909 op/s 2024-09-21T06:18:25.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.991+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc5c005850 con 0x7fbc701083a0 2024-09-21T06:18:25.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.992+0000 7fbc6cff9640 1 -- 172.21.15.31:0/2153159326 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fbc640b21d0 con 0x7fbc701083a0 2024-09-21T06:18:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:18:25.995 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:18:25.995 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:18:25.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc580b67d0 msgr2=0x7fbc580b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:25.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc580b67d0 0x7fbc580b8ce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbc70119e30 tx=0x7fbc68009990 comp rx=0 tx=0).stop 2024-09-21T06:18:25.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 msgr2=0x7fbc70113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:25.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc70113ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400e9c0 tx=0x7fbc6400ee80 comp rx=0 tx=0).stop 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 shutdown_connections 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbc580b67d0 0x7fbc580b8ce0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc70108cf0 0x7fbc7011df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc701083a0 0x7fbc70113ae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 --2- 172.21.15.31:0/2153159326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc70102a60 0x7fbc70113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 >> 172.21.15.31:0/2153159326 conn(0x7fbc70079730 msgr2=0x7fbc701043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 shutdown_connections 2024-09-21T06:18:25.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:25.996+0000 7fbc76243640 1 -- 172.21.15.31:0/2153159326 wait complete. 2024-09-21T06:18:26.689 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:26 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2153159326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:18:26.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:18:26.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:26 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2153159326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:18:26.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:26 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2153159326' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 -- 172.21.15.31:0/2828300218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10cdc0 msgr2=0x7efe1c10d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 --2- 172.21.15.31:0/2828300218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10cdc0 0x7efe1c10d1d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efe0c014ce0 tx=0x7efe0c0327c0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 -- 172.21.15.31:0/2828300218 shutdown_connections 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 --2- 172.21.15.31:0/2828300218 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10ad10 0x7efe1c116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 --2- 172.21.15.31:0/2828300218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c10a3c0 0x7efe1c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 --2- 172.21.15.31:0/2828300218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10cdc0 0x7efe1c10d1d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.240+0000 7efe22750640 1 -- 172.21.15.31:0/2828300218 >> 172.21.15.31:0/2828300218 conn(0x7efe1c071080 msgr2=0x7efe1c073bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.242+0000 7efe22750640 1 -- 172.21.15.31:0/2828300218 shutdown_connections 2024-09-21T06:18:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.245+0000 7efe22750640 1 -- 172.21.15.31:0/2828300218 wait complete. 2024-09-21T06:18:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 Processor -- start 2024-09-21T06:18:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 -- start start 2024-09-21T06:18:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c10a3c0 0x7efe1c1b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10ad10 0x7efe1c1b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 0x7efe1c1b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c12ae70 con 0x7efe1c10a3c0 2024-09-21T06:18:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c12acf0 con 0x7efe1c10ad10 2024-09-21T06:18:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.246+0000 7efe22750640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c12aff0 con 0x7efe1c10cdc0 2024-09-21T06:18:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.247+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c10a3c0 0x7efe1c1b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.247+0000 7efe20ced640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 0x7efe1c1b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.247+0000 7efe20ced640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 0x7efe1c1b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34270/0 (socket says 172.21.15.31:34270) 2024-09-21T06:18:27.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.247+0000 7efe20ced640 1 -- 172.21.15.31:0/4096750700 learned_addr learned my addr 172.21.15.31:0/4096750700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.248+0000 7efe20ced640 1 -- 172.21.15.31:0/4096750700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10ad10 msgr2=0x7efe1c1b1e30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:18:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.248+0000 7efe20ced640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10ad10 0x7efe1c1b1e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.248+0000 7efe20ced640 1 -- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c10a3c0 msgr2=0x7efe1c1b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.248+0000 7efe20ced640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c10a3c0 0x7efe1c1b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.248+0000 7efe20ced640 1 -- 172.21.15.31:0/4096750700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c1b77c0 con 0x7efe1c10cdc0 2024-09-21T06:18:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.249+0000 7efe20ced640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 0x7efe1c1b70c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe1400bd10 tx=0x7efe14008600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.251+0000 7efe197fa640 1 -- 172.21.15.31:0/4096750700 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14004b20 con 0x7efe1c10cdc0 2024-09-21T06:18:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.251+0000 7efe22750640 1 -- 172.21.15.31:0/4096750700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c1b7ab0 con 0x7efe1c10cdc0 2024-09-21T06:18:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.251+0000 7efe22750640 1 -- 172.21.15.31:0/4096750700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c1be240 con 0x7efe1c10cdc0 2024-09-21T06:18:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.256+0000 7efe296bb740 1 Processor -- start 2024-09-21T06:18:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.256+0000 7efe296bb740 1 -- start start 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe296bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1acceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe296bb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c16fe2f0 0x5614c1acd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe296bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1acd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe296bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5614c1a416f0 con 0x5614c17064d0 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe296bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5614c1a30b20 con 0x5614c17271b0 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe296bb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5614c1a34930 con 0x5614c16fe2f0 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1acceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1acceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49980/0 (socket says 172.21.15.31:49980) 2024-09-21T06:18:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.257+0000 7efe1b7fe640 1 -- 172.21.15.31:0/2727637052 learned_addr learned my addr 172.21.15.31:0/2727637052 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.258+0000 7efe197fa640 1 -- 172.21.15.31:0/4096750700 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe14004cc0 con 0x7efe1c10cdc0 2024-09-21T06:18:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.258+0000 7efe197fa640 1 -- 172.21.15.31:0/4096750700 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14015420 con 0x7efe1c10cdc0 2024-09-21T06:18:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.258+0000 7efe1b7fe640 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1acd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.259+0000 7efe1bfff640 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c16fe2f0 0x5614c1acd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.259+0000 7efe1b7fe640 1 -- 172.21.15.31:0/2727637052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c16fe2f0 msgr2=0x5614c1acd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.259+0000 7efe1b7fe640 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c16fe2f0 0x5614c1acd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.259+0000 7efe1b7fe640 1 -- 172.21.15.31:0/2727637052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 msgr2=0x5614c1acd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.259+0000 7efe1b7fe640 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1acd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.260+0000 7efe1b7fe640 1 -- 172.21.15.31:0/2727637052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614c19f6530 con 0x5614c17271b0 2024-09-21T06:18:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.261+0000 7efe197fa640 1 -- 172.21.15.31:0/4096750700 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efe1401c020 con 0x7efe1c10cdc0 2024-09-21T06:18:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.262+0000 7efe197fa640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efe040b6850 0x7efe040b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.263+0000 7efe1b7fe640 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1acceb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efe100176f0 tx=0x7efe1000ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bfcab8622d4c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.263+0000 7efe197fa640 1 -- 172.21.15.31:0/4096750700 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7efe140e7360 con 0x7efe1c10cdc0 2024-09-21T06:18:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.267+0000 7efe01ffb640 1 -- 172.21.15.31:0/2727637052 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10017d30 con 0x5614c17271b0 2024-09-21T06:18:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.267+0000 7efe01ffb640 1 -- 172.21.15.31:0/2727637052 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe10005020 con 0x5614c17271b0 2024-09-21T06:18:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.267+0000 7efe01ffb640 1 -- 172.21.15.31:0/2727637052 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100049e0 con 0x5614c17271b0 2024-09-21T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.267+0000 7efe1bfff640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efe040b6850 0x7efe040b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.268+0000 7efe296bb740 1 -- 172.21.15.31:0/2727637052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 msgr2=0x5614c1acceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.268+0000 7efe296bb740 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1acceb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efe100176f0 tx=0x7efe1000ac00 comp rx=0 tx=0).stop 2024-09-21T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.269+0000 7efe296bb740 1 -- 172.21.15.31:0/2727637052 shutdown_connections 2024-09-21T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.269+0000 7efe296bb740 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1acd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.269+0000 7efe296bb740 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c16fe2f0 0x5614c1acd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.269+0000 7efe296bb740 1 --2- 172.21.15.31:0/2727637052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1acceb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.269+0000 7efe296bb740 1 -- 172.21.15.31:0/2727637052 >> 172.21.15.31:0/2727637052 conn(0x5614c171b300 msgr2=0x5614c1acada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.274+0000 7efe296bb740 1 -- 172.21.15.31:0/2727637052 shutdown_connections 2024-09-21T06:18:27.279 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.278+0000 7efe296bb740 1 -- 172.21.15.31:0/2727637052 wait complete. 2024-09-21T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.278+0000 7efe296bb740 1 -- start start 2024-09-21T06:18:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.278+0000 7efe296bb740 1 Processor -- start 2024-09-21T06:18:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.279+0000 7efe296bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ad8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.279+0000 7efe296bb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ad88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.280+0000 7efe296bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 0x5614c1ad8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.280+0000 7efe296bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5614c1968ab0 con 0x5614c17271b0 2024-09-21T06:18:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.280+0000 7efe296bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5614c19d6730 con 0x5614c16fe2f0 2024-09-21T06:18:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.280+0000 7efe296bb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5614c1adb480 con 0x5614c17064d0 2024-09-21T06:18:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.280+0000 7efe296bb740 10 client.0 _ll_register_callbacks cb 0x5614c1a2a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.281+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ad88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.283+0000 7efe20ced640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 0x5614c1ad8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.283+0000 7efe20ced640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 0x5614c1ad8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35394/0 (socket says 172.21.15.31:35394) 2024-09-21T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.283+0000 7efe20ced640 1 -- 172.21.15.31:0/4142702317 learned_addr learned my addr 172.21.15.31:0/4142702317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe1bfff640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ad8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe1bfff640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efe040b6850 0x7efe040b8d60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe0c032ce0 tx=0x7efe0c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe20ced640 1 -- 172.21.15.31:0/4142702317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 msgr2=0x5614c1ad88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe20ced640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ad88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe20ced640 1 -- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 msgr2=0x5614c1ad8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe20ced640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ad8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe20ced640 1 -- 172.21.15.31:0/4142702317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614c1792790 con 0x5614c17271b0 2024-09-21T06:18:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe1b7fe640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ad88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.284+0000 7efe1bfff640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ad8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.285+0000 7efe20ced640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 0x5614c1ad8e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe140ae0c0 tx=0x7efe140ad760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ef20ccf6ea51b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.285+0000 7efdfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:18:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.285+0000 7efdfb7fe640 1 -- 172.21.15.31:0/4142702317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140b42d0 con 0x5614c17271b0 2024-09-21T06:18:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.286+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe10005020 con 0x5614c17271b0 2024-09-21T06:18:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.286+0000 7efdfb7fe640 1 -- 172.21.15.31:0/4142702317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7efe140a9a80 con 0x5614c17271b0 2024-09-21T06:18:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.287+0000 7efdfb7fe640 1 -- 172.21.15.31:0/4142702317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efe140adc40 con 0x5614c17271b0 2024-09-21T06:18:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.289+0000 7efe296bb740 10 client.16968 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.289+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5614c180b620 con 0x5614c17271b0 2024-09-21T06:18:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.290+0000 7efdfb7fe640 1 -- 172.21.15.31:0/4142702317 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe10005020 con 0x5614c17271b0 2024-09-21T06:18:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.290+0000 7efdfb7fe640 1 -- 172.21.15.31:0/4142702317 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efe140991f0 con 0x5614c17271b0 2024-09-21T06:18:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.291+0000 7efe296bb740 10 client.16968 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.291+0000 7efe296bb740 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.291+0000 7efe296bb740 20 client.16968 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.291+0000 7efe296bb740 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16e5d80 0x5614c1aea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.291+0000 7efe296bb740 4 client.16968 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:27.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.291+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614c1a43640 con 0x5614c16e5d80 2024-09-21T06:18:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.293+0000 7efe1bfff640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16e5d80 0x5614c1aea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.295+0000 7efe1bfff640 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16e5d80 0x5614c1aea880 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa4b292ecfd399a server_cookie=19fc9a994621992e in_seq=0 out_seq=0 2024-09-21T06:18:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.295+0000 7efdfb7fe640 10 client.16968 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.296+0000 7efdfb7fe640 1 -- 172.21.15.31:0/4142702317 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614c1a43640 con 0x5614c16e5d80 2024-09-21T06:18:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.296+0000 7efdfb7fe640 10 client.16968 handle_command_reply: tid=0 2024-09-21T06:18:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.296+0000 7efe296bb740 1 client.16968 shutdown 2024-09-21T06:18:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.297+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 msgr2=0x5614c1ad8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.297+0000 7efe296bb740 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 0x5614c1ad8e00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efe140ae0c0 tx=0x7efe140ad760 comp rx=0 tx=0).stop 2024-09-21T06:18:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.300+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 shutdown_connections 2024-09-21T06:18:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.300+0000 7efe296bb740 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16e5d80 0x5614c1aea880 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.300+0000 7efe296bb740 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17271b0 0x5614c1ad8e00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.300+0000 7efe296bb740 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ad88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.300+0000 7efe296bb740 1 --2- 172.21.15.31:0/4142702317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ad8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.300+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 >> 172.21.15.31:0/4142702317 conn(0x5614c171b300 msgr2=0x5614c1acdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.301+0000 7efdfb7fe640 0 client.16968 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.305+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 shutdown_connections 2024-09-21T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.305+0000 7efe296bb740 1 -- 172.21.15.31:0/4142702317 wait complete. 2024-09-21T06:18:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.305+0000 7efe296bb740 20 client.16968 trim_cache size 0 max 16384 2024-09-21T06:18:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.382+0000 7efe296bb740 1 Processor -- start 2024-09-21T06:18:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.382+0000 7efe296bb740 1 -- start start 2024-09-21T06:18:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.401+0000 7efe296bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ace8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.401+0000 7efe296bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1ad9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.401+0000 7efe296bb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17271b0 0x5614c1ad5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.401+0000 7efe296bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800c880 con 0x5614c17064d0 2024-09-21T06:18:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.401+0000 7efe296bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800f0e0 con 0x5614c16fe2f0 2024-09-21T06:18:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.401+0000 7efe296bb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800ced0 con 0x5614c17271b0 2024-09-21T06:18:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.406+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ace8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.407+0000 7efe1bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ace8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50002/0 (socket says 172.21.15.31:50002) 2024-09-21T06:18:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.407+0000 7efe1bfff640 1 -- 172.21.15.31:0/942407001 learned_addr learned my addr 172.21.15.31:0/942407001 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.407+0000 7efe20ced640 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17271b0 0x5614c1ad5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.407+0000 7efe1b7fe640 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1ad9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.408+0000 7efe1bfff640 1 -- 172.21.15.31:0/942407001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17271b0 msgr2=0x5614c1ad5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.408+0000 7efe1bfff640 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17271b0 0x5614c1ad5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.408+0000 7efe1bfff640 1 -- 172.21.15.31:0/942407001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 msgr2=0x5614c1ad9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.408+0000 7efe1bfff640 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1ad9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.408+0000 7efe1bfff640 1 -- 172.21.15.31:0/942407001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614c1acfcc0 con 0x5614c16fe2f0 2024-09-21T06:18:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.408+0000 7efe20ced640 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17271b0 0x5614c1ad5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.416+0000 7efe1bfff640 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ace8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe0c03ac60 tx=0x7efe0c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101d24f392559b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.424+0000 7efe017fa640 1 -- 172.21.15.31:0/942407001 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c052020 con 0x5614c16fe2f0 2024-09-21T06:18:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.424+0000 7efe017fa640 1 -- 172.21.15.31:0/942407001 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe0c03a8f0 con 0x5614c16fe2f0 2024-09-21T06:18:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.424+0000 7efe017fa640 1 -- 172.21.15.31:0/942407001 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c012070 con 0x5614c16fe2f0 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.425+0000 7efe296bb740 1 -- 172.21.15.31:0/942407001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 msgr2=0x5614c1ace8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.425+0000 7efe296bb740 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ace8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe0c03ac60 tx=0x7efe0c014000 comp rx=0 tx=0).stop 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 -- 172.21.15.31:0/942407001 shutdown_connections 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17271b0 0x5614c1ad5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c17064d0 0x5614c1ad9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 --2- 172.21.15.31:0/942407001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c16fe2f0 0x5614c1ace8a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 -- 172.21.15.31:0/942407001 >> 172.21.15.31:0/942407001 conn(0x5614c171b300 msgr2=0x5614c1ad98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 -- 172.21.15.31:0/942407001 shutdown_connections 2024-09-21T06:18:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.426+0000 7efe296bb740 1 -- 172.21.15.31:0/942407001 wait complete. 2024-09-21T06:18:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.427+0000 7efe296bb740 1 -- start start 2024-09-21T06:18:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.427+0000 7efe296bb740 1 Processor -- start 2024-09-21T06:18:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.428+0000 7efe296bb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c16fe2f0 0x5614c1addec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.428+0000 7efe296bb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ade400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe296bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1b53240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe296bb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800d5a0 con 0x5614c16fe2f0 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe296bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800ea10 con 0x5614c17271b0 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ade400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe1b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ade400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34302/0 (socket says 172.21.15.31:34302) 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe1b7fe640 1 -- 172.21.15.31:0/1527476632 learned_addr learned my addr 172.21.15.31:0/1527476632 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe20ced640 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1b53240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800dc70 con 0x5614c17064d0 2024-09-21T06:18:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.429+0000 7efe296bb740 10 client.0 _ll_register_callbacks cb 0x7efe14004ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.431+0000 7efe20ced640 1 -- 172.21.15.31:0/1527476632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 msgr2=0x5614c1ade400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.431+0000 7efe20ced640 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ade400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.431+0000 7efe20ced640 1 -- 172.21.15.31:0/1527476632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c16fe2f0 msgr2=0x5614c1addec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.431+0000 7efe20ced640 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c16fe2f0 0x5614c1addec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.431+0000 7efe20ced640 1 -- 172.21.15.31:0/1527476632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614c1ae46d0 con 0x5614c17271b0 2024-09-21T06:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.432+0000 7efe20ced640 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1b53240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe140aa700 tx=0x7efe140b4300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ea38fdfedf28e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.432+0000 7efdfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.434+0000 7efdfaffd640 1 -- 172.21.15.31:0/1527476632 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140aa780 con 0x5614c17271b0 2024-09-21T06:18:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.434+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0c03a8f0 con 0x5614c17271b0 2024-09-21T06:18:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.434+0000 7efdfaffd640 1 -- 172.21.15.31:0/1527476632 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7efe140a9a80 con 0x5614c17271b0 2024-09-21T06:18:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.435+0000 7efdfaffd640 1 -- 172.21.15.31:0/1527476632 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efe140adf00 con 0x5614c17271b0 2024-09-21T06:18:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.435+0000 7efe296bb740 10 client.26705 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.436+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5614c1b54880 con 0x5614c17271b0 2024-09-21T06:18:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.437+0000 7efdfaffd640 1 -- 172.21.15.31:0/1527476632 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe0c03a8f0 con 0x5614c17271b0 2024-09-21T06:18:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efdfaffd640 1 -- 172.21.15.31:0/1527476632 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efe1405d1c0 con 0x5614c17271b0 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe296bb740 10 client.26705 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe296bb740 10 client.26705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe296bb740 20 client.26705 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe296bb740 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16bb1c0 0x5614c1b57040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe296bb740 4 client.26705 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614c1a3f0f0 con 0x5614c16bb1c0 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.439+0000 7efe1bfff640 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16bb1c0 0x5614c1b57040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.444+0000 7efe1bfff640 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16bb1c0 0x5614c1b57040 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4213df114520737 server_cookie=56b742b7581e8f17 in_seq=0 out_seq=0 2024-09-21T06:18:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.444+0000 7efdfaffd640 10 client.26705 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:27 smithi110 ceph-mon[34550]: pgmap v527: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.7 MiB/s wr, 775 op/s 2024-09-21T06:18:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:27 smithi107 ceph-mon[34380]: pgmap v527: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.7 MiB/s wr, 775 op/s 2024-09-21T06:18:27.810 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:27 smithi031 ceph-mon[31237]: pgmap v527: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.7 MiB/s wr, 775 op/s 2024-09-21T06:18:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.877+0000 7efdfaffd640 1 -- 172.21.15.31:0/1527476632 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614c1a3f0f0 con 0x5614c16bb1c0 2024-09-21T06:18:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.877+0000 7efdfaffd640 10 client.26705 handle_command_reply: tid=0 2024-09-21T06:18:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.877+0000 7efe296bb740 1 client.26705 shutdown 2024-09-21T06:18:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 msgr2=0x5614c1b53240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1b53240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe140aa700 tx=0x7efe140b4300 comp rx=0 tx=0).stop 2024-09-21T06:18:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 shutdown_connections 2024-09-21T06:18:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614c16bb1c0 0x5614c1b57040 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614c17271b0 0x5614c1b53240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614c17064d0 0x5614c1ade400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 --2- 172.21.15.31:0/1527476632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614c16fe2f0 0x5614c1addec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 >> 172.21.15.31:0/1527476632 conn(0x5614c171b300 msgr2=0x5614c1ace5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efdfaffd640 0 client.26705 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 shutdown_connections 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 1 -- 172.21.15.31:0/1527476632 wait complete. 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.878+0000 7efe296bb740 20 client.26705 trim_cache size 0 max 16384 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:18:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:18:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:18:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.880+0000 7efe02ffd640 1 -- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efe040b6850 msgr2=0x7efe040b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.880+0000 7efe02ffd640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efe040b6850 0x7efe040b8d60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe0c032ce0 tx=0x7efe0c005850 comp rx=0 tx=0).stop 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.880+0000 7efe02ffd640 1 -- 172.21.15.31:0/4096750700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 msgr2=0x7efe1c1b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.880+0000 7efe02ffd640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 0x7efe1c1b70c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efe1400bd10 tx=0x7efe14008600 comp rx=0 tx=0).stop 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.880+0000 7efe02ffd640 1 -- 172.21.15.31:0/4096750700 shutdown_connections 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.880+0000 7efe02ffd640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe1c10cdc0 0x7efe1c1b70c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.881+0000 7efe02ffd640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c10ad10 0x7efe1c1b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.881+0000 7efe02ffd640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efe040b6850 0x7efe040b8d60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.881+0000 7efe02ffd640 1 --2- 172.21.15.31:0/4096750700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe1c10a3c0 0x7efe1c1b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.881+0000 7efe02ffd640 1 -- 172.21.15.31:0/4096750700 >> 172.21.15.31:0/4096750700 conn(0x7efe1c071080 msgr2=0x7efe1c10f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.882+0000 7efe02ffd640 1 -- 172.21.15.31:0/4096750700 shutdown_connections 2024-09-21T06:18:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:27.882+0000 7efe02ffd640 1 -- 172.21.15.31:0/4096750700 wait complete. 2024-09-21T06:18:28.420 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 -- 172.21.15.31:0/3822376788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0103110 msgr2=0x7f71f0107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 --2- 172.21.15.31:0/3822376788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0103110 0x7f71f0107640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f71e001a940 tx=0x7f71e003b490 comp rx=0 tx=0).stop 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 -- 172.21.15.31:0/3822376788 shutdown_connections 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 --2- 172.21.15.31:0/3822376788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f0107c00 0x7f71f01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 --2- 172.21.15.31:0/3822376788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0103110 0x7f71f0107640 secure :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f71e001a940 tx=0x7f71e003b490 comp rx=0 tx=0).stop 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 --2- 172.21.15.31:0/3822376788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f01027c0 0x7f71f0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.978+0000 7f71f8287640 1 -- 172.21.15.31:0/3822376788 >> 172.21.15.31:0/3822376788 conn(0x7f71f0079740 msgr2=0x7f71f0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.979+0000 7f71f8287640 1 -- 172.21.15.31:0/3822376788 shutdown_connections 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.979+0000 7f71f8287640 1 -- 172.21.15.31:0/3822376788 wait complete. 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 Processor -- start 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 -- start start 2024-09-21T06:18:28.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f01027c0 0x7f71f00744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0107c00 0x7f71f0074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0110440 0x7f71f0110850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71f010b5a0 con 0x7f71f0107c00 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71f010b420 con 0x7f71f01027c0 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.980+0000 7f71f8287640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71f010b720 con 0x7f71f0110440 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.981+0000 7f71f6023640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f01027c0 0x7f71f00744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.981+0000 7f71f6023640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f01027c0 0x7f71f00744a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50030/0 (socket says 172.21.15.31:50030) 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.981+0000 7f71f6023640 1 -- 172.21.15.31:0/4013822406 learned_addr learned my addr 172.21.15.31:0/4013822406 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.981+0000 7f71f6824640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0110440 0x7f71f0110850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.982+0000 7f71f5822640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0107c00 0x7f71f0074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.982+0000 7f71f6824640 1 -- 172.21.15.31:0/4013822406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f01027c0 msgr2=0x7f71f00744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.982+0000 7f71f6824640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f01027c0 0x7f71f00744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.982+0000 7f71f6824640 1 -- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0107c00 msgr2=0x7f71f0074a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.982+0000 7f71f6824640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0107c00 0x7f71f0074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.982+0000 7f71f6824640 1 -- 172.21.15.31:0/4013822406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f0111070 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.984+0000 7f71f6824640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0110440 0x7f71f0110850 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71e800be20 tx=0x7f71e800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.984+0000 7f71e77fe640 1 -- 172.21.15.31:0/4013822406 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8019070 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.984+0000 7f71e77fe640 1 -- 172.21.15.31:0/4013822406 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e80148f0 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.985+0000 7f71e77fe640 1 -- 172.21.15.31:0/4013822406 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8012840 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.985+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f0115520 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.986+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f0115a60 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.990+0000 7f71e77fe640 1 -- 172.21.15.31:0/4013822406 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f71e8014440 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.991+0000 7f71e77fe640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71dc0b6880 0x7f71dc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.991+0000 7f71e77fe640 1 -- 172.21.15.31:0/4013822406 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f71e80e6d20 con 0x7f71f0110440 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.994+0000 7f71f6023640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71dc0b6880 0x7f71dc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.995+0000 7f71fe7fb740 1 Processor -- start 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 -- start start 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad0396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad039be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad03a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5578acfafea0 con 0x5578acc691b0 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5578acf9f2d0 con 0x5578acc92070 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71fe7fb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5578acfa30e0 con 0x5578acc71390 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.996+0000 7f71f6023640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad0396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad0396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50034/0 (socket says 172.21.15.31:50034) 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 -- 172.21.15.31:0/3643846793 learned_addr learned my addr 172.21.15.31:0/3643846793 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6824640 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad03a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71dc0b6880 0x7f71dc0b8d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71ec02bec0 tx=0x7f71ec0376e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 -- 172.21.15.31:0/3643846793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 msgr2=0x5578ad03a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad03a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 -- 172.21.15.31:0/3643846793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 msgr2=0x5578ad039be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad039be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 -- 172.21.15.31:0/3643846793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578accfd650 con 0x5578acc92070 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.997+0000 7f71f6023640 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad0396a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f71ec02bfd0 tx=0x7f71ec03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e63ed89ab94b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.998+0000 7f71e4d9a640 1 -- 172.21.15.31:0/3643846793 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0423f0 con 0x5578acc92070 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.998+0000 7f71e4d9a640 1 -- 172.21.15.31:0/3643846793 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec047070 con 0x5578acc92070 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.998+0000 7f71e4d9a640 1 -- 172.21.15.31:0/3643846793 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0347d0 con 0x5578acc92070 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.999+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3643846793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 msgr2=0x5578ad0396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:27.999+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad0396a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f71ec02bfd0 tx=0x7f71ec03f000 comp rx=0 tx=0).stop 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.000+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3643846793 shutdown_connections 2024-09-21T06:18:28.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.000+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad03a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.000+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad039be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.000+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3643846793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad0396a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.000+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3643846793 >> 172.21.15.31:0/3643846793 conn(0x5578acc86460 msgr2=0x5578ad037590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.001+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3643846793 shutdown_connections 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.001+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3643846793 wait complete. 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.002+0000 7f71fe7fb740 1 -- start start 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.003+0000 7f71fe7fb740 1 Processor -- start 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.004+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad044de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.004+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad045320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.004+0000 7f71f6023640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad044de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.004+0000 7f71f6023640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad044de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35456/0 (socket says 172.21.15.31:35456) 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.005+0000 7f71f6023640 1 -- 172.21.15.31:0/3693722798 learned_addr learned my addr 172.21.15.31:0/3693722798 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.005+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad051110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.005+0000 7f71f5822640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad045320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.005+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5578ace00700 con 0x5578acc691b0 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.005+0000 7f71f6824640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad051110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5578acf3a160 con 0x5578acc92070 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5578ad047e20 con 0x5578acc71390 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71fe7fb740 10 client.0 _ll_register_callbacks cb 0x5578acf539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71f6824640 1 -- 172.21.15.31:0/3693722798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 msgr2=0x5578ad045320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71f6824640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad045320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71f6824640 1 -- 172.21.15.31:0/3693722798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 msgr2=0x5578ad044de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.006+0000 7f71f6824640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad044de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.007+0000 7f71f6824640 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578acf67c90 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.007+0000 7f71f6824640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad051110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f71e80e4420 tx=0x7f71e80a8340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ce4d0fdf14e26d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.007+0000 7f71d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.007+0000 7f71d9ffb640 1 -- 172.21.15.31:0/3693722798 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80b1070 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.007+0000 7f71d9ffb640 1 -- 172.21.15.31:0/3693722798 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f71e80b1210 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.008+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec047070 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.010+0000 7f71d9ffb640 1 -- 172.21.15.31:0/3693722798 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f71e80b1600 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.010+0000 7f71fe7fb740 10 client.26714 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.010+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5578ad053a20 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71d9ffb640 1 -- 172.21.15.31:0/3693722798 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f71e80e46a0 con 0x5578acc92070 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71fe7fb740 10 client.26714 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71fe7fb740 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:28.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71fe7fb740 20 client.26714 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc509f0 0x5578ad056e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71fe7fb740 4 client.26714 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578acfb1df0 con 0x5578acc509f0 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.011+0000 7f71f6023640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc509f0 0x5578ad056e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.012+0000 7f71f6023640 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc509f0 0x5578ad056e10 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5e4a11df2b0917 server_cookie=664c9260c10b6292 in_seq=0 out_seq=0 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.014+0000 7f71d9ffb640 10 client.26714 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.014+0000 7f71d9ffb640 1 -- 172.21.15.31:0/3693722798 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578acfb1df0 con 0x5578acc509f0 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.014+0000 7f71d9ffb640 10 client.26714 handle_command_reply: tid=0 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.014+0000 7f71fe7fb740 1 client.26714 shutdown 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.014+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 msgr2=0x5578ad051110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.014+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad051110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f71e80e4420 tx=0x7f71e80a8340 comp rx=0 tx=0).stop 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 shutdown_connections 2024-09-21T06:18:28.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc509f0 0x5578ad056e10 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71d9ffb640 0 client.26714 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc92070 0x5578ad051110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc71390 0x5578ad045320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/3693722798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc691b0 0x5578ad044de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 >> 172.21.15.31:0/3693722798 conn(0x5578acc86460 msgr2=0x5578ad03c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 shutdown_connections 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.016+0000 7f71fe7fb740 1 -- 172.21.15.31:0/3693722798 wait complete. 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.017+0000 7f71fe7fb740 20 client.26714 trim_cache size 0 max 16384 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 Processor -- start 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 -- start start 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc509f0 0x5578ad0452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc691b0 0x5578ad036bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad037100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b4350 con 0x5578acc71390 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b6b90 con 0x5578acc509f0 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71fe7fb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b49a0 con 0x5578acc691b0 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.046+0000 7f71f6023640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc509f0 0x5578ad0452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6023640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc509f0 0x5578ad0452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50058/0 (socket says 172.21.15.31:50058) 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6023640 1 -- 172.21.15.31:0/1320946544 learned_addr learned my addr 172.21.15.31:0/1320946544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6824640 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad037100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6824640 1 -- 172.21.15.31:0/1320946544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc691b0 msgr2=0x5578ad036bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6824640 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc691b0 0x5578ad036bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6824640 1 -- 172.21.15.31:0/1320946544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc509f0 msgr2=0x5578ad0452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6824640 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc509f0 0x5578ad0452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.047+0000 7f71f6824640 1 -- 172.21.15.31:0/1320946544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ad0460a0 con 0x5578acc71390 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.048+0000 7f71f6824640 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad037100 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009fb0 tx=0x7f71e80e6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401eae5fbc1a8e79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.048+0000 7f71dbfff640 1 -- 172.21.15.31:0/1320946544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80c9070 con 0x5578acc71390 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.048+0000 7f71dbfff640 1 -- 172.21.15.31:0/1320946544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec047070 con 0x5578acc71390 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.048+0000 7f71dbfff640 1 -- 172.21.15.31:0/1320946544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80b1040 con 0x5578acc71390 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.049+0000 7f71fe7fb740 1 -- 172.21.15.31:0/1320946544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 msgr2=0x5578ad037100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.049+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad037100 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009fb0 tx=0x7f71e80e6000 comp rx=0 tx=0).stop 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 -- 172.21.15.31:0/1320946544 shutdown_connections 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad037100 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc691b0 0x5578ad036bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/1320946544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc509f0 0x5578ad0452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 -- 172.21.15.31:0/1320946544 >> 172.21.15.31:0/1320946544 conn(0x5578acc86460 msgr2=0x5578ad044b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 -- 172.21.15.31:0/1320946544 shutdown_connections 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.050+0000 7f71fe7fb740 1 -- 172.21.15.31:0/1320946544 wait complete. 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.051+0000 7f71fe7fb740 1 -- start start 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.051+0000 7f71fe7fb740 1 Processor -- start 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc509f0 0x5578ad0490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 0x5578ad0495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71fe7fb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad049b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71f6023640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc509f0 0x5578ad0490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71f6023640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc509f0 0x5578ad0490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34348/0 (socket says 172.21.15.31:34348) 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71f5822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 0x5578ad0495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71f5822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 0x5578ad0495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50068/0 (socket says 172.21.15.31:50068) 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.052+0000 7f71f6023640 1 -- 172.21.15.31:0/897069911 learned_addr learned my addr 172.21.15.31:0/897069911 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b5050 con 0x5578acc71390 2024-09-21T06:18:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b64c0 con 0x5578acc691b0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71f01b5720 con 0x5578acc509f0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71fe7fb740 10 client.0 _ll_register_callbacks cb 0x7f71e800ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71f5822640 1 -- 172.21.15.31:0/897069911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc509f0 msgr2=0x5578ad0490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71f5822640 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc509f0 0x5578ad0490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71f5822640 1 -- 172.21.15.31:0/897069911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 msgr2=0x5578ad049b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71f5822640 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad049b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71f5822640 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ad0c0860 con 0x5578acc691b0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.053+0000 7f71f5822640 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 0x5578ad0495e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f71f006bb30 tx=0x7f71e0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2624913dbd11ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71d9ffb640 1 -- 172.21.15.31:0/897069911 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0013070 con 0x5578acc691b0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71d9ffb640 1 -- 172.21.15.31:0/897069911 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f71e0040990 con 0x5578acc691b0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec047070 con 0x5578acc691b0 2024-09-21T06:18:28.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71d9ffb640 1 -- 172.21.15.31:0/897069911 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f71e0046040 con 0x5578acc691b0 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71fe7fb740 10 client.26720 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.054+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5578ad0c1aa0 con 0x5578acc691b0 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.055+0000 7f71d9ffb640 1 -- 172.21.15.31:0/897069911 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec047070 con 0x5578acc691b0 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.055+0000 7f71d9ffb640 1 -- 172.21.15.31:0/897069911 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f71e00046c0 con 0x5578acc691b0 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.055+0000 7f71fe7fb740 10 client.26720 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.055+0000 7f71fe7fb740 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71fe7fb740 20 client.26720 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc92070 0x5578ad0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71fe7fb740 4 client.26720 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578acfad8a0 con 0x5578acc92070 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71f6023640 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc92070 0x5578ad0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71d9ffb640 10 client.26720 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:28.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.056+0000 7f71f6023640 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc92070 0x5578ad0c4220 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0e14451a71cc0c server_cookie=3b53d3520b5ecbdb in_seq=0 out_seq=0 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71d9ffb640 1 -- 172.21.15.31:0/897069911 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578acfad8a0 con 0x5578acc92070 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71d9ffb640 10 client.26720 handle_command_reply: tid=0 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 client.26720 shutdown 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 msgr2=0x5578ad0495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 0x5578ad0495e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f71f006bb30 tx=0x7f71e0004270 comp rx=0 tx=0).stop 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 shutdown_connections 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5578acc92070 0x5578ad0c4220 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5578acc71390 0x5578ad049b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5578acc691b0 0x5578ad0495e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 --2- 172.21.15.31:0/897069911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5578acc509f0 0x5578ad0490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 >> 172.21.15.31:0/897069911 conn(0x5578acc86460 msgr2=0x5578ad03c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71d9ffb640 0 client.26720 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:28.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 shutdown_connections 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.057+0000 7f71fe7fb740 1 -- 172.21.15.31:0/897069911 wait complete. 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71fe7fb740 20 client.26720 trim_cache size 0 max 16384 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71dc0b6880 msgr2=0x7f71dc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71dc0b6880 0x7f71dc0b8d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f71ec02bec0 tx=0x7f71ec0376e0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0110440 msgr2=0x7f71f0110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0110440 0x7f71f0110850 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71e800be20 tx=0x7f71e800bf10 comp rx=0 tx=0).stop 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 shutdown_connections 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71f0110440 0x7f71f0110850 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f71dc0b6880 0x7f71dc0b8d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71f0107c00 0x7f71f0074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.058+0000 7f71f8287640 1 --2- 172.21.15.31:0/4013822406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71f01027c0 0x7f71f00744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.059+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 >> 172.21.15.31:0/4013822406 conn(0x7f71f0079740 msgr2=0x7f71f01058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.059+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 shutdown_connections 2024-09-21T06:18:28.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:28.059+0000 7f71f8287640 1 -- 172.21.15.31:0/4013822406 wait complete. 2024-09-21T06:18:29.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:29 smithi031 ceph-mon[31237]: pgmap v528: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.4 MiB/s wr, 731 op/s 2024-09-21T06:18:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:29 smithi110 ceph-mon[34550]: pgmap v528: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.4 MiB/s wr, 731 op/s 2024-09-21T06:18:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:29 smithi107 ceph-mon[34380]: pgmap v528: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.4 MiB/s wr, 731 op/s 2024-09-21T06:18:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 410.0 s, 2317.6 tps, lat 43.082 ms stddev 111.390 2024-09-21T06:18:30.340 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:18:30.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:18:30.747 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:31.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.355+0000 7f7048a58640 1 -- 172.21.15.31:0/500186712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410a490 msgr2=0x7f704410a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.355+0000 7f7048a58640 1 --2- 172.21.15.31:0/500186712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410a490 0x7f704410a8a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f703c009ca0 tx=0x7f703c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.356+0000 7f7048a58640 1 -- 172.21.15.31:0/500186712 shutdown_connections 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.356+0000 7f7048a58640 1 --2- 172.21.15.31:0/500186712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410d210 0x7f7044116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.356+0000 7f7048a58640 1 --2- 172.21.15.31:0/500186712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704410c8c0 0x7f704410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.356+0000 7f7048a58640 1 --2- 172.21.15.31:0/500186712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410a490 0x7f704410a8a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.356+0000 7f7048a58640 1 -- 172.21.15.31:0/500186712 >> 172.21.15.31:0/500186712 conn(0x7f70440796d0 msgr2=0x7f704407baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.357+0000 7f7048a58640 1 -- 172.21.15.31:0/500186712 shutdown_connections 2024-09-21T06:18:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.357+0000 7f7048a58640 1 -- 172.21.15.31:0/500186712 wait complete. 2024-09-21T06:18:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 Processor -- start 2024-09-21T06:18:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 -- start start 2024-09-21T06:18:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 0x7f70441a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410c8c0 0x7f70441a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704410d210 0x7f70441a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f704411a160 con 0x7f704410c8c0 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7044119fe0 con 0x7f704410a490 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.358+0000 7f7048a58640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f704411a2e0 con 0x7f704410d210 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f7041d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410c8c0 0x7f70441a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 0x7f70441a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704410d210 0x7f70441a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 0x7f70441a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:40966/0 (socket says 172.21.15.31:40966) 2024-09-21T06:18:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 -- 172.21.15.31:0/310960700 learned_addr learned my addr 172.21.15.31:0/310960700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 -- 172.21.15.31:0/310960700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704410d210 msgr2=0x7f70441a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704410d210 0x7f70441a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 -- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410c8c0 msgr2=0x7f70441a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410c8c0 0x7f70441a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.359+0000 7f704259c640 1 -- 172.21.15.31:0/310960700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441a6b20 con 0x7f704410a490 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.360+0000 7f704259c640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 0x7f70441a0bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f703c000f80 tx=0x7f703c0316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.360+0000 7f703b7fe640 1 -- 172.21.15.31:0/310960700 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c003b70 con 0x7f704410a490 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.360+0000 7f703b7fe640 1 -- 172.21.15.31:0/310960700 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c0318e0 con 0x7f704410a490 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.360+0000 7f703b7fe640 1 -- 172.21.15.31:0/310960700 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c02f970 con 0x7f704410a490 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.360+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70441a6db0 con 0x7f704410a490 2024-09-21T06:18:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.360+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441ad570 con 0x7f704410a490 2024-09-21T06:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.365+0000 7f704efcc740 1 Processor -- start 2024-09-21T06:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.365+0000 7f703b7fe640 1 -- 172.21.15.31:0/310960700 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f703c02fb10 con 0x7f704410a490 2024-09-21T06:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 -- start start 2024-09-21T06:18:31.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 0x55a06607b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06607b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f703b7fe640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70300b6700 0x7f70300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06607beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a065ff1e60 con 0x55a065cab490 2024-09-21T06:18:31.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a065fe1290 con 0x55a065cb3670 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.366+0000 7f704efcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a065fe50a0 con 0x55a065cd4350 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f704259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06607b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 0x55a06607b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 0x55a06607b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49682/0 (socket says 172.21.15.31:49682) 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 -- 172.21.15.31:0/529937556 learned_addr learned my addr 172.21.15.31:0/529937556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7041d9b640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70300b6700 0x7f70300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06607beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f703b7fe640 1 -- 172.21.15.31:0/310960700 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f703c1086e0 con 0x7f704410a490 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 -- 172.21.15.31:0/529937556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 msgr2=0x55a06607beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06607beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 -- 172.21.15.31:0/529937556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 msgr2=0x55a06607b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.367+0000 7f7042d9d640 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06607b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f7042d9d640 1 -- 172.21.15.31:0/529937556 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a065d3f930 con 0x55a065cd4350 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f7041d9b640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70300b6700 0x7f70300b8c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f704407cf20 tx=0x7f702c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f704259c640 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06607b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f7042d9d640 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 0x55a06607b430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f70340137b0 tx=0x7f7034009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa9e3911c8f32fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f7038ff9640 1 -- 172.21.15.31:0/529937556 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034013d30 con 0x55a065cd4350 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f7038ff9640 1 -- 172.21.15.31:0/529937556 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7034013170 con 0x55a065cd4350 2024-09-21T06:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.368+0000 7f7038ff9640 1 -- 172.21.15.31:0/529937556 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340108c0 con 0x55a065cd4350 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 -- 172.21.15.31:0/529937556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 msgr2=0x55a06607b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 0x55a06607b430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f70340137b0 tx=0x7f7034009780 comp rx=0 tx=0).stop 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 -- 172.21.15.31:0/529937556 shutdown_connections 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06607beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06607b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 --2- 172.21.15.31:0/529937556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cd4350 0x55a06607b430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 -- 172.21.15.31:0/529937556 >> 172.21.15.31:0/529937556 conn(0x55a065cc8740 msgr2=0x55a066079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.369+0000 7f704efcc740 1 -- 172.21.15.31:0/529937556 shutdown_connections 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 -- 172.21.15.31:0/529937556 wait complete. 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 -- start start 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 Processor -- start 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066087040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a066087580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cd4350 0x55a066092f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a065e42890 con 0x55a065cb3670 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a065f7bd40 con 0x55a065cd4350 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704efcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a066089c10 con 0x55a065cab490 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.370+0000 7f704259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066087040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704efcc740 10 client.0 _ll_register_callbacks cb 0x55a065fdfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066087040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49696/0 (socket says 172.21.15.31:49696) 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 -- 172.21.15.31:0/1685995076 learned_addr learned my addr 172.21.15.31:0/1685995076 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f7041d9b640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a066087580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f7042d9d640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cd4350 0x55a066092f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 -- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cd4350 msgr2=0x55a066092f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cd4350 0x55a066092f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 -- 172.21.15.31:0/1685995076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 msgr2=0x55a066087580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a066087580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 -- 172.21.15.31:0/1685995076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a066093520 con 0x55a065cab490 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f704259c640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066087040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f703c0cab10 tx=0x7f703c0980a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b96f7b13a6379b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:31.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.371+0000 7f7029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:31.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.372+0000 7f7029ffb640 1 -- 172.21.15.31:0/1685995076 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c0cec40 con 0x55a065cab490 2024-09-21T06:18:31.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.372+0000 7f7029ffb640 1 -- 172.21.15.31:0/1685995076 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f703c1060c0 con 0x55a065cab490 2024-09-21T06:18:31.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.373+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7034013170 con 0x55a065cab490 2024-09-21T06:18:31.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.373+0000 7f7029ffb640 1 -- 172.21.15.31:0/1685995076 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f703c1062c0 con 0x55a065cab490 2024-09-21T06:18:31.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.373+0000 7f704efcc740 10 client.26788 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.373+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a066095e50 con 0x55a065cab490 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f7029ffb640 1 -- 172.21.15.31:0/1685995076 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c036070 con 0x55a065cab490 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f7029ffb640 1 -- 172.21.15.31:0/1685995076 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f703c0fe0b0 con 0x55a065cab490 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f704efcc740 10 client.26788 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f704efcc740 10 client.26788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f704efcc740 20 client.26788 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f704efcc740 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065c92cd0 0x55a066099380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f704efcc740 4 client.26788 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7034013170 con 0x55a065c92cd0 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.374+0000 7f7041d9b640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065c92cd0 0x55a066099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.375+0000 7f7029ffb640 10 client.26788 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:31.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.375+0000 7f7041d9b640 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065c92cd0 0x55a066099380 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10688164386be563 server_cookie=635bd96838cebe39 in_seq=0 out_seq=0 2024-09-21T06:18:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f7029ffb640 1 -- 172.21.15.31:0/1685995076 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7034013170 con 0x55a065c92cd0 2024-09-21T06:18:31.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f7029ffb640 10 client.26788 handle_command_reply: tid=0 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 client.26788 shutdown 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 msgr2=0x55a066087040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066087040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f703c0cab10 tx=0x7f703c0980a0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 shutdown_connections 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065c92cd0 0x55a066099380 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cd4350 0x55a066092f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a066087580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f7029ffb640 0 client.26788 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 --2- 172.21.15.31:0/1685995076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066087040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 >> 172.21.15.31:0/1685995076 conn(0x55a065cc8740 msgr2=0x55a06607e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 shutdown_connections 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.376+0000 7f704efcc740 1 -- 172.21.15.31:0/1685995076 wait complete. 2024-09-21T06:18:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.377+0000 7f704efcc740 20 client.26788 trim_cache size 0 max 16384 2024-09-21T06:18:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.402+0000 7f704efcc740 1 Processor -- start 2024-09-21T06:18:31.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.402+0000 7f704efcc740 1 -- start start 2024-09-21T06:18:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704efcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065c92cd0 0x55a0660866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704efcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066088d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704efcc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a06607eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704efcc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b3ef0 con 0x55a065cb3670 2024-09-21T06:18:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704efcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b6750 con 0x55a065c92cd0 2024-09-21T06:18:31.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704efcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b4540 con 0x55a065cab490 2024-09-21T06:18:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066088d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065c92cd0 0x55a0660866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066088d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49710/0 (socket says 172.21.15.31:49710) 2024-09-21T06:18:31.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 -- 172.21.15.31:0/185945463 learned_addr learned my addr 172.21.15.31:0/185945463 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f704259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065c92cd0 0x55a0660866d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:40996/0 (socket says 172.21.15.31:40996) 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7042d9d640 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a06607eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 -- 172.21.15.31:0/185945463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065c92cd0 msgr2=0x55a0660866d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065c92cd0 0x55a0660866d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 -- 172.21.15.31:0/185945463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 msgr2=0x55a06607eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a06607eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 -- 172.21.15.31:0/185945463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a066079860 con 0x55a065cab490 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.403+0000 7f7041d9b640 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066088d20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f704406bb30 tx=0x7f702c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c768c2fe406d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.404+0000 7f702bfff640 1 -- 172.21.15.31:0/185945463 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c02d070 con 0x55a065cab490 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.404+0000 7f702bfff640 1 -- 172.21.15.31:0/185945463 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702c003e30 con 0x55a065cab490 2024-09-21T06:18:31.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.404+0000 7f702bfff640 1 -- 172.21.15.31:0/185945463 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c01f820 con 0x55a065cab490 2024-09-21T06:18:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 -- 172.21.15.31:0/185945463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 msgr2=0x55a066088d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066088d20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f704406bb30 tx=0x7f702c014000 comp rx=0 tx=0).stop 2024-09-21T06:18:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 -- 172.21.15.31:0/185945463 shutdown_connections 2024-09-21T06:18:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cb3670 0x55a06607eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065cab490 0x55a066088d20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 --2- 172.21.15.31:0/185945463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065c92cd0 0x55a0660866d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 -- 172.21.15.31:0/185945463 >> 172.21.15.31:0/185945463 conn(0x55a065cc8740 msgr2=0x55a06607c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 -- 172.21.15.31:0/185945463 shutdown_connections 2024-09-21T06:18:31.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 -- 172.21.15.31:0/185945463 wait complete. 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.405+0000 7f704efcc740 1 -- start start 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 Processor -- start 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065c92cd0 0x55a06608f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06608ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06608bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b4c10 con 0x55a065cab490 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b6080 con 0x55a065cb3670 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f704efcc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f70441b52e0 con 0x55a065c92cd0 2024-09-21T06:18:31.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.406+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06608bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06608bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41004/0 (socket says 172.21.15.31:41004) 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 -- 172.21.15.31:0/3608570813 learned_addr learned my addr 172.21.15.31:0/3608570813 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f704efcc740 10 client.0 _ll_register_callbacks cb 0x7f703c0ca3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 -- 172.21.15.31:0/3608570813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065c92cd0 msgr2=0x55a06608f5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065c92cd0 0x55a06608f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 -- 172.21.15.31:0/3608570813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 msgr2=0x55a06608ba90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06608ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.407+0000 7f7042d9d640 1 -- 172.21.15.31:0/3608570813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06608c6c0 con 0x55a065cb3670 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.408+0000 7f7042d9d640 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06608bfd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7034009250 tx=0x7f7034009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413d506c3be1ccbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.408+0000 7f7029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.408+0000 7f7029ffb640 1 -- 172.21.15.31:0/3608570813 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703400f990 con 0x55a065cb3670 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.408+0000 7f7029ffb640 1 -- 172.21.15.31:0/3608570813 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f703400fb30 con 0x55a065cb3670 2024-09-21T06:18:31.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.408+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702c003e30 con 0x55a065cb3670 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.408+0000 7f7029ffb640 1 -- 172.21.15.31:0/3608570813 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f703400fd30 con 0x55a065cb3670 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 10 client.26738 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a0660fbeb0 con 0x55a065cb3670 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f7029ffb640 1 -- 172.21.15.31:0/3608570813 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7034003aa0 con 0x55a065cb3670 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f7029ffb640 1 -- 172.21.15.31:0/3608570813 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f70340104a0 con 0x55a065cb3670 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 10 client.26738 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 10 client.26738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 20 client.26738 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065cd4350 0x55a0660fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 4 client.26738 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.409+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f702c003e30 con 0x55a065cd4350 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.410+0000 7f704259c640 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065cd4350 0x55a0660fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.410+0000 7f704259c640 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065cd4350 0x55a0660fe2f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f914ebd334bd3745 server_cookie=5523cfd0efb36358 in_seq=0 out_seq=0 2024-09-21T06:18:31.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.410+0000 7f7029ffb640 10 client.26738 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:31.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.414+0000 7f7029ffb640 1 -- 172.21.15.31:0/3608570813 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f702c003e30 con 0x55a065cd4350 2024-09-21T06:18:31.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.415+0000 7f7029ffb640 10 client.26738 handle_command_reply: tid=0 2024-09-21T06:18:31.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.415+0000 7f704efcc740 1 client.26738 shutdown 2024-09-21T06:18:31.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.429+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 msgr2=0x55a06608bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.429+0000 7f704efcc740 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06608bfd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7034009250 tx=0x7f7034009ab0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 shutdown_connections 2024-09-21T06:18:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f704efcc740 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a065cd4350 0x55a0660fe2f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f704efcc740 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a065cb3670 0x55a06608bfd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f704efcc740 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a065cab490 0x55a06608ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f7029ffb640 0 client.26738 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f704efcc740 1 --2- 172.21.15.31:0/3608570813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a065c92cd0 0x55a06608f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.433+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 >> 172.21.15.31:0/3608570813 conn(0x55a065cc8740 msgr2=0x55a066088a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.434+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 shutdown_connections 2024-09-21T06:18:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.434+0000 7f704efcc740 1 -- 172.21.15.31:0/3608570813 wait complete. 2024-09-21T06:18:31.437 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:18:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:18:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "54b7927b-7db3-4bc0-bddf-a85c18744a1c", 2024-09-21T06:18:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:18:31.437 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:18:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.434+0000 7f704efcc740 20 client.26738 trim_cache size 0 max 16384 2024-09-21T06:18:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70300b6700 msgr2=0x7f70300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70300b6700 0x7f70300b8c10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f704407cf20 tx=0x7f702c005f90 comp rx=0 tx=0).stop 2024-09-21T06:18:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 msgr2=0x7f70441a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 0x7f70441a0bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f703c000f80 tx=0x7f703c0316f0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 shutdown_connections 2024-09-21T06:18:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704410d210 0x7f70441a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f70300b6700 0x7f70300b8c10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704410c8c0 0x7f70441a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 --2- 172.21.15.31:0/310960700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704410a490 0x7f70441a0bb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.437+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 >> 172.21.15.31:0/310960700 conn(0x7f70440796d0 msgr2=0x7f704410f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.438+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 shutdown_connections 2024-09-21T06:18:31.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:31.438+0000 7f7048a58640 1 -- 172.21.15.31:0/310960700 wait complete. 2024-09-21T06:18:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:31 smithi110 ceph-mon[34550]: pgmap v529: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-09-21T06:18:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:31 smithi107 ceph-mon[34380]: pgmap v529: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-09-21T06:18:31.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:31 smithi031 ceph-mon[31237]: pgmap v529: 97 pgs: 97 active+clean; 4.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-09-21T06:18:32.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:18:32.616 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:18:32.771 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:32 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:18:32.771 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:32 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:32.771 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:32 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:18:32.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:32 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:18:32.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:32 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:32.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:32 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:18:32.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:32 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:18:32.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:32 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:32.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:32 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:18:33.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 -- 172.21.15.31:0/3844291922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc350111cd0 msgr2=0x7fc35010d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 --2- 172.21.15.31:0/3844291922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc350111cd0 0x7fc35010d6b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009860 tx=0x7fc34c02a360 comp rx=0 tx=0).stop 2024-09-21T06:18:33.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 -- 172.21.15.31:0/3844291922 shutdown_connections 2024-09-21T06:18:33.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 --2- 172.21.15.31:0/3844291922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc35011cba0 0x7fc35011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 --2- 172.21.15.31:0/3844291922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 0x7fc35010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 --2- 172.21.15.31:0/3844291922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc350111cd0 0x7fc35010d6b0 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009860 tx=0x7fc34c02a360 comp rx=0 tx=0).stop 2024-09-21T06:18:33.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.145+0000 7fc356312640 1 -- 172.21.15.31:0/3844291922 >> 172.21.15.31:0/3844291922 conn(0x7fc35006e9f0 msgr2=0x7fc35006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:33.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.146+0000 7fc356312640 1 -- 172.21.15.31:0/3844291922 shutdown_connections 2024-09-21T06:18:33.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.146+0000 7fc356312640 1 -- 172.21.15.31:0/3844291922 wait complete. 2024-09-21T06:18:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.148+0000 7fc356312640 1 Processor -- start 2024-09-21T06:18:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.148+0000 7fc356312640 1 -- start start 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.148+0000 7fc356312640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 0x7fc3501a9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc356312640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc35011cba0 0x7fc3501a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc356312640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 0x7fc3501aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc356312640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc350122790 con 0x7fc3501aeac0 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc356312640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc350122610 con 0x7fc35011cba0 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc356312640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc350122910 con 0x7fc35010dbf0 2024-09-21T06:18:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc355b11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 0x7fc3501aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc355b11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 0x7fc3501aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42368/0 (socket says 172.21.15.31:42368) 2024-09-21T06:18:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc355b11640 1 -- 172.21.15.31:0/97358943 learned_addr learned my addr 172.21.15.31:0/97358943 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.149+0000 7fc355310640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 0x7fc3501a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc354b0f640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc35011cba0 0x7fc3501a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc354b0f640 1 -- 172.21.15.31:0/97358943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 msgr2=0x7fc3501a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc354b0f640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 0x7fc3501a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc354b0f640 1 -- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 msgr2=0x7fc3501aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc354b0f640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 0x7fc3501aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc354b0f640 1 -- 172.21.15.31:0/97358943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3501af6a0 con 0x7fc35011cba0 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc355b11640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 0x7fc3501aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.150+0000 7fc355310640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 0x7fc3501a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.151+0000 7fc354b0f640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc35011cba0 0x7fc3501a98e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc35006c910 tx=0x7fc34000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.151+0000 7fc3467fc640 1 -- 172.21.15.31:0/97358943 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34000cc90 con 0x7fc35011cba0 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.151+0000 7fc3467fc640 1 -- 172.21.15.31:0/97358943 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc34000ee70 con 0x7fc35011cba0 2024-09-21T06:18:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.151+0000 7fc3467fc640 1 -- 172.21.15.31:0/97358943 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340018660 con 0x7fc35011cba0 2024-09-21T06:18:33.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.158+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3501b5c40 con 0x7fc35011cba0 2024-09-21T06:18:33.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.158+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3501b5fe0 con 0x7fc35011cba0 2024-09-21T06:18:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.162+0000 7fc35dae8740 1 Processor -- start 2024-09-21T06:18:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.162+0000 7fc3467fc640 1 -- 172.21.15.31:0/97358943 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc34001f020 con 0x7fc35011cba0 2024-09-21T06:18:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.162+0000 7fc35dae8740 1 -- start start 2024-09-21T06:18:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09ba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc3467fc640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc33c0b6800 0x7fc33c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc355b11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc355b11640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09ba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42372/0 (socket says 172.21.15.31:42372) 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc354b0f640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc33c0b6800 0x7fc33c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc355310640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc355310640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09bad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41030/0 (socket says 172.21.15.31:41030) 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc355b11640 1 -- 172.21.15.31:0/272642602 learned_addr learned my addr 172.21.15.31:0/272642602 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5614b0930e50 con 0x5614b0613070 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.163+0000 7fc3467fc640 1 -- 172.21.15.31:0/97358943 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fc3400e76e0 con 0x7fc35011cba0 2024-09-21T06:18:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.164+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5614b0920280 con 0x5614b05ea1b0 2024-09-21T06:18:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.164+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5614b0924090 con 0x5614b05f2390 2024-09-21T06:18:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.164+0000 7fc354b0f640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc33c0b6800 0x7fc33c0b8d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc3400e4ef0 tx=0x7fc3400760e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.164+0000 7fc355b11640 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355b11640 1 -- 172.21.15.31:0/272642602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 msgr2=0x5614b09bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355b11640 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355b11640 1 -- 172.21.15.31:0/272642602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 msgr2=0x5614b09ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355b11640 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09ba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355b11640 1 -- 172.21.15.31:0/272642602 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b067e650 con 0x5614b05f2390 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355310640 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc355b11640 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09bb290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc348010d20 tx=0x7fc348004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d4dbc3b2009fee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc33b7fe640 1 -- 172.21.15.31:0/272642602 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348017d90 con 0x5614b05f2390 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc33b7fe640 1 -- 172.21.15.31:0/272642602 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc34801e070 con 0x5614b05f2390 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.165+0000 7fc33b7fe640 1 -- 172.21.15.31:0/272642602 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348007370 con 0x5614b05f2390 2024-09-21T06:18:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.166+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 msgr2=0x5614b09bb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.166+0000 7fc35dae8740 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09bb290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc348010d20 tx=0x7fc348004a40 comp rx=0 tx=0).stop 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.166+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 shutdown_connections 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.166+0000 7fc35dae8740 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09bb290 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.166+0000 7fc35dae8740 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.166+0000 7fc35dae8740 1 --2- 172.21.15.31:0/272642602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.167+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 >> 172.21.15.31:0/272642602 conn(0x5614b0607460 msgr2=0x5614b09b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.167+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 shutdown_connections 2024-09-21T06:18:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.167+0000 7fc35dae8740 1 -- 172.21.15.31:0/272642602 wait complete. 2024-09-21T06:18:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.167+0000 7fc35dae8740 1 -- start start 2024-09-21T06:18:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.167+0000 7fc35dae8740 1 Processor -- start 2024-09-21T06:18:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc355b11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc35dae8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5614b0781700 con 0x5614b0613070 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc35dae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5614b08bb110 con 0x5614b05ea1b0 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc35dae8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5614b09c8f90 con 0x5614b05f2390 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc355b11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41032/0 (socket says 172.21.15.31:41032) 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.168+0000 7fc355b11640 1 -- 172.21.15.31:0/3424528358 learned_addr learned my addr 172.21.15.31:0/3424528358 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.169+0000 7fc35dae8740 10 client.0 _ll_register_callbacks cb 0x5614b08d4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355310640 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355b11640 1 -- 172.21.15.31:0/3424528358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 msgr2=0x5614b09c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355b11640 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355b11640 1 -- 172.21.15.31:0/3424528358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 msgr2=0x5614b09d2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355b11640 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355b11640 1 -- 172.21.15.31:0/3424528358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b08e8c40 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc355b11640 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09c5f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc348002230 tx=0x7fc348009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc9f8f0aac6c1ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.173+0000 7fc338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.174+0000 7fc338ff9640 1 -- 172.21.15.31:0/3424528358 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3480109e0 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.174+0000 7fc338ff9640 1 -- 172.21.15.31:0/3424528358 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fc348019610 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.174+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc34801e070 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.174+0000 7fc338ff9640 1 -- 172.21.15.31:0/3424528358 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc348032050 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.174+0000 7fc35dae8740 10 client.26750 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.174+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5614b09d4d00 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc338ff9640 1 -- 172.21.15.31:0/3424528358 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc34801e070 con 0x5614b05ea1b0 2024-09-21T06:18:33.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc338ff9640 1 -- 172.21.15.31:0/3424528358 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc348018040 con 0x5614b05ea1b0 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc35dae8740 10 client.26750 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc35dae8740 10 client.26750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc35dae8740 20 client.26750 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc35dae8740 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05d19f0 0x5614b09d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc35dae8740 4 client.26750 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.175+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b0932da0 con 0x5614b05d19f0 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.176+0000 7fc355310640 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05d19f0 0x5614b09d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.176+0000 7fc338ff9640 10 client.26750 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:33.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.176+0000 7fc355310640 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05d19f0 0x5614b09d80c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4634c3e34bb6fea server_cookie=167e7652b71b05b8 in_seq=0 out_seq=0 2024-09-21T06:18:33.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc338ff9640 1 -- 172.21.15.31:0/3424528358 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5614b0932da0 con 0x5614b05d19f0 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc338ff9640 10 client.26750 handle_command_reply: tid=0 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 client.26750 shutdown 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 msgr2=0x5614b09c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09c5f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc348002230 tx=0x7fc348009490 comp rx=0 tx=0).stop 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 shutdown_connections 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05d19f0 0x5614b09d80c0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05f2390 0x5614b09c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 --2- 172.21.15.31:0/3424528358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09c5f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 >> 172.21.15.31:0/3424528358 conn(0x5614b0607460 msgr2=0x5614b09bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc338ff9640 0 client.26750 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 shutdown_connections 2024-09-21T06:18:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 1 -- 172.21.15.31:0/3424528358 wait complete. 2024-09-21T06:18:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.177+0000 7fc35dae8740 20 client.26750 trim_cache size 0 max 16384 2024-09-21T06:18:33.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.202+0000 7fc35dae8740 1 Processor -- start 2024-09-21T06:18:33.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.202+0000 7fc35dae8740 1 -- start start 2024-09-21T06:18:33.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 0x5614b09bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05f2390 0x5614b09c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc35dae8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501bca90 con 0x5614b0613070 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc35dae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501bf2f0 con 0x5614b05f2390 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc35dae8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501bd0e0 con 0x5614b05ea1b0 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc355b11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 0x5614b09bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc355b11640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 0x5614b09bbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49746/0 (socket says 172.21.15.31:49746) 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc355b11640 1 -- 172.21.15.31:0/4248878563 learned_addr learned my addr 172.21.15.31:0/4248878563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc355310640 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05f2390 0x5614b09c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.203+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc355b11640 1 -- 172.21.15.31:0/4248878563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05f2390 msgr2=0x5614b09c3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05f2390 0x5614b09c3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc355b11640 1 -- 172.21.15.31:0/4248878563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 msgr2=0x5614b09c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc355b11640 1 -- 172.21.15.31:0/4248878563 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b09c8530 con 0x5614b05ea1b0 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 0x5614b09bbf50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc348026040 tx=0x7fc348010620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f596d5052df757 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc33affd640 1 -- 172.21.15.31:0/4248878563 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348019db0 con 0x5614b05ea1b0 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc33affd640 1 -- 172.21.15.31:0/4248878563 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc34801e040 con 0x5614b05ea1b0 2024-09-21T06:18:33.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.204+0000 7fc33affd640 1 -- 172.21.15.31:0/4248878563 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348023400 con 0x5614b05ea1b0 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248878563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 msgr2=0x5614b09bbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 0x5614b09bbf50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc348026040 tx=0x7fc348010620 comp rx=0 tx=0).stop 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248878563 shutdown_connections 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b0613070 0x5614b09c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05f2390 0x5614b09c3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248878563 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b05ea1b0 0x5614b09bbf50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.205+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248878563 >> 172.21.15.31:0/4248878563 conn(0x5614b0607460 msgr2=0x5614b09bb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.206+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248878563 shutdown_connections 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.206+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248878563 wait complete. 2024-09-21T06:18:33.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.206+0000 7fc35dae8740 1 -- start start 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.206+0000 7fc35dae8740 1 Processor -- start 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b05f2390 0x5614b09dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b0613070 0x5614b0a3a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501bd7b0 con 0x5614b05f2390 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501bec20 con 0x5614b05ea1b0 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3501bde80 con 0x5614b0613070 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09daa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41040/0 (socket says 172.21.15.31:41040) 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355310640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b05f2390 0x5614b09dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 -- 172.21.15.31:0/4248213256 learned_addr learned my addr 172.21.15.31:0/4248213256 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc35dae8740 10 client.0 _ll_register_callbacks cb 0x7fc348010440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b0613070 0x5614b0a3a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 -- 172.21.15.31:0/4248213256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b0613070 msgr2=0x5614b0a3a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b0613070 0x5614b0a3a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 -- 172.21.15.31:0/4248213256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b05f2390 msgr2=0x5614b09dafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b05f2390 0x5614b09dafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.207+0000 7fc355b11640 1 -- 172.21.15.31:0/4248213256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b0a3ae80 con 0x5614b05ea1b0 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.208+0000 7fc355b11640 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09daa00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc348000f80 tx=0x7fc3480049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2d4161598c866f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:33.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.208+0000 7fc338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:33.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.208+0000 7fc338ff9640 1 -- 172.21.15.31:0/4248213256 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348046060 con 0x5614b05ea1b0 2024-09-21T06:18:33.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.209+0000 7fc338ff9640 1 -- 172.21.15.31:0/4248213256 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fc348019b10 con 0x5614b05ea1b0 2024-09-21T06:18:33.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.209+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc34801e040 con 0x5614b05ea1b0 2024-09-21T06:18:33.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.210+0000 7fc338ff9640 1 -- 172.21.15.31:0/4248213256 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc348019d10 con 0x5614b05ea1b0 2024-09-21T06:18:33.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.211+0000 7fc35dae8740 10 client.26756 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:33.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.211+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5614b0a421b0 con 0x5614b05ea1b0 2024-09-21T06:18:33.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.212+0000 7fc338ff9640 1 -- 172.21.15.31:0/4248213256 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc348027030 con 0x5614b05ea1b0 2024-09-21T06:18:33.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.212+0000 7fc338ff9640 1 -- 172.21.15.31:0/4248213256 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc348023930 con 0x5614b05ea1b0 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.212+0000 7fc35dae8740 10 client.26756 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.212+0000 7fc35dae8740 10 client.26756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.212+0000 7fc35dae8740 20 client.26756 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.212+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05a6da0 0x5614b0a448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.213+0000 7fc35dae8740 4 client.26756 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.213+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc34801e040 con 0x5614b05a6da0 2024-09-21T06:18:33.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.213+0000 7fc355310640 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05a6da0 0x5614b0a448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:33.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.213+0000 7fc338ff9640 10 client.26756 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:33.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.213+0000 7fc355310640 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05a6da0 0x5614b0a448f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2561450194a99d6 server_cookie=e6068f03d917df98 in_seq=0 out_seq=0 2024-09-21T06:18:33.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc338ff9640 1 -- 172.21.15.31:0/4248213256 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x7fc34801e040 con 0x5614b05a6da0 2024-09-21T06:18:33.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc338ff9640 10 client.26756 handle_command_reply: tid=0 2024-09-21T06:18:33.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc35dae8740 1 client.26756 shutdown 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 msgr2=0x5614b09daa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09daa00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc348000f80 tx=0x7fc3480049c0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 shutdown_connections 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614b05a6da0 0x5614b0a448f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614b0613070 0x5614b0a3a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.214+0000 7fc338ff9640 0 client.26756 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.215+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614b05f2390 0x5614b09dafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.215+0000 7fc35dae8740 1 --2- 172.21.15.31:0/4248213256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614b05ea1b0 0x5614b09daa00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.215+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 >> 172.21.15.31:0/4248213256 conn(0x5614b0607460 msgr2=0x5614b09bb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:33.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.215+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 shutdown_connections 2024-09-21T06:18:33.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.215+0000 7fc35dae8740 1 -- 172.21.15.31:0/4248213256 wait complete. 2024-09-21T06:18:33.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.215+0000 7fc35dae8740 20 client.26756 trim_cache size 0 max 16384 2024-09-21T06:18:33.218 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:18:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:18:33.218 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "54b7927b-7db3-4bc0-bddf-a85c18744a1c", 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:18:33.220 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.221 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.222 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.223 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.224 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.225 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:18:33.226 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.228 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.229 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.230 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:18:33.231 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc33c0b6800 msgr2=0x7fc33c0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc33c0b6800 0x7fc33c0b8d10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc3400e4ef0 tx=0x7fc3400760e0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc35011cba0 msgr2=0x7fc3501a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc35011cba0 0x7fc3501a98e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc35006c910 tx=0x7fc34000ec20 comp rx=0 tx=0).stop 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 shutdown_connections 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc33c0b6800 0x7fc33c0b8d10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3501aeac0 0x7fc3501aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc35011cba0 0x7fc3501a98e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 --2- 172.21.15.31:0/97358943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc35010dbf0 0x7fc3501a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 >> 172.21.15.31:0/97358943 conn(0x7fc35006e9f0 msgr2=0x7fc350110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 shutdown_connections 2024-09-21T06:18:33.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:18:33.216+0000 7fc356312640 1 -- 172.21.15.31:0/97358943 wait complete. 2024-09-21T06:18:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:33 smithi110 ceph-mon[34550]: pgmap v530: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 2.13k op/s 2024-09-21T06:18:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:33 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:33 smithi107 ceph-mon[34380]: pgmap v530: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 2.13k op/s 2024-09-21T06:18:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:33 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:33.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:33 smithi031 ceph-mon[31237]: pgmap v530: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 MiB/s wr, 2.13k op/s 2024-09-21T06:18:33.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:33 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:18:34.170 INFO:tasks.cephfs.filesystem:scrub status for tag:54b7927b-7db3-4bc0-bddf-a85c18744a1c - {'path': '/', 'tag': '54b7927b-7db3-4bc0-bddf-a85c18744a1c', 'options': 'recursive,force'} 2024-09-21T06:18:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 415.0 s, 2153.8 tps, lat 46.700 ms stddev 51.205 2024-09-21T06:18:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:35 smithi110 ceph-mon[34550]: pgmap v531: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 2.04k op/s 2024-09-21T06:18:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:35 smithi107 ceph-mon[34380]: pgmap v531: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 2.04k op/s 2024-09-21T06:18:35.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:35 smithi031 ceph-mon[31237]: pgmap v531: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 MiB/s wr, 2.04k op/s 2024-09-21T06:18:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:37 smithi110 ceph-mon[34550]: pgmap v532: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 2.03k op/s 2024-09-21T06:18:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:37 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:18:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:37 smithi107 ceph-mon[34380]: pgmap v532: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 2.03k op/s 2024-09-21T06:18:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:37 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:18:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:37 smithi031 ceph-mon[31237]: pgmap v532: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 28 MiB/s wr, 2.03k op/s 2024-09-21T06:18:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:37 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:18:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:39 smithi107 ceph-mon[34380]: pgmap v533: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 25 MiB/s wr, 1.85k op/s 2024-09-21T06:18:39.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:39 smithi031 ceph-mon[31237]: pgmap v533: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 25 MiB/s wr, 1.85k op/s 2024-09-21T06:18:40.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:39 smithi110 ceph-mon[34550]: pgmap v533: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 25 MiB/s wr, 1.85k op/s 2024-09-21T06:18:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 420.0 s, 2693.0 tps, lat 36.965 ms stddev 44.069 2024-09-21T06:18:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.044+0000 7f1f3484a640 1 -- 172.21.15.31:0/3525057354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109d50 msgr2=0x7f1f30110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.044+0000 7f1f3484a640 1 --2- 172.21.15.31:0/3525057354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109d50 0x7f1f30110580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009ca0 tx=0x7f1f2802a8e0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.048+0000 7f1f3484a640 1 -- 172.21.15.31:0/3525057354 shutdown_connections 2024-09-21T06:18:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.048+0000 7f1f3484a640 1 --2- 172.21.15.31:0/3525057354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109d50 0x7f1f30110580 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.048+0000 7f1f3484a640 1 --2- 172.21.15.31:0/3525057354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30109400 0x7f1f30109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.048+0000 7f1f3484a640 1 --2- 172.21.15.31:0/3525057354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30103ad0 0x7f1f30103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.048+0000 7f1f3484a640 1 -- 172.21.15.31:0/3525057354 >> 172.21.15.31:0/3525057354 conn(0x7f1f300ff510 msgr2=0x7f1f30101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.051+0000 7f1f3484a640 1 -- 172.21.15.31:0/3525057354 shutdown_connections 2024-09-21T06:18:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.052+0000 7f1f3484a640 1 -- 172.21.15.31:0/3525057354 wait complete. 2024-09-21T06:18:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.053+0000 7f1f3484a640 1 Processor -- start 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 -- start start 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 0x7f1f301a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109400 0x7f1f301a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30109d50 0x7f1f301a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f301118c0 con 0x7f1f30109400 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30111740 con 0x7f1f30103ad0 2024-09-21T06:18:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f3484a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30111a40 con 0x7f1f30109d50 2024-09-21T06:18:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f2e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 0x7f1f301a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f2e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 0x7f1f301a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:45318/0 (socket says 172.21.15.31:45318) 2024-09-21T06:18:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.054+0000 7f1f2e59c640 1 -- 172.21.15.31:0/984246164 learned_addr learned my addr 172.21.15.31:0/984246164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.055+0000 7f1f2e59c640 1 -- 172.21.15.31:0/984246164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30109d50 msgr2=0x7f1f301a6380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:18:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.055+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30109d50 0x7f1f301a6380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.055+0000 7f1f2e59c640 1 -- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109400 msgr2=0x7f1f301a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.055+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109400 0x7f1f301a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.055+0000 7f1f2e59c640 1 -- 172.21.15.31:0/984246164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f301a6a80 con 0x7f1f30103ad0 2024-09-21T06:18:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.055+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 0x7f1f301a0b50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000b4a0 tx=0x7f1f2000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.060+0000 7f1f277fe640 1 -- 172.21.15.31:0/984246164 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20014070 con 0x7f1f30103ad0 2024-09-21T06:18:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.060+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f301a6d70 con 0x7f1f30103ad0 2024-09-21T06:18:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.060+0000 7f1f277fe640 1 -- 172.21.15.31:0/984246164 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f20010ac0 con 0x7f1f30103ad0 2024-09-21T06:18:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.060+0000 7f1f277fe640 1 -- 172.21.15.31:0/984246164 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20017b70 con 0x7f1f30103ad0 2024-09-21T06:18:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.060+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f301ad500 con 0x7f1f30103ad0 2024-09-21T06:18:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.067+0000 7f1f3b88f740 1 Processor -- start 2024-09-21T06:18:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f277fe640 1 -- 172.21.15.31:0/984246164 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1f20017070 con 0x7f1f30103ad0 2024-09-21T06:18:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 -- start start 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55639e0ca6f0 con 0x55639ddb01b0 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55639e0b9b20 con 0x55639dd8f4d0 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55639e0bd930 con 0x55639dd872f0 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.068+0000 7f1f2e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52658/0 (socket says 172.21.15.31:52658) 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:45732/0 (socket says 172.21.15.31:45732) 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2ed9d640 1 -- 172.21.15.31:0/1536089502 learned_addr learned my addr 172.21.15.31:0/1536089502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1536089502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 msgr2=0x55639e156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1536089502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 msgr2=0x55639e1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e1559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1536089502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55639e07f530 con 0x55639dd872f0 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.069+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:18:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.070+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e155ef0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f200e3e10 tx=0x7f1f200ee750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1453583a88cbb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.071+0000 7f1f277fe640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f1c0b6890 0x7f1f1c0b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.071+0000 7f1f277fe640 1 -- 172.21.15.31:0/984246164 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f1f200e6740 con 0x7f1f30103ad0 2024-09-21T06:18:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.071+0000 7f1f24ff9640 1 -- 172.21.15.31:0/1536089502 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200a8b90 con 0x55639dd872f0 2024-09-21T06:18:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.071+0000 7f1f24ff9640 1 -- 172.21.15.31:0/1536089502 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f200f3070 con 0x55639dd872f0 2024-09-21T06:18:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.071+0000 7f1f24ff9640 1 -- 172.21.15.31:0/1536089502 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200abe90 con 0x55639dd872f0 2024-09-21T06:18:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.072+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f1c0b6890 0x7f1f1c0b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1536089502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 msgr2=0x55639e155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e155ef0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1f200e3e10 tx=0x7f1f200ee750 comp rx=0 tx=0).stop 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f1c0b6890 0x7f1f1c0b8da0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009a60 tx=0x7f1f28033000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1536089502 shutdown_connections 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e155ef0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1536089502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1536089502 >> 172.21.15.31:0/1536089502 conn(0x55639dda4300 msgr2=0x55639e1538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1536089502 shutdown_connections 2024-09-21T06:18:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1536089502 wait complete. 2024-09-21T06:18:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 -- start start 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.073+0000 7f1f3b88f740 1 Processor -- start 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e1610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e17e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55639dff1ab0 con 0x55639ddb01b0 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55639e05f730 con 0x55639dd8f4d0 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55639e175140 con 0x55639dd872f0 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f3b88f740 10 client.0 _ll_register_callbacks cb 0x55639e0b36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e1610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e1610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:45748/0 (socket says 172.21.15.31:45748) 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1171422707 learned_addr learned my addr 172.21.15.31:0/1171422707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e17e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 msgr2=0x55639e1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e1615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1171422707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 msgr2=0x55639e17e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e17e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.074+0000 7f1f2e59c640 1 -- 172.21.15.31:0/1171422707 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55639de1b790 con 0x55639dd872f0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.075+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.075+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e1610b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000bfd0 tx=0x7f1f200eca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ac17bb27e79ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.075+0000 7f1f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.075+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1171422707 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2001e030 con 0x55639dd872f0 2024-09-21T06:18:41.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.075+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1171422707 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f1f20017360 con 0x55639dd872f0 2024-09-21T06:18:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.075+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f200f3070 con 0x55639dd872f0 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1171422707 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1f200ac080 con 0x55639dd872f0 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f3b88f740 10 client.26812 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55639de94620 con 0x55639dd872f0 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1171422707 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f200f3070 con 0x55639dd872f0 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1171422707 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1f200a87b0 con 0x55639dd872f0 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f3b88f740 10 client.26812 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.076+0000 7f1f3b88f740 10 client.26812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.077+0000 7f1f3b88f740 20 client.26812 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.077+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd6ed80 0x55639e1840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.077+0000 7f1f3b88f740 4 client.26812 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.077+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55639e0cc640 con 0x55639dd6ed80 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.077+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd6ed80 0x55639e1840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.078+0000 7f1f15ffb640 10 client.26812 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.078+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd6ed80 0x55639e1840d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae669ce8ebf7566f server_cookie=30e749e6fdad473c in_seq=0 out_seq=0 2024-09-21T06:18:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1171422707 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55639e0cc640 con 0x55639dd6ed80 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f15ffb640 10 client.26812 handle_command_reply: tid=0 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 client.26812 shutdown 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 msgr2=0x55639e1610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e1610b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000bfd0 tx=0x7f1f200eca20 comp rx=0 tx=0).stop 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 shutdown_connections 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd6ed80 0x55639e1840d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f15ffb640 0 client.26812 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639ddb01b0 0x55639e17e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd8f4d0 0x55639e1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1171422707 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd872f0 0x55639e1610b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 >> 172.21.15.31:0/1171422707 conn(0x55639dda4300 msgr2=0x55639e1569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 shutdown_connections 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1171422707 wait complete. 2024-09-21T06:18:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.079+0000 7f1f3b88f740 20 client.26812 trim_cache size 0 max 16384 2024-09-21T06:18:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.102+0000 7f1f3b88f740 1 Processor -- start 2024-09-21T06:18:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.102+0000 7f1f3b88f740 1 -- start start 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.102+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd872f0 0x55639e1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.102+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd8f4d0 0x55639e162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.102+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639ddb01b0 0x55639e17e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1000c7b0 con 0x55639dd8f4d0 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1000f010 con 0x55639dd872f0 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1000ce00 con 0x55639ddb01b0 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f27fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd8f4d0 0x55639e162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f27fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd8f4d0 0x55639e162ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52668/0 (socket says 172.21.15.31:52668) 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f27fff640 1 -- 172.21.15.31:0/3836348022 learned_addr learned my addr 172.21.15.31:0/3836348022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f2ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639ddb01b0 0x55639e17e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f2ed9d640 1 -- 172.21.15.31:0/3836348022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd872f0 msgr2=0x55639e1573a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd872f0 0x55639e1573a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f2ed9d640 1 -- 172.21.15.31:0/3836348022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd8f4d0 msgr2=0x55639e162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd8f4d0 0x55639e162ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.103+0000 7f1f2ed9d640 1 -- 172.21.15.31:0/3836348022 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55639e15f590 con 0x55639ddb01b0 2024-09-21T06:18:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.104+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639ddb01b0 0x55639e17e430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009bd0 tx=0x7f1f2802d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ecf52ae7f3e99ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.104+0000 7f1f17fff640 1 -- 172.21.15.31:0/3836348022 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28066070 con 0x55639ddb01b0 2024-09-21T06:18:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.104+0000 7f1f17fff640 1 -- 172.21.15.31:0/3836348022 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f28036040 con 0x55639ddb01b0 2024-09-21T06:18:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.104+0000 7f1f17fff640 1 -- 172.21.15.31:0/3836348022 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28030e10 con 0x55639ddb01b0 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 -- 172.21.15.31:0/3836348022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639ddb01b0 msgr2=0x55639e17e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639ddb01b0 0x55639e17e430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009bd0 tx=0x7f1f2802d000 comp rx=0 tx=0).stop 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 -- 172.21.15.31:0/3836348022 shutdown_connections 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639ddb01b0 0x55639e17e430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd8f4d0 0x55639e162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/3836348022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639dd872f0 0x55639e1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 -- 172.21.15.31:0/3836348022 >> 172.21.15.31:0/3836348022 conn(0x55639dda4300 msgr2=0x55639e1625d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 -- 172.21.15.31:0/3836348022 shutdown_connections 2024-09-21T06:18:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.105+0000 7f1f3b88f740 1 -- 172.21.15.31:0/3836348022 wait complete. 2024-09-21T06:18:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.106+0000 7f1f3b88f740 1 -- start start 2024-09-21T06:18:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.106+0000 7f1f3b88f740 1 Processor -- start 2024-09-21T06:18:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.106+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd872f0 0x55639e186860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 0x55639e186da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f3b88f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639ddb01b0 0x55639e1ec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1000d4d0 con 0x55639dd872f0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1000e940 con 0x55639ddb01b0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f3b88f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1000dba0 con 0x55639dd8f4d0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f27fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 0x55639e186da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f27fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 0x55639e186da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:45754/0 (socket says 172.21.15.31:45754) 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f27fff640 1 -- 172.21.15.31:0/1989323986 learned_addr learned my addr 172.21.15.31:0/1989323986 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd872f0 0x55639e186860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f3b88f740 10 client.0 _ll_register_callbacks cb 0x7f1f28009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f2ed9d640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639ddb01b0 0x55639e1ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f27fff640 1 -- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639ddb01b0 msgr2=0x55639e1ec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.107+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639ddb01b0 0x55639e1ec850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f27fff640 1 -- 172.21.15.31:0/1989323986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd872f0 msgr2=0x55639e186860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd872f0 0x55639e186860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f27fff640 1 -- 172.21.15.31:0/1989323986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55639e1ecdc0 con 0x55639dd8f4d0 2024-09-21T06:18:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f27fff640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 0x55639e186da0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f3006c8b0 tx=0x7f1f18009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8519dda5b5e81669 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:18:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1989323986 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1801c7d0 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1989323986 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f1f1801c970 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.108+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f28036040 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.109+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1989323986 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1f1801cb70 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.109+0000 7f1f3b88f740 10 client.26818 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.109+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55639e1ee070 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1989323986 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f28036040 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1989323986 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1f18024400 con 0x55639dd8f4d0 2024-09-21T06:18:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f3b88f740 10 client.26818 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f3b88f740 10 client.26818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f3b88f740 20 client.26818 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd441c0 0x55639e1f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f3b88f740 4 client.26818 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55639e0c80f0 con 0x55639dd441c0 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.110+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd441c0 0x55639e1f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.111+0000 7f1f2e59c640 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd441c0 0x55639e1f0890 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6dd15abfe290b1 server_cookie=abe3db9749410cb in_seq=0 out_seq=0 2024-09-21T06:18:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.111+0000 7f1f15ffb640 10 client.26818 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f15ffb640 1 -- 172.21.15.31:0/1989323986 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55639e0c80f0 con 0x55639dd441c0 2024-09-21T06:18:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f15ffb640 10 client.26818 handle_command_reply: tid=0 2024-09-21T06:18:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 client.26818 shutdown 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 msgr2=0x55639e186da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 0x55639e186da0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f3006c8b0 tx=0x7f1f18009d30 comp rx=0 tx=0).stop 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 shutdown_connections 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55639dd441c0 0x55639e1f0890 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55639ddb01b0 0x55639e1ec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55639dd8f4d0 0x55639e186da0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 --2- 172.21.15.31:0/1989323986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55639dd872f0 0x55639e186860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 >> 172.21.15.31:0/1989323986 conn(0x55639dda4300 msgr2=0x55639e157070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.136+0000 7f1f15ffb640 0 client.26818 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.137+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 shutdown_connections 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.137+0000 7f1f3b88f740 1 -- 172.21.15.31:0/1989323986 wait complete. 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.137+0000 7f1f3b88f740 20 client.26818 trim_cache size 0 max 16384 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:18:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:18:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f1c0b6890 msgr2=0x7f1f1c0b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f1c0b6890 0x7f1f1c0b8da0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009a60 tx=0x7f1f28033000 comp rx=0 tx=0).stop 2024-09-21T06:18:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 msgr2=0x7f1f301a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 0x7f1f301a0b50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000b4a0 tx=0x7f1f2000b960 comp rx=0 tx=0).stop 2024-09-21T06:18:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 shutdown_connections 2024-09-21T06:18:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f30109d50 0x7f1f301a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1f1c0b6890 0x7f1f1c0b8da0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f30109400 0x7f1f301a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 --2- 172.21.15.31:0/984246164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f30103ad0 0x7f1f301a0b50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 >> 172.21.15.31:0/984246164 conn(0x7f1f300ff510 msgr2=0x7f1f30101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 shutdown_connections 2024-09-21T06:18:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:41.138+0000 7f1f3484a640 1 -- 172.21.15.31:0/984246164 wait complete. 2024-09-21T06:18:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:41 smithi107 ceph-mon[34380]: pgmap v534: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 26 MiB/s wr, 1.88k op/s 2024-09-21T06:18:41.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:41 smithi031 ceph-mon[31237]: pgmap v534: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 26 MiB/s wr, 1.88k op/s 2024-09-21T06:18:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:41 smithi110 ceph-mon[34550]: pgmap v534: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 26 MiB/s wr, 1.88k op/s 2024-09-21T06:18:43.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:43 smithi031 ceph-mon[31237]: pgmap v535: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 25 MiB/s wr, 1.89k op/s 2024-09-21T06:18:44.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:43 smithi110 ceph-mon[34550]: pgmap v535: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 25 MiB/s wr, 1.89k op/s 2024-09-21T06:18:44.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:43 smithi107 ceph-mon[34380]: pgmap v535: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 25 MiB/s wr, 1.89k op/s 2024-09-21T06:18:44.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:44 smithi031 ceph-mon[31237]: pgmap v536: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.1 MiB/s wr, 633 op/s 2024-09-21T06:18:45.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:44 smithi110 ceph-mon[34550]: pgmap v536: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.1 MiB/s wr, 633 op/s 2024-09-21T06:18:45.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:44 smithi107 ceph-mon[34380]: pgmap v536: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.1 MiB/s wr, 633 op/s 2024-09-21T06:18:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 425.0 s, 3638.4 tps, lat 27.484 ms stddev 33.939 2024-09-21T06:18:47.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:46 smithi110 ceph-mon[34550]: pgmap v537: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 10 MiB/s wr, 923 op/s 2024-09-21T06:18:47.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:46 smithi107 ceph-mon[34380]: pgmap v537: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 10 MiB/s wr, 923 op/s 2024-09-21T06:18:47.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:46 smithi031 ceph-mon[31237]: pgmap v537: 97 pgs: 97 active+clean; 4.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 10 MiB/s wr, 923 op/s 2024-09-21T06:18:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:48 smithi110 ceph-mon[34550]: pgmap v538: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.7 MiB/s wr, 810 op/s 2024-09-21T06:18:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:48 smithi107 ceph-mon[34380]: pgmap v538: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.7 MiB/s wr, 810 op/s 2024-09-21T06:18:49.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:48 smithi031 ceph-mon[31237]: pgmap v538: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.7 MiB/s wr, 810 op/s 2024-09-21T06:18:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 430.0 s, 3424.4 tps, lat 29.089 ms stddev 32.920 2024-09-21T06:18:51.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: pgmap v539: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.3 MiB/s wr, 763 op/s 2024-09-21T06:18:51.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: Unable to set osd_memory_target on smithi110 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:18:51.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:18:51.280 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:50 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: pgmap v539: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.3 MiB/s wr, 763 op/s 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: Unable to set osd_memory_target on smithi110 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:18:51.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:18:51.291 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:50 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: pgmap v539: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.3 MiB/s wr, 763 op/s 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi110 to 176.0M 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: Unable to set osd_memory_target on smithi107 to 184531558: error parsing value: Value '184531558' is below minimum 939524096 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: Unable to set osd_memory_target on smithi110 to 184566681: error parsing value: Value '184566681' is below minimum 939524096 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:18:51.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:50 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:18:52.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:51 smithi110 ceph-mon[34550]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-21T06:18:52.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:51 smithi110 ceph-mon[34550]: Unable to set osd_memory_target on smithi031 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-21T06:18:52.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:51 smithi107 ceph-mon[34380]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-21T06:18:52.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:51 smithi107 ceph-mon[34380]: Unable to set osd_memory_target on smithi031 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-21T06:18:52.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:51 smithi031 ceph-mon[31237]: Adjusting osd_memory_target on smithi031 to 177.0M 2024-09-21T06:18:52.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:51 smithi031 ceph-mon[31237]: Unable to set osd_memory_target on smithi031 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-21T06:18:53.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:52 smithi110 ceph-mon[34550]: pgmap v540: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-09-21T06:18:53.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:52 smithi107 ceph-mon[34380]: pgmap v540: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-09-21T06:18:53.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:52 smithi031 ceph-mon[31237]: pgmap v540: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 953 op/s 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 -- 172.21.15.31:0/3662048152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a81047e0 msgr2=0x7f63a8104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 --2- 172.21.15.31:0/3662048152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a81047e0 0x7f63a8104bf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009900 tx=0x7f63a402e800 comp rx=0 tx=0).stop 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 -- 172.21.15.31:0/3662048152 shutdown_connections 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 --2- 172.21.15.31:0/3662048152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a810dd90 0x7f63a8110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 --2- 172.21.15.31:0/3662048152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81051c0 0x7f63a810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 --2- 172.21.15.31:0/3662048152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a81047e0 0x7f63a8104bf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 -- 172.21.15.31:0/3662048152 >> 172.21.15.31:0/3662048152 conn(0x7f63a807a740 msgr2=0x7f63a807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 -- 172.21.15.31:0/3662048152 shutdown_connections 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.288+0000 7f63b007e640 1 -- 172.21.15.31:0/3662048152 wait complete. 2024-09-21T06:18:54.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 Processor -- start 2024-09-21T06:18:54.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 -- start start 2024-09-21T06:18:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 0x7f63a8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a81051c0 0x7f63a81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a810dd90 0x7f63a819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111810 con 0x7f63a810dd90 2024-09-21T06:18:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111690 con 0x7f63a81047e0 2024-09-21T06:18:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.289+0000 7f63b007e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8111990 con 0x7f63a81051c0 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ae61b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a810dd90 0x7f63a819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ade1a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 0x7f63a8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a81051c0 0x7f63a81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ade1a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 0x7f63a8198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48934/0 (socket says 172.21.15.31:48934) 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ade1a640 1 -- 172.21.15.31:0/1590949177 learned_addr learned my addr 172.21.15.31:0/1590949177 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 -- 172.21.15.31:0/1590949177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 msgr2=0x7f63a8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 0x7f63a8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 -- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a810dd90 msgr2=0x7f63a819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a810dd90 0x7f63a819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 -- 172.21.15.31:0/1590949177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a819e140 con 0x7f63a81051c0 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ade1a640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 0x7f63a8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:18:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f63ad619640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a81051c0 0x7f63a81987b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f63a806c9c0 tx=0x7f639800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.290+0000 7f639effd640 1 -- 172.21.15.31:0/1590949177 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800cc40 con 0x7f63a81051c0 2024-09-21T06:18:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.291+0000 7f639effd640 1 -- 172.21.15.31:0/1590949177 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f639800cde0 con 0x7f63a81051c0 2024-09-21T06:18:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.291+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a819e430 con 0x7f63a81051c0 2024-09-21T06:18:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.291+0000 7f639effd640 1 -- 172.21.15.31:0/1590949177 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398018780 con 0x7f63a81051c0 2024-09-21T06:18:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.291+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a4bc0 con 0x7f63a81051c0 2024-09-21T06:18:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.295+0000 7f63b6fec740 1 Processor -- start 2024-09-21T06:18:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.295+0000 7f63b6fec740 1 -- start start 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.295+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba5f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.295+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba5fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.295+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6994d0 0x55f4bba60430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63b6fec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb9d46f0 con 0x55f4bb6994d0 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f639effd640 1 -- 172.21.15.31:0/1590949177 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6398010a90 con 0x7f63a81051c0 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba5f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba5f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58042/0 (socket says 172.21.15.31:58042) 2024-09-21T06:18:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 -- 172.21.15.31:0/1084076402 learned_addr learned my addr 172.21.15.31:0/1084076402 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ade1a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba5fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb9c3b20 con 0x55f4bb6912f0 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ad619640 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6994d0 0x55f4bba60430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb9c7930 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 -- 172.21.15.31:0/1084076402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 msgr2=0x55f4bba5fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba5fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 -- 172.21.15.31:0/1084076402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6994d0 msgr2=0x55f4bba60430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6994d0 0x55f4bba60430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 -- 172.21.15.31:0/1084076402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bb989530 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.296+0000 7f63ae61b640 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba5f9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f63a0011440 tx=0x7f63a0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be337cadd3b1e496 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f6393fff640 1 -- 172.21.15.31:0/1084076402 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004d20 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f6393fff640 1 -- 172.21.15.31:0/1084076402 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a0004050 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f6393fff640 1 -- 172.21.15.31:0/1084076402 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a000f7a0 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f639effd640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63940b6700 0x7f63940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63ade1a640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63940b6700 0x7f63940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f639effd640 1 -- 172.21.15.31:0/1590949177 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f63980e7940 con 0x7f63a81051c0 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 msgr2=0x55f4bba5f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba5f9b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f63a0011440 tx=0x7f63a0011900 comp rx=0 tx=0).stop 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 shutdown_connections 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6994d0 0x55f4bba60430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba5fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 --2- 172.21.15.31:0/1084076402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba5f9b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 >> 172.21.15.31:0/1084076402 conn(0x55f4bb6ae300 msgr2=0x55f4bba5d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.297+0000 7f63ade1a640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63940b6700 0x7f63940b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.298+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 shutdown_connections 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.298+0000 7f63b6fec740 1 -- 172.21.15.31:0/1084076402 wait complete. 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.298+0000 7f63b6fec740 1 -- start start 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.298+0000 7f63b6fec740 1 Processor -- start 2024-09-21T06:18:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6912f0 0x55f4bba6b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba77420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ad619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ad619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58056/0 (socket says 172.21.15.31:58056) 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ad619640 1 -- 172.21.15.31:0/3804760623 learned_addr learned my addr 172.21.15.31:0/3804760623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb8fbab0 con 0x55f4bb6912f0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ae61b640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba77420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f4bb969730 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4bba6e130 con 0x55f4bb6994d0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63b6fec740 10 client.0 _ll_register_callbacks cb 0x55f4bb9bd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6912f0 0x55f4bba6b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ae61b640 1 -- 172.21.15.31:0/3804760623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 msgr2=0x55f4bba6b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ae61b640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ae61b640 1 -- 172.21.15.31:0/3804760623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6912f0 msgr2=0x55f4bba6b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ae61b640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6912f0 0x55f4bba6b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ae61b640 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bb725790 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.299+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6912f0 0x55f4bba6b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.300+0000 7f63ae61b640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba77420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63a00049d0 tx=0x7f63a00044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe3a71d265277542 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.300+0000 7f63917fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.301+0000 7f63917fa640 1 -- 172.21.15.31:0/3804760623 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a001d040 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.301+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a0004050 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.301+0000 7f63917fa640 1 -- 172.21.15.31:0/3804760623 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f63a00046a0 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63917fa640 1 -- 172.21.15.31:0/3804760623 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63a0030050 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 10 client.26777 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f4bb79e620 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63917fa640 1 -- 172.21.15.31:0/3804760623 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a0004050 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63917fa640 1 -- 172.21.15.31:0/3804760623 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f63a0020030 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 10 client.26777 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 10 client.26777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 20 client.26777 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb678d80 0x55f4bba8e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 4 client.26777 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.302+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4bb9d6640 con 0x55f4bb678d80 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.303+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb678d80 0x55f4bba8e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.303+0000 7f63917fa640 10 client.26777 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.303+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb678d80 0x55f4bba8e1a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a0bbffe2ac15d3 server_cookie=d4cf636d23d6ef49 in_seq=0 out_seq=0 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.304+0000 7f63917fa640 1 -- 172.21.15.31:0/3804760623 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4bb9d6640 con 0x55f4bb678d80 2024-09-21T06:18:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.304+0000 7f63917fa640 10 client.26777 handle_command_reply: tid=0 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.304+0000 7f63b6fec740 1 client.26777 shutdown 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.304+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6ba1b0 msgr2=0x55f4bba77420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba77420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63a00049d0 tx=0x7f63a00044b0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 shutdown_connections 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb678d80 0x55f4bba8e1a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6ba1b0 0x55f4bba77420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63917fa640 0 client.26777 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3804760623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb6912f0 0x55f4bba6b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 >> 172.21.15.31:0/3804760623 conn(0x55f4bb6ae300 msgr2=0x55f4bba609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 shutdown_connections 2024-09-21T06:18:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 1 -- 172.21.15.31:0/3804760623 wait complete. 2024-09-21T06:18:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.305+0000 7f63b6fec740 20 client.26777 trim_cache size 0 max 16384 2024-09-21T06:18:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 Processor -- start 2024-09-21T06:18:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 -- start start 2024-09-21T06:18:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 0x55f4bba77320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6994d0 0x55f4bba77860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ab5e0 con 0x55f4bb678d80 2024-09-21T06:18:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ade40 con 0x55f4bb6994d0 2024-09-21T06:18:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.327+0000 7f63b6fec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81abc30 con 0x55f4bb6912f0 2024-09-21T06:18:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ade1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 0x55f4bba77320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ae61b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6994d0 0x55f4bba77860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ae61b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6994d0 0x55f4bba77860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48960/0 (socket says 172.21.15.31:48960) 2024-09-21T06:18:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ae61b640 1 -- 172.21.15.31:0/750817163 learned_addr learned my addr 172.21.15.31:0/750817163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 0x55f4bba77320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58070/0 (socket says 172.21.15.31:58070) 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 -- 172.21.15.31:0/750817163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6994d0 msgr2=0x55f4bba77860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6994d0 0x55f4bba77860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 -- 172.21.15.31:0/750817163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 msgr2=0x55f4bba610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 -- 172.21.15.31:0/750817163 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bba69630 con 0x55f4bb6912f0 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.328+0000 7f63ad619640 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 0x55f4bba77320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63980aaac0 tx=0x7f6398009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9642ee19af328731 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.329+0000 7f63937fe640 1 -- 172.21.15.31:0/750817163 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63980ad620 con 0x55f4bb6912f0 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.329+0000 7f63937fe640 1 -- 172.21.15.31:0/750817163 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63980ad360 con 0x55f4bb6912f0 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.329+0000 7f63937fe640 1 -- 172.21.15.31:0/750817163 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63980acfc0 con 0x55f4bb6912f0 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.330+0000 7f63b6fec740 1 -- 172.21.15.31:0/750817163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 msgr2=0x55f4bba77320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.330+0000 7f63b6fec740 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 0x55f4bba77320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63980aaac0 tx=0x7f6398009700 comp rx=0 tx=0).stop 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 -- 172.21.15.31:0/750817163 shutdown_connections 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6994d0 0x55f4bba77860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6912f0 0x55f4bba77320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 --2- 172.21.15.31:0/750817163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 -- 172.21.15.31:0/750817163 >> 172.21.15.31:0/750817163 conn(0x55f4bb6ae300 msgr2=0x55f4bba609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 -- 172.21.15.31:0/750817163 shutdown_connections 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 -- 172.21.15.31:0/750817163 wait complete. 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.331+0000 7f63b6fec740 1 -- start start 2024-09-21T06:18:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.332+0000 7f63b6fec740 1 Processor -- start 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.332+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba6e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.332+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba6e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.332+0000 7f63b6fec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ade1a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba6e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63b6fec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ac300 con 0x55f4bb678d80 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63b6fec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ad770 con 0x55f4bb6912f0 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63b6fec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81ac9d0 con 0x55f4bb6994d0 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba6e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba6e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48964/0 (socket says 172.21.15.31:48964) 2024-09-21T06:18:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 -- 172.21.15.31:0/3345195156 learned_addr learned my addr 172.21.15.31:0/3345195156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba6e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57340/0 (socket says 172.21.15.31:57340) 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63b6fec740 10 client.0 _ll_register_callbacks cb 0x7f63980ed3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ae61b640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 -- 172.21.15.31:0/3345195156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 msgr2=0x55f4bba6ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 -- 172.21.15.31:0/3345195156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 msgr2=0x55f4bba6e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba6e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 -- 172.21.15.31:0/3345195156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bbaf69e0 con 0x55f4bb6912f0 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63ad619640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba6e8d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63980a9e20 tx=0x7f63980ed250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a6d615c3c98655 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:18:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63917fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63917fa640 1 -- 172.21.15.31:0/3345195156 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63980ad1a0 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63917fa640 1 -- 172.21.15.31:0/3345195156 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f63980a93a0 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.333+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63980ad360 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63917fa640 1 -- 172.21.15.31:0/3345195156 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f63980b2280 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 10 client.26783 fetch_fsmap learned FSMap version 19 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f4bbaf7c70 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63917fa640 1 -- 172.21.15.31:0/3345195156 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63980ad360 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63917fa640 1 -- 172.21.15.31:0/3345195156 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f63980b3410 con 0x55f4bb6912f0 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 10 client.26783 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 10 client.26783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 20 client.26783 populate_metadata read hostname 'smithi031' 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb6ba1b0 0x55f4bbafa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:18:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 4 client.26783 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:18:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4bb9d20f0 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.334+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb6ba1b0 0x55f4bbafa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:18:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.335+0000 7f63ade1a640 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb6ba1b0 0x55f4bbafa3e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbde1bc2366c417 server_cookie=f1e6ce11082e0e2e in_seq=0 out_seq=0 2024-09-21T06:18:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.335+0000 7f63917fa640 10 client.26783 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63917fa640 1 -- 172.21.15.31:0/3345195156 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4bb9d20f0 con 0x55f4bb6ba1b0 2024-09-21T06:18:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63917fa640 10 client.26783 handle_command_reply: tid=0 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 client.26783 shutdown 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 msgr2=0x55f4bba6e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba6e8d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63980a9e20 tx=0x7f63980ed250 comp rx=0 tx=0).stop 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 shutdown_connections 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f4bb6ba1b0 0x55f4bbafa3e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4bb6994d0 0x55f4bba6ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f4bb6912f0 0x55f4bba6e8d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 --2- 172.21.15.31:0/3345195156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4bb678d80 0x55f4bba6e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63917fa640 0 client.26783 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 >> 172.21.15.31:0/3345195156 conn(0x55f4bb6ae300 msgr2=0x55f4bba60930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 shutdown_connections 2024-09-21T06:18:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 1 -- 172.21.15.31:0/3345195156 wait complete. 2024-09-21T06:18:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.358+0000 7f63b6fec740 20 client.26783 trim_cache size 0 max 16384 2024-09-21T06:18:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:18:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:18:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:18:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:18:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63940b6700 msgr2=0x7f63940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63940b6700 0x7f63940b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63a4000c00 tx=0x7f63a4005a20 comp rx=0 tx=0).stop 2024-09-21T06:18:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a81051c0 msgr2=0x7f63a81987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:18:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a81051c0 0x7f63a81987b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f63a806c9c0 tx=0x7f639800ee50 comp rx=0 tx=0).stop 2024-09-21T06:18:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 shutdown_connections 2024-09-21T06:18:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63940b6700 0x7f63940b8c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a810dd90 0x7f63a819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a81051c0 0x7f63a81987b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 --2- 172.21.15.31:0/1590949177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a81047e0 0x7f63a8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:18:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 >> 172.21.15.31:0/1590949177 conn(0x7f63a807a740 msgr2=0x7f63a81007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:18:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 shutdown_connections 2024-09-21T06:18:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:18:54.359+0000 7f63b007e640 1 -- 172.21.15.31:0/1590949177 wait complete. 2024-09-21T06:18:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 435.0 s, 2070.4 tps, lat 48.694 ms stddev 54.152 2024-09-21T06:18:55.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:54 smithi110 ceph-mon[34550]: pgmap v541: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.5 MiB/s wr, 741 op/s 2024-09-21T06:18:55.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:54 smithi107 ceph-mon[34380]: pgmap v541: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.5 MiB/s wr, 741 op/s 2024-09-21T06:18:55.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:54 smithi031 ceph-mon[31237]: pgmap v541: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.5 MiB/s wr, 741 op/s 2024-09-21T06:18:57.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:56 smithi110 ceph-mon[34550]: pgmap v542: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 937 op/s 2024-09-21T06:18:57.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:56 smithi107 ceph-mon[34380]: pgmap v542: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 937 op/s 2024-09-21T06:18:57.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:56 smithi031 ceph-mon[31237]: pgmap v542: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 10 MiB/s wr, 937 op/s 2024-09-21T06:18:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:18:58 smithi110 ceph-mon[34550]: pgmap v543: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-21T06:18:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:18:58 smithi107 ceph-mon[34380]: pgmap v543: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-21T06:18:59.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:18:58 smithi031 ceph-mon[31237]: pgmap v543: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.2 MiB/s wr, 691 op/s 2024-09-21T06:19:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 440.0 s, 2457.0 tps, lat 40.469 ms stddev 46.241 2024-09-21T06:19:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:00 smithi110 ceph-mon[34550]: pgmap v544: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.0 MiB/s wr, 603 op/s 2024-09-21T06:19:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:00 smithi107 ceph-mon[34380]: pgmap v544: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.0 MiB/s wr, 603 op/s 2024-09-21T06:19:01.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:00 smithi031 ceph-mon[31237]: pgmap v544: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.0 MiB/s wr, 603 op/s 2024-09-21T06:19:03.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:02 smithi110 ceph-mon[34550]: pgmap v545: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.0 MiB/s wr, 794 op/s 2024-09-21T06:19:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:02 smithi107 ceph-mon[34380]: pgmap v545: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.0 MiB/s wr, 794 op/s 2024-09-21T06:19:03.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:02 smithi031 ceph-mon[31237]: pgmap v545: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 8.0 MiB/s wr, 794 op/s 2024-09-21T06:19:04.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:19:04.458 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/1846309240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88113eb0 msgr2=0x7f0b881142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/1846309240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88113eb0 0x7f0b881142c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009900 tx=0x7f0b7802a410 comp rx=0 tx=0).stop 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/1846309240 shutdown_connections 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/1846309240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88114800 0x7f0b8811b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/1846309240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88113eb0 0x7f0b881142c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/1846309240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8810e5c0 0x7f0b8810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/1846309240 >> 172.21.15.31:0/1846309240 conn(0x7f0b8810a3c0 msgr2=0x7f0b8810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/1846309240 shutdown_connections 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.985+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/1846309240 wait complete. 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.986+0000 7f0b8f7e1640 1 Processor -- start 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.986+0000 7f0b8f7e1640 1 -- start start 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8f7e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8810e5c0 0x7f0b881184f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8f7e1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 0x7f0b88118a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8f7e1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88114800 0x7f0b88118f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8f7e1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8811e470 con 0x7f0b88114800 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8f7e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8811e2f0 con 0x7f0b8810e5c0 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8f7e1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8811e5f0 con 0x7f0b88113eb0 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 0x7f0b88118a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 0x7f0b88118a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:35850/0 (socket says 172.21.15.31:35850) 2024-09-21T06:19:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 -- 172.21.15.31:0/2875475766 learned_addr learned my addr 172.21.15.31:0/2875475766 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88114800 0x7f0b88118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8810e5c0 0x7f0b881184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 -- 172.21.15.31:0/2875475766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8810e5c0 msgr2=0x7f0b881184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8810e5c0 0x7f0b881184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 -- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88114800 msgr2=0x7f0b88118f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88114800 0x7f0b88118f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.987+0000 7f0b8cd7c640 1 -- 172.21.15.31:0/2875475766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881b5340 con 0x7f0b88113eb0 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.988+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 0x7f0b88118a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0b7802a930 tx=0x7f0b78031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.988+0000 7f0b7e7fc640 1 -- 172.21.15.31:0/2875475766 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7803f030 con 0x7f0b88113eb0 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.988+0000 7f0b7e7fc640 1 -- 172.21.15.31:0/2875475766 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b78004490 con 0x7f0b88113eb0 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.988+0000 7f0b7e7fc640 1 -- 172.21.15.31:0/2875475766 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78035070 con 0x7f0b88113eb0 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.988+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881b55d0 con 0x7f0b88113eb0 2024-09-21T06:19:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.988+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881b5b00 con 0x7f0b88113eb0 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.991+0000 7f0b95d55740 1 Processor -- start 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.991+0000 7f0b95d55740 1 -- start start 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd98d1b0 0x5648bdd5d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd5dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b95d55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5648bdcd3e50 con 0x5648bd995390 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b95d55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648bdcc3280 con 0x5648bd9b6070 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b95d55740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5648bdcc7090 con 0x5648bd98d1b0 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd5d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58856/0 (socket says 172.21.15.31:58856) 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b7e7fc640 1 -- 172.21.15.31:0/2875475766 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0b78042030 con 0x7f0b88113eb0 2024-09-21T06:19:04.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/1908663512 learned_addr learned my addr 172.21.15.31:0/1908663512 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd98d1b0 0x5648bdd5d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/1908663512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd98d1b0 msgr2=0x5648bdd5d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd98d1b0 0x5648bdd5d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/1908663512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 msgr2=0x5648bdd5dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd5dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd5dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/1908663512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bda21650 con 0x5648bd9b6070 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.992+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd5dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.993+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd5d270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fd0 tx=0x7f0b800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b97463982c40e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:04.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.993+0000 7f0b737fe640 1 -- 172.21.15.31:0/1908663512 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80004d20 con 0x5648bd9b6070 2024-09-21T06:19:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.993+0000 7f0b737fe640 1 -- 172.21.15.31:0/1908663512 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80004050 con 0x5648bd9b6070 2024-09-21T06:19:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.993+0000 7f0b737fe640 1 -- 172.21.15.31:0/1908663512 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80005280 con 0x5648bd9b6070 2024-09-21T06:19:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.993+0000 7f0b7e7fc640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b740b6700 0x7f0b740b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.993+0000 7f0b7e7fc640 1 -- 172.21.15.31:0/2875475766 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f0b78107a20 con 0x7f0b88113eb0 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b740b6700 0x7f0b740b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b740b6700 0x7f0b740b8c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5648bdd5bef0 tx=0x7f0b84005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 -- 172.21.15.31:0/1908663512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 msgr2=0x5648bdd5d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd5d270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fd0 tx=0x7f0b800118d0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 -- 172.21.15.31:0/1908663512 shutdown_connections 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd5dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd98d1b0 0x5648bdd5d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 --2- 172.21.15.31:0/1908663512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd5d270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.994+0000 7f0b95d55740 1 -- 172.21.15.31:0/1908663512 >> 172.21.15.31:0/1908663512 conn(0x5648bd9aa460 msgr2=0x5648bdd5b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.995+0000 7f0b95d55740 1 -- 172.21.15.31:0/1908663512 shutdown_connections 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.995+0000 7f0b95d55740 1 -- 172.21.15.31:0/1908663512 wait complete. 2024-09-21T06:19:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.995+0000 7f0b95d55740 1 -- start start 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.995+0000 7f0b95d55740 1 Processor -- start 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd68dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd9b6070 0x5648bdd69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5648bdb24700 con 0x5648bd9b6070 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5648bdc5e110 con 0x5648bd98d1b0 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5648bdd6ba50 con 0x5648bd995390 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b95d55740 10 client.0 _ll_register_callbacks cb 0x5648bdc77990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8dd7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd9b6070 0x5648bdd69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58858/0 (socket says 172.21.15.31:58858) 2024-09-21T06:19:05.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 -- 172.21.15.31:0/838829556 learned_addr learned my addr 172.21.15.31:0/838829556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd68dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 -- 172.21.15.31:0/838829556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 msgr2=0x5648bdd68dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd68dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 -- 172.21.15.31:0/838829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd9b6070 msgr2=0x5648bdd69330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd9b6070 0x5648bdd69330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8d57d640 1 -- 172.21.15.31:0/838829556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bdc8bc40 con 0x5648bd98d1b0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.996+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd9b6070 0x5648bdd69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.997+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b840058a0 tx=0x7f0b84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbd103157f376bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.997+0000 7f0b70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.997+0000 7f0b70ff9640 1 -- 172.21.15.31:0/838829556 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8401f640 con 0x5648bd98d1b0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.997+0000 7f0b70ff9640 1 -- 172.21.15.31:0/838829556 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f0b8401f7e0 con 0x5648bd98d1b0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.997+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b80004050 con 0x5648bd98d1b0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b70ff9640 1 -- 172.21.15.31:0/838829556 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b8401f9e0 con 0x5648bd98d1b0 2024-09-21T06:19:05.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 10 client.26795 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5648bdd77720 con 0x5648bd98d1b0 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b70ff9640 1 -- 172.21.15.31:0/838829556 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80004050 con 0x5648bd98d1b0 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b70ff9640 1 -- 172.21.15.31:0/838829556 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0b84027070 con 0x5648bd98d1b0 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 10 client.26795 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 10 client.26795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 20 client.26795 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd9749f0 0x5648bdd7ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 4 client.26795 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.998+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648bdcd5da0 con 0x5648bd9749f0 2024-09-21T06:19:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.999+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd9749f0 0x5648bdd7ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.999+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd9749f0 0x5648bdd7ac30 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885ebd8c6664078a server_cookie=57f988451ede13c8 in_seq=0 out_seq=0 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:04.999+0000 7f0b70ff9640 10 client.26795 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b70ff9640 1 -- 172.21.15.31:0/838829556 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5648bdcd5da0 con 0x5648bd9749f0 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b70ff9640 10 client.26795 handle_command_reply: tid=0 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 client.26795 shutdown 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 msgr2=0x5648bdd68890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b840058a0 tx=0x7f0b84004730 comp rx=0 tx=0).stop 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 shutdown_connections 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd9749f0 0x5648bdd7ac30 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd9b6070 0x5648bdd69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd68dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 --2- 172.21.15.31:0/838829556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68890 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 >> 172.21.15.31:0/838829556 conn(0x5648bd9aa460 msgr2=0x5648bdd60400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.000+0000 7f0b70ff9640 0 client.26795 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.001+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 shutdown_connections 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.001+0000 7f0b95d55740 1 -- 172.21.15.31:0/838829556 wait complete. 2024-09-21T06:19:05.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.001+0000 7f0b95d55740 20 client.26795 trim_cache size 0 max 16384 2024-09-21T06:19:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.019+0000 7f0b95d55740 1 Processor -- start 2024-09-21T06:19:05.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.019+0000 7f0b95d55740 1 -- start start 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.019+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd9b6070 0x5648bdd65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b95d55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881bc5d0 con 0x5648bd995390 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b95d55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881bee30 con 0x5648bd98d1b0 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b95d55740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881bcc20 con 0x5648bd9b6070 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8cd7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58868/0 (socket says 172.21.15.31:58868) 2024-09-21T06:19:05.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8cd7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd74c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56082/0 (socket says 172.21.15.31:56082) 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 -- 172.21.15.31:0/2946440872 learned_addr learned my addr 172.21.15.31:0/2946440872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd9b6070 0x5648bdd65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 -- 172.21.15.31:0/2946440872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd9b6070 msgr2=0x5648bdd65a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd9b6070 0x5648bdd65a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 -- 172.21.15.31:0/2946440872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 msgr2=0x5648bdd74c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd74c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 -- 172.21.15.31:0/2946440872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bdd677f0 con 0x5648bd98d1b0 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.020+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009b80 tx=0x7f0b8401d600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3574efd24a89136 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.021+0000 7f0b72ffd640 1 -- 172.21.15.31:0/2946440872 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8401f840 con 0x5648bd98d1b0 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.021+0000 7f0b72ffd640 1 -- 172.21.15.31:0/2946440872 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b840298f0 con 0x5648bd98d1b0 2024-09-21T06:19:05.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.021+0000 7f0b72ffd640 1 -- 172.21.15.31:0/2946440872 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8401d980 con 0x5648bd98d1b0 2024-09-21T06:19:05.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 -- 172.21.15.31:0/2946440872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 msgr2=0x5648bdd68d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009b80 tx=0x7f0b8401d600 comp rx=0 tx=0).stop 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 -- 172.21.15.31:0/2946440872 shutdown_connections 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd9b6070 0x5648bdd65a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd995390 0x5648bdd74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 --2- 172.21.15.31:0/2946440872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd98d1b0 0x5648bdd68d50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 -- 172.21.15.31:0/2946440872 >> 172.21.15.31:0/2946440872 conn(0x5648bd9aa460 msgr2=0x5648bdd68640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 -- 172.21.15.31:0/2946440872 shutdown_connections 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 -- 172.21.15.31:0/2946440872 wait complete. 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.022+0000 7f0b95d55740 1 -- start start 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 Processor -- start 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 0x5648bdd713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd6c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd6ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881bd2f0 con 0x5648bd98d1b0 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881be760 con 0x5648bd9b6070 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881bd9c0 con 0x5648bd995390 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b95d55740 10 client.0 _ll_register_callbacks cb 0x7f0b84009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b8d57d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 0x5648bdd713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b8dd7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd6ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b8d57d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 0x5648bdd713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56086/0 (socket says 172.21.15.31:56086) 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.023+0000 7f0b8d57d640 1 -- 172.21.15.31:0/3994580373 learned_addr learned my addr 172.21.15.31:0/3994580373 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8cd7c640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd6c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/3994580373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 msgr2=0x5648bdd6c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd6c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/3994580373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 msgr2=0x5648bdd713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 0x5648bdd713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8dd7e640 1 -- 172.21.15.31:0/3994580373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bdd6d4b0 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 0x5648bdd713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b8dd7e640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd6ce10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b80003fd0 tx=0x7f0b80009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1112152d330ade server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b70ff9640 1 -- 172.21.15.31:0/3994580373 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800043a0 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b840298f0 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.024+0000 7f0b70ff9640 1 -- 172.21.15.31:0/3994580373 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f0b80004540 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b70ff9640 1 -- 172.21.15.31:0/3994580373 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0b80004740 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 10 client.26801 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5648bdde5450 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b70ff9640 1 -- 172.21.15.31:0/3994580373 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b840298f0 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b70ff9640 1 -- 172.21.15.31:0/3994580373 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0b8000f670 con 0x5648bd9b6070 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 10 client.26801 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:05.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 20 client.26801 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd949da0 0x5648bdde7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 4 client.26801 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648bdcd1850 con 0x5648bd949da0 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.025+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd949da0 0x5648bdde7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b70ff9640 10 client.26801 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:05.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b8d57d640 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd949da0 0x5648bdde7940 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a25dc1ece968ef8 server_cookie=74e9bf1d685f174 in_seq=0 out_seq=0 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b70ff9640 1 -- 172.21.15.31:0/3994580373 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x5648bdcd1850 con 0x5648bd949da0 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b70ff9640 10 client.26801 handle_command_reply: tid=0 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 client.26801 shutdown 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 msgr2=0x5648bdd6ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd6ce10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b80003fd0 tx=0x7f0b80009d60 comp rx=0 tx=0).stop 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 shutdown_connections 2024-09-21T06:19:05.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5648bd949da0 0x5648bdde7940 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5648bd9b6070 0x5648bdd6ce10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b70ff9640 0 client.26801 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.026+0000 7f0b95d55740 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648bd995390 0x5648bdd6c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.027+0000 7f0b95d55740 1 --2- 172.21.15.31:0/3994580373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5648bd98d1b0 0x5648bdd713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.027+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 >> 172.21.15.31:0/3994580373 conn(0x5648bd9aa460 msgr2=0x5648bdd5e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.027+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 shutdown_connections 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.027+0000 7f0b95d55740 1 -- 172.21.15.31:0/3994580373 wait complete. 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.027+0000 7f0b95d55740 20 client.26801 trim_cache size 0 max 16384 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:19:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:19:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:05.045 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:19:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b740b6700 msgr2=0x7f0b740b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b740b6700 0x7f0b740b8c10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x5648bdd5bef0 tx=0x7f0b84005f90 comp rx=0 tx=0).stop 2024-09-21T06:19:05.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 msgr2=0x7f0b88118a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 0x7f0b88118a30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0b7802a930 tx=0x7f0b78031a20 comp rx=0 tx=0).stop 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 shutdown_connections 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b740b6700 0x7f0b740b8c10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b88114800 0x7f0b88118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b88113eb0 0x7f0b88118a30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 --2- 172.21.15.31:0/2875475766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b8810e5c0 0x7f0b881184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 >> 172.21.15.31:0/2875475766 conn(0x7f0b8810a3c0 msgr2=0x7f0b881134e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 shutdown_connections 2024-09-21T06:19:05.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:05.028+0000 7f0b8f7e1640 1 -- 172.21.15.31:0/2875475766 wait complete. 2024-09-21T06:19:05.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:04 smithi031 ceph-mon[31237]: pgmap v546: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 534 op/s 2024-09-21T06:19:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 445.0 s, 586.6 tps, lat 164.647 ms stddev 259.691 2024-09-21T06:19:05.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:04 smithi110 ceph-mon[34550]: pgmap v546: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 534 op/s 2024-09-21T06:19:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:04 smithi107 ceph-mon[34380]: pgmap v546: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.2 MiB/s wr, 534 op/s 2024-09-21T06:19:05.960 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:19:05.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:19:06.252 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 -- 172.21.15.31:0/834614176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8109e40 msgr2=0x7f48a8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 --2- 172.21.15.31:0/834614176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8109e40 0x7f48a8110670 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009900 tx=0x7f48a002a410 comp rx=0 tx=0).stop 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 -- 172.21.15.31:0/834614176 shutdown_connections 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 --2- 172.21.15.31:0/834614176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8109e40 0x7f48a8110670 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 --2- 172.21.15.31:0/834614176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a81094f0 0x7f48a8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 --2- 172.21.15.31:0/834614176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48a8103bb0 0x7f48a8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 -- 172.21.15.31:0/834614176 >> 172.21.15.31:0/834614176 conn(0x7f48a80ff600 msgr2=0x7f48a8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 -- 172.21.15.31:0/834614176 shutdown_connections 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 -- 172.21.15.31:0/834614176 wait complete. 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 Processor -- start 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.777+0000 7f48b01fe640 1 -- start start 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48b01fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a8103bb0 0x7f48a819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48b01fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 0x7f48a819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48b01fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48a8109e40 0x7f48a81a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48b01fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8113b00 con 0x7f48a81094f0 2024-09-21T06:19:06.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48b01fe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8113980 con 0x7f48a8103bb0 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48b01fe640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8113c80 con 0x7f48a8109e40 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 0x7f48a819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 0x7f48a819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56106/0 (socket says 172.21.15.31:56106) 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 -- 172.21.15.31:0/3456196361 learned_addr learned my addr 172.21.15.31:0/3456196361 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48adf9a640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a8103bb0 0x7f48a819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ae79b640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48a8109e40 0x7f48a81a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 -- 172.21.15.31:0/3456196361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48a8109e40 msgr2=0x7f48a81a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48a8109e40 0x7f48a81a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 -- 172.21.15.31:0/3456196361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a8103bb0 msgr2=0x7f48a819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a8103bb0 0x7f48a819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 -- 172.21.15.31:0/3456196361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a81a2640 con 0x7f48a81094f0 2024-09-21T06:19:06.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.778+0000 7f48ad799640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 0x7f48a819cc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9f0 tx=0x7f489800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.779+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489800ccc0 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.779+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48980108e0 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.779+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a81a2930 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.779+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898018860 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.779+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a81a9090 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.781+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a8103fc0 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.781+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4898010430 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.782+0000 7f489effd640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48940b6680 0x7f48940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.782+0000 7f48adf9a640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48940b6680 0x7f48940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.782+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7f48980e79b0 con 0x7f48a81094f0 2024-09-21T06:19:06.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.783+0000 7f48adf9a640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48940b6680 0x7f48940b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f48a4005fd0 tx=0x7f48a4009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:06.787+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f48980b22f0 con 0x7f48a81094f0 2024-09-21T06:19:07.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.059+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48a81a92c0 con 0x7f48a81094f0 2024-09-21T06:19:07.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.059+0000 7f489effd640 1 -- 172.21.15.31:0/3456196361 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f48980b7230 con 0x7f48a81094f0 2024-09-21T06:19:07.062 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:19:07.062 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:19:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48940b6680 msgr2=0x7f48940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48940b6680 0x7f48940b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f48a4005fd0 tx=0x7f48a4009500 comp rx=0 tx=0).stop 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 msgr2=0x7f48a819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 0x7f48a819cc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f489800e9f0 tx=0x7f489800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 shutdown_connections 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48a8109e40 0x7f48a81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f48940b6680 0x7f48940b8b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a81094f0 0x7f48a819cc70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 --2- 172.21.15.31:0/3456196361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48a8103bb0 0x7f48a819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 >> 172.21.15.31:0/3456196361 conn(0x7f48a80ff600 msgr2=0x7f48a8106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 shutdown_connections 2024-09-21T06:19:07.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:07.062+0000 7f48b01fe640 1 -- 172.21.15.31:0/3456196361 wait complete. 2024-09-21T06:19:07.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:06 smithi110 ceph-mon[34550]: pgmap v547: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.1 MiB/s wr, 684 op/s 2024-09-21T06:19:07.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:06 smithi107 ceph-mon[34380]: pgmap v547: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.1 MiB/s wr, 684 op/s 2024-09-21T06:19:07.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:06 smithi031 ceph-mon[31237]: pgmap v547: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.1 MiB/s wr, 684 op/s 2024-09-21T06:19:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/1762217147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109d80 msgr2=0x7fa9f41105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/1762217147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109d80 0x7fa9f41105b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009ca0 tx=0x7fa9ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/1762217147 shutdown_connections 2024-09-21T06:19:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/1762217147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109d80 0x7fa9f41105b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/1762217147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4109430 0x7fa9f4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/1762217147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f4103b00 0x7fa9f4103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/1762217147 >> 172.21.15.31:0/1762217147 conn(0x7fa9f40ff500 msgr2=0x7fa9f4101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/1762217147 shutdown_connections 2024-09-21T06:19:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.473+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/1762217147 wait complete. 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 Processor -- start 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 -- start start 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 0x7fa9f419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 0x7fa9f419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f4109d80 0x7fa9f419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4113a90 con 0x7fa9f4109430 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4113910 con 0x7fa9f4103b00 2024-09-21T06:19:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f9ca9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4113c10 con 0x7fa9f4109d80 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 0x7fa9f419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 0x7fa9f419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 0x7fa9f419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56118/0 (socket says 172.21.15.31:56118) 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 0x7fa9f419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58922/0 (socket says 172.21.15.31:58922) 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/855071824 learned_addr learned my addr 172.21.15.31:0/855071824 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.474+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f4109d80 0x7fa9f419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/855071824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f4109d80 msgr2=0x7fa9f419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f4109d80 0x7fa9f419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/855071824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 msgr2=0x7fa9f419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 0x7fa9f419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/855071824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f41a0400 con 0x7fa9f4109430 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 0x7fa9f419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 0x7fa9f419aa70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e9f0 tx=0x7fa9e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f0ff9640 1 -- 172.21.15.31:0/855071824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e800ccc0 con 0x7fa9f4109430 2024-09-21T06:19:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f0ff9640 1 -- 172.21.15.31:0/855071824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e80108e0 con 0x7fa9f4109430 2024-09-21T06:19:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41a06f0 con 0x7fa9f4109430 2024-09-21T06:19:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f0ff9640 1 -- 172.21.15.31:0/855071824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8018860 con 0x7fa9f4109430 2024-09-21T06:19:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.475+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a6e80 con 0x7fa9f4109430 2024-09-21T06:19:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.478+0000 7fa9f0ff9640 1 -- 172.21.15.31:0/855071824 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa9e8010430 con 0x7fa9f4109430 2024-09-21T06:19:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.478+0000 7faa00c1a740 1 Processor -- start 2024-09-21T06:19:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.478+0000 7faa00c1a740 1 -- start start 2024-09-21T06:19:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91ef79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91ef8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7faa00c1a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560c91e6c6f0 con 0x560c91b314d0 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7faa00c1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c91e5bb20 con 0x560c91b292f0 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7faa00c1a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560c91e5f930 con 0x560c91b521b0 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91ef8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91ef8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56132/0 (socket says 172.21.15.31:56132) 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/3689300219 learned_addr learned my addr 172.21.15.31:0/3689300219 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91ef79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3689300219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 msgr2=0x560c91ef79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91ef79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3689300219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 msgr2=0x560c91ef8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91ef8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3689300219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c91e21530 con 0x560c91b292f0 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f0ff9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa9dc0b6680 0x7fa9dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.479+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef7ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4011440 tx=0x7fa9e4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82842e415e1701e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.480+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa9dc0b6680 0x7fa9dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.480+0000 7fa9f0ff9640 1 -- 172.21.15.31:0/855071824 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fa9e80e7960 con 0x7fa9f4109430 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.480+0000 7fa9d9ffb640 1 -- 172.21.15.31:0/3689300219 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4004020 con 0x560c91b292f0 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.480+0000 7fa9d9ffb640 1 -- 172.21.15.31:0/3689300219 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e40041c0 con 0x560c91b292f0 2024-09-21T06:19:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.480+0000 7fa9d9ffb640 1 -- 172.21.15.31:0/3689300219 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400f840 con 0x560c91b292f0 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 -- 172.21.15.31:0/3689300219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 msgr2=0x560c91ef7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef7ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4011440 tx=0x7fa9e4011900 comp rx=0 tx=0).stop 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa9dc0b6680 0x7fa9dc0b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec006010 tx=0x7fa9ec005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 -- 172.21.15.31:0/3689300219 shutdown_connections 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91ef8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef7ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3689300219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91ef79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 -- 172.21.15.31:0/3689300219 >> 172.21.15.31:0/3689300219 conn(0x560c91b46300 msgr2=0x560c91ef58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 -- 172.21.15.31:0/3689300219 shutdown_connections 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.481+0000 7faa00c1a740 1 -- 172.21.15.31:0/3689300219 wait complete. 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 -- start start 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 Processor -- start 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91f030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91f035e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f0f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560c91d93ab0 con 0x560c91b314d0 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c91e01730 con 0x560c91b292f0 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560c91f06130 con 0x560c91b521b0 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.482+0000 7faa00c1a740 10 client.0 _ll_register_callbacks cb 0x560c91e556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91f035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91f030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f0f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91f030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58936/0 (socket says 172.21.15.31:58936) 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3354404873 learned_addr learned my addr 172.21.15.31:0/3354404873 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91f035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56140/0 (socket says 172.21.15.31:56140) 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3354404873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 msgr2=0x560c91f0f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f0f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3354404873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 msgr2=0x560c91f035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91f035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 -- 172.21.15.31:0/3354404873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c91bbd790 con 0x560c91b292f0 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91f030a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9e401b040 tx=0x7fa9e4003a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15776a0a4fc85e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/3354404873 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4025020 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/3354404873 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fa9e4004370 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.483+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e40041c0 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/3354404873 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa9e4030050 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 10 client.26819 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560c91c36620 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/3354404873 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e40041c0 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/3354404873 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa9e4004850 con 0x560c91b292f0 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 10 client.26819 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 10 client.26819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 20 client.26819 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91b10d80 0x560c91f150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 4 client.26819 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c91e6e640 con 0x560c91b10d80 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.484+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91b10d80 0x560c91f150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.485+0000 7fa9d2ffd640 10 client.26819 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.485+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91b10d80 0x560c91f150c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa6f0acb36c98ba server_cookie=c1ec39526fa0df1b in_seq=0 out_seq=0 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.485+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/3354404873 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c91e6e640 con 0x560c91b10d80 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.485+0000 7fa9d2ffd640 10 client.26819 handle_command_reply: tid=0 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.485+0000 7faa00c1a740 1 client.26819 shutdown 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 msgr2=0x560c91f030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91f030a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa9e401b040 tx=0x7fa9e4003a80 comp rx=0 tx=0).stop 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 shutdown_connections 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91b10d80 0x560c91f150c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f0f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b314d0 0x560c91f035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3354404873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91f030a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 >> 172.21.15.31:0/3354404873 conn(0x560c91b46300 msgr2=0x560c91ef89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7fa9d2ffd640 0 client.26819 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 shutdown_connections 2024-09-21T06:19:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 1 -- 172.21.15.31:0/3354404873 wait complete. 2024-09-21T06:19:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.486+0000 7faa00c1a740 20 client.26819 trim_cache size 0 max 16384 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 Processor -- start 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 -- start start 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b314d0 0x560c91f0f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 0x560c91ef4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41ad8a0 con 0x560c91b521b0 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41b0100 con 0x560c91b292f0 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7faa00c1a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41adef0 con 0x560c91b314d0 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 0x560c91ef4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 0x560c91ef4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56152/0 (socket says 172.21.15.31:56152) 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b314d0 0x560c91f0f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 -- 172.21.15.31:0/3762019057 learned_addr learned my addr 172.21.15.31:0/3762019057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 -- 172.21.15.31:0/3762019057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b314d0 msgr2=0x560c91f0f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b314d0 0x560c91f0f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 -- 172.21.15.31:0/3762019057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 msgr2=0x560c91ef90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f3fff640 1 -- 172.21.15.31:0/3762019057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c91f01580 con 0x560c91b521b0 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.502+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.503+0000 7fa9f3fff640 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 0x560c91ef4eb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009ca0 tx=0x7fa9ec041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0a7457aa8f06d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.503+0000 7fa9d97fa640 1 -- 172.21.15.31:0/3762019057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0316d0 con 0x560c91b521b0 2024-09-21T06:19:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.503+0000 7fa9d97fa640 1 -- 172.21.15.31:0/3762019057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ec036050 con 0x560c91b521b0 2024-09-21T06:19:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.503+0000 7fa9d97fa640 1 -- 172.21.15.31:0/3762019057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0045b0 con 0x560c91b521b0 2024-09-21T06:19:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 -- 172.21.15.31:0/3762019057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 msgr2=0x560c91ef4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 0x560c91ef4eb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009ca0 tx=0x7fa9ec041000 comp rx=0 tx=0).stop 2024-09-21T06:19:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 -- 172.21.15.31:0/3762019057 shutdown_connections 2024-09-21T06:19:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b521b0 0x560c91ef4eb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b314d0 0x560c91f0f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 --2- 172.21.15.31:0/3762019057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b292f0 0x560c91ef90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 -- 172.21.15.31:0/3762019057 >> 172.21.15.31:0/3762019057 conn(0x560c91b46300 msgr2=0x560c91ef89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 -- 172.21.15.31:0/3762019057 shutdown_connections 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 -- 172.21.15.31:0/3762019057 wait complete. 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 -- start start 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.504+0000 7faa00c1a740 1 Processor -- start 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 0x560c91f0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 0x560c91f063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f06ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41ae5c0 con 0x560c91b292f0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41afa30 con 0x560c91b314d0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41aec90 con 0x560c91b521b0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7faa00c1a740 10 client.0 _ll_register_callbacks cb 0x7fa9e80ed420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 0x560c91f0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 0x560c91f063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f06ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 0x560c91f063d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58950/0 (socket says 172.21.15.31:58950) 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 0x560c91f0bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56158/0 (socket says 172.21.15.31:56158) 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/717818399 learned_addr learned my addr 172.21.15.31:0/717818399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/717818399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 msgr2=0x560c91f06ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f06ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/717818399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 msgr2=0x560c91f0bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 0x560c91f0bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f2ffd640 1 -- 172.21.15.31:0/717818399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c91f81d20 con 0x560c91b314d0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.505+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 0x560c91f0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9f2ffd640 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 0x560c91f063d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80096e0 tx=0x7fa9e80e7d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650f9f0534e5365a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/717818399 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80ad1c0 con 0x560c91b314d0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/717818399 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8499+0+0 (secure 0 0 0) 0x7fa9e80a91c0 con 0x560c91b314d0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ec036050 con 0x560c91b314d0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/717818399 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa9e80acfe0 con 0x560c91b314d0 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 10 client.26825 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560c91f82f10 con 0x560c91b314d0 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/717818399 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ec036050 con 0x560c91b314d0 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/717818399 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa9e80a93c0 con 0x560c91b314d0 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 10 client.26825 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 10 client.26825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 20 client.26825 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91ae61c0 0x560c91f85630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 4 client.26825 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.506+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c91e6a0f0 con 0x560c91ae61c0 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.507+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91ae61c0 0x560c91f85630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.507+0000 7fa9d2ffd640 10 client.26825 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.507+0000 7fa9f37fe640 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91ae61c0 0x560c91f85630 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bec9bb96640843 server_cookie=77b7f595bdda40d in_seq=0 out_seq=0 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.522+0000 7fa9d2ffd640 1 -- 172.21.15.31:0/717818399 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c91e6a0f0 con 0x560c91ae61c0 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.522+0000 7fa9d2ffd640 10 client.26825 handle_command_reply: tid=0 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.522+0000 7faa00c1a740 1 client.26825 shutdown 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.522+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 msgr2=0x560c91f063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.522+0000 7faa00c1a740 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 0x560c91f063d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9e80096e0 tx=0x7fa9e80e7d50 comp rx=0 tx=0).stop 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 shutdown_connections 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560c91ae61c0 0x560c91f85630 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560c91b521b0 0x560c91f06ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c91b314d0 0x560c91f063d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9d2ffd640 0 client.26825 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 --2- 172.21.15.31:0/717818399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560c91b292f0 0x560c91f0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 >> 172.21.15.31:0/717818399 conn(0x560c91b46300 msgr2=0x560c91f04ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 shutdown_connections 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 1 -- 172.21.15.31:0/717818399 wait complete. 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7faa00c1a740 20 client.26825 trim_cache size 0 max 16384 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:19:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:19:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:19:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:19:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa9dc0b6680 msgr2=0x7fa9dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa9dc0b6680 0x7fa9dc0b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec006010 tx=0x7fa9ec005d80 comp rx=0 tx=0).stop 2024-09-21T06:19:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 msgr2=0x7fa9f419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 0x7fa9f419aa70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800e9f0 tx=0x7fa9e800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 shutdown_connections 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9f4109d80 0x7fa9f419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa9dc0b6680 0x7fa9dc0b8b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9f4109430 0x7fa9f419aa70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 --2- 172.21.15.31:0/855071824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f4103b00 0x7fa9f419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.523+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 >> 172.21.15.31:0/855071824 conn(0x7fa9f40ff500 msgr2=0x7fa9f4108a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.524+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 shutdown_connections 2024-09-21T06:19:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:07.524+0000 7fa9f9ca9640 1 -- 172.21.15.31:0/855071824 wait complete. 2024-09-21T06:19:07.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:19:08.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:07 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3456196361' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:08.173 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:08.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:07 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3456196361' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:08.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:07 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3456196361' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/4280597212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb58108d00 msgr2=0x7fcb5810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/4280597212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb58108d00 0x7fcb5810f530 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009900 tx=0x7fcb5002a410 comp rx=0 tx=0).stop 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/4280597212 shutdown_connections 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/4280597212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb58108d00 0x7fcb5810f530 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/4280597212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb581083b0 0x7fcb581087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/4280597212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb58102ac0 0x7fcb58102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/4280597212 >> 172.21.15.31:0/4280597212 conn(0x7fcb58079720 msgr2=0x7fcb58079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/4280597212 shutdown_connections 2024-09-21T06:19:08.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.683+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/4280597212 wait complete. 2024-09-21T06:19:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.684+0000 7fcb5f1a4640 1 Processor -- start 2024-09-21T06:19:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.684+0000 7fcb5f1a4640 1 -- start start 2024-09-21T06:19:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5f1a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58102ac0 0x7fcb5819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5f1a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 0x7fcb5819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5f1a4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb58108d00 0x7fcb581a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5f1a4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581129c0 con 0x7fcb581083b0 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5f1a4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb58112840 con 0x7fcb58102ac0 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5f1a4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb58112b40 con 0x7fcb58108d00 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 0x7fcb5819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5cf40640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58102ac0 0x7fcb5819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 0x7fcb5819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56170/0 (socket says 172.21.15.31:56170) 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb57fff640 1 -- 172.21.15.31:0/3971418316 learned_addr learned my addr 172.21.15.31:0/3971418316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5d741640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb58108d00 0x7fcb581a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5cf40640 1 -- 172.21.15.31:0/3971418316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb58108d00 msgr2=0x7fcb581a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb58108d00 0x7fcb581a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5cf40640 1 -- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 msgr2=0x7fcb5819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 0x7fcb5819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb5cf40640 1 -- 172.21.15.31:0/3971418316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb581a2680 con 0x7fcb58102ac0 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.685+0000 7fcb57fff640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 0x7fcb5819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.686+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58102ac0 0x7fcb5819c710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9f0 tx=0x7fcb4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.686+0000 7fcb55ffb640 1 -- 172.21.15.31:0/3971418316 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000ccc0 con 0x7fcb58102ac0 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.686+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb581a2970 con 0x7fcb58102ac0 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.686+0000 7fcb55ffb640 1 -- 172.21.15.31:0/3971418316 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb40005020 con 0x7fcb58102ac0 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.686+0000 7fcb55ffb640 1 -- 172.21.15.31:0/3971418316 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40004a00 con 0x7fcb58102ac0 2024-09-21T06:19:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.686+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb581a90d0 con 0x7fcb58102ac0 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb55ffb640 1 -- 172.21.15.31:0/3971418316 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcb40004ba0 con 0x7fcb58102ac0 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 Processor -- start 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 -- start start 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef699f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65cf1b0 0x556ef699f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef699fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556ef6915ea0 con 0x556ef65cf1b0 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556ef69052d0 con 0x556ef65d7390 2024-09-21T06:19:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb65718740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556ef69090e0 con 0x556ef65f8070 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef699fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb5cf40640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65cf1b0 0x556ef699f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb5d741640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef699f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef699fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58976/0 (socket says 172.21.15.31:58976) 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.689+0000 7fcb57fff640 1 -- 172.21.15.31:0/3296054380 learned_addr learned my addr 172.21.15.31:0/3296054380 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5cf40640 1 -- 172.21.15.31:0/3296054380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 msgr2=0x556ef699f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef699f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5cf40640 1 -- 172.21.15.31:0/3296054380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 msgr2=0x556ef699fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef699fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5cf40640 1 -- 172.21.15.31:0/3296054380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef6663650 con 0x556ef65cf1b0 2024-09-21T06:19:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb55ffb640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb440b6680 0x7fcb440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65cf1b0 0x556ef699f800 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcb400e4dc0 tx=0x7fcb400edcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3fe6cdfd95d8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:08.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb5d741640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb440b6680 0x7fcb440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.690+0000 7fcb55ffb640 1 -- 172.21.15.31:0/3971418316 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7fcb400e7680 con 0x7fcb58102ac0 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.691+0000 7fcb4effd640 1 -- 172.21.15.31:0/3296054380 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400a9b10 con 0x556ef65cf1b0 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.691+0000 7fcb4effd640 1 -- 172.21.15.31:0/3296054380 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb400acfd0 con 0x556ef65cf1b0 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.691+0000 7fcb4effd640 1 -- 172.21.15.31:0/3296054380 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400acc30 con 0x556ef65cf1b0 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 -- 172.21.15.31:0/3296054380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65cf1b0 msgr2=0x556ef699f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65cf1b0 0x556ef699f800 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcb400e4dc0 tx=0x7fcb400edcd0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb5d741640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb440b6680 0x7fcb440b8b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcb50006010 tx=0x7fcb500058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 -- 172.21.15.31:0/3296054380 shutdown_connections 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef699fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65cf1b0 0x556ef699f800 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 --2- 172.21.15.31:0/3296054380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef699f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 -- 172.21.15.31:0/3296054380 >> 172.21.15.31:0/3296054380 conn(0x556ef65ec460 msgr2=0x556ef699d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 -- 172.21.15.31:0/3296054380 shutdown_connections 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 -- 172.21.15.31:0/3296054380 wait complete. 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 -- start start 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.692+0000 7fcb65718740 1 Processor -- start 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69aa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65d7390 0x556ef69aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef69b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556ef6766700 con 0x556ef65d7390 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556ef68a0160 con 0x556ef65cf1b0 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556ef69adaf0 con 0x556ef65f8070 2024-09-21T06:19:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb65718740 10 client.0 _ll_register_callbacks cb 0x556ef68b99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58992/0 (socket says 172.21.15.31:58992) 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2652328679 learned_addr learned my addr 172.21.15.31:0/2652328679 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5d741640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef69b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb57fff640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65d7390 0x556ef69aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2652328679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 msgr2=0x556ef69b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef69b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2652328679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65d7390 msgr2=0x556ef69aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65d7390 0x556ef69aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.693+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2652328679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef68cdc90 con 0x556ef65cf1b0 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb57fff640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65d7390 0x556ef69aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69aa970 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb400e4e70 tx=0x7fcb400096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb10a66037f69888 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb37fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb37fff640 1 -- 172.21.15.31:0/2652328679 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4001c040 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb400acfd0 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb37fff640 1 -- 172.21.15.31:0/2652328679 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7fcb400054f0 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb37fff640 1 -- 172.21.15.31:0/2652328679 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcb400ace10 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb65718740 10 client.26837 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556ef69b97e0 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.694+0000 7fcb37fff640 1 -- 172.21.15.31:0/2652328679 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb400acfd0 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb37fff640 1 -- 172.21.15.31:0/2652328679 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fcb400c3070 con 0x556ef65cf1b0 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb65718740 10 client.26837 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb65718740 10 client.26837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb65718740 20 client.26837 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb65718740 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef65b69f0 0x556ef69bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb65718740 4 client.26837 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ef6917df0 con 0x556ef65b69f0 2024-09-21T06:19:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb57fff640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef65b69f0 0x556ef69bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb37fff640 10 client.26837 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.695+0000 7fcb57fff640 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef65b69f0 0x556ef69bcb40 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97916d333b1f13 server_cookie=6fe0f107f25e7f51 in_seq=0 out_seq=0 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb37fff640 1 -- 172.21.15.31:0/2652328679 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ef6917df0 con 0x556ef65b69f0 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb37fff640 10 client.26837 handle_command_reply: tid=0 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 client.26837 shutdown 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 msgr2=0x556ef69aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69aa970 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcb400e4e70 tx=0x7fcb400096e0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 shutdown_connections 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef65b69f0 0x556ef69bcb40 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65f8070 0x556ef69b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65d7390 0x556ef69aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb37fff640 0 client.26837 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 --2- 172.21.15.31:0/2652328679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69aa970 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 >> 172.21.15.31:0/2652328679 conn(0x556ef65ec460 msgr2=0x556ef69a2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 shutdown_connections 2024-09-21T06:19:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.696+0000 7fcb65718740 1 -- 172.21.15.31:0/2652328679 wait complete. 2024-09-21T06:19:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.697+0000 7fcb65718740 20 client.26837 trim_cache size 0 max 16384 2024-09-21T06:19:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 Processor -- start 2024-09-21T06:19:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 -- start start 2024-09-21T06:19:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65cf1b0 0x556ef69aae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef69a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef69a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581afa80 con 0x556ef65f8070 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b22e0 con 0x556ef65d7390 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb65718740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b00d0 con 0x556ef65cf1b0 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb5d741640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef69a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.710+0000 7fcb5d741640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef69a80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56198/0 (socket says 172.21.15.31:56198) 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 -- 172.21.15.31:0/1613325700 learned_addr learned my addr 172.21.15.31:0/1613325700 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65cf1b0 0x556ef69aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb57fff640 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef69a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 -- 172.21.15.31:0/1613325700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65cf1b0 msgr2=0x556ef69aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65cf1b0 0x556ef69aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 -- 172.21.15.31:0/1613325700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 msgr2=0x556ef69a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef69a7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 -- 172.21.15.31:0/1613325700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef69985d0 con 0x556ef65f8070 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb57fff640 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef69a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb5d741640 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef69a80a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb50033000 tx=0x7fcb5003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e31899093dcc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb4e7fc640 1 -- 172.21.15.31:0/1613325700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50035040 con 0x556ef65f8070 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb4e7fc640 1 -- 172.21.15.31:0/1613325700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5004e070 con 0x556ef65f8070 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.711+0000 7fcb4e7fc640 1 -- 172.21.15.31:0/1613325700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50004c00 con 0x556ef65f8070 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 -- 172.21.15.31:0/1613325700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 msgr2=0x556ef69a80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef69a80a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcb50033000 tx=0x7fcb5003f000 comp rx=0 tx=0).stop 2024-09-21T06:19:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 -- 172.21.15.31:0/1613325700 shutdown_connections 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef69a80a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65d7390 0x556ef69a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 --2- 172.21.15.31:0/1613325700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65cf1b0 0x556ef69aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 -- 172.21.15.31:0/1613325700 >> 172.21.15.31:0/1613325700 conn(0x556ef65ec460 msgr2=0x556ef69aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 -- 172.21.15.31:0/1613325700 shutdown_connections 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.712+0000 7fcb65718740 1 -- 172.21.15.31:0/1613325700 wait complete. 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 -- start start 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 Processor -- start 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69bdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65d7390 0x556ef69be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef6a262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b07a0 con 0x556ef65f8070 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b1c10 con 0x556ef65cf1b0 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcb581b0e70 con 0x556ef65d7390 2024-09-21T06:19:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb65718740 10 client.0 _ll_register_callbacks cb 0x7fcb40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb5cf40640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69bdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb5d741640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef6a262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65d7390 0x556ef69be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb5cf40640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69bdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59022/0 (socket says 172.21.15.31:59022) 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.713+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2190320984 learned_addr learned my addr 172.21.15.31:0/2190320984 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2190320984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65d7390 msgr2=0x556ef69be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65d7390 0x556ef69be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2190320984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 msgr2=0x556ef6a262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef6a262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb5cf40640 1 -- 172.21.15.31:0/2190320984 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef69beb30 con 0x556ef65cf1b0 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb5cf40640 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69bdde0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b6fe0 tx=0x7fcb40009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a63c8a890866de4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb37fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb37fff640 1 -- 172.21.15.31:0/2190320984 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400c8060 con 0x556ef65cf1b0 2024-09-21T06:19:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb37fff640 1 -- 172.21.15.31:0/2190320984 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7fcb400ac720 con 0x556ef65cf1b0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb5004e070 con 0x556ef65cf1b0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb37fff640 1 -- 172.21.15.31:0/2190320984 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcb400cf050 con 0x556ef65cf1b0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb65718740 10 client.26843 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.714+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556ef6a277c0 con 0x556ef65cf1b0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb37fff640 1 -- 172.21.15.31:0/2190320984 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5004e070 con 0x556ef65cf1b0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb37fff640 1 -- 172.21.15.31:0/2190320984 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fcb400c3040 con 0x556ef65cf1b0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb65718740 10 client.26843 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb65718740 10 client.26843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb65718740 20 client.26843 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb65718740 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef658bda0 0x556ef6a29d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb65718740 4 client.26843 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ef69138a0 con 0x556ef658bda0 2024-09-21T06:19:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb57fff640 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef658bda0 0x556ef6a29d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb37fff640 10 client.26843 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.715+0000 7fcb57fff640 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef658bda0 0x556ef6a29d90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32149bbe85a8373c server_cookie=c5265d5e1ee171e3 in_seq=0 out_seq=0 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb37fff640 1 -- 172.21.15.31:0/2190320984 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ef69138a0 con 0x556ef658bda0 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb37fff640 10 client.26843 handle_command_reply: tid=0 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 client.26843 shutdown 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 msgr2=0x556ef69bdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69bdde0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b6fe0 tx=0x7fcb40009f60 comp rx=0 tx=0).stop 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 shutdown_connections 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556ef658bda0 0x556ef6a29d90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ef65f8070 0x556ef6a262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556ef65d7390 0x556ef69be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb37fff640 0 client.26843 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 --2- 172.21.15.31:0/2190320984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ef65cf1b0 0x556ef69bdde0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 >> 172.21.15.31:0/2190320984 conn(0x556ef65ec460 msgr2=0x556ef69a8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 shutdown_connections 2024-09-21T06:19:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 1 -- 172.21.15.31:0/2190320984 wait complete. 2024-09-21T06:19:08.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.716+0000 7fcb65718740 20 client.26843 trim_cache size 0 max 16384 2024-09-21T06:19:08.723 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb440b6680 msgr2=0x7fcb440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb440b6680 0x7fcb440b8b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcb50006010 tx=0x7fcb500058d0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58102ac0 msgr2=0x7fcb5819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58102ac0 0x7fcb5819c710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9f0 tx=0x7fcb4000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 shutdown_connections 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb58108d00 0x7fcb581a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcb440b6680 0x7fcb440b8b90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb581083b0 0x7fcb5819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 --2- 172.21.15.31:0/3971418316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58102ac0 0x7fcb5819c710 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 >> 172.21.15.31:0/3971418316 conn(0x7fcb58079720 msgr2=0x7fcb5806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 shutdown_connections 2024-09-21T06:19:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:08.717+0000 7fcb5f1a4640 1 -- 172.21.15.31:0/3971418316 wait complete. 2024-09-21T06:19:09.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:08 smithi110 ceph-mon[34550]: pgmap v548: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.3 MiB/s wr, 511 op/s 2024-09-21T06:19:09.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:08 smithi110 ceph-mon[34550]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:19:09.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:08 smithi107 ceph-mon[34380]: pgmap v548: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.3 MiB/s wr, 511 op/s 2024-09-21T06:19:09.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:08 smithi107 ceph-mon[34380]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:19:09.324 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:08 smithi031 ceph-mon[31237]: pgmap v548: 97 pgs: 97 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.3 MiB/s wr, 511 op/s 2024-09-21T06:19:09.324 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:08 smithi031 ceph-mon[31237]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:19:10.206 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 450.0 s, 105.6 tps, lat 501.183 ms stddev 317.318 2024-09-21T06:19:10.686 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:19:10.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:19:10.971 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:11.093 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:10 smithi031 ceph-mon[31237]: pgmap v550: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.4 MiB/s wr, 545 op/s 2024-09-21T06:19:11.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:10 smithi110 ceph-mon[34550]: pgmap v550: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.4 MiB/s wr, 545 op/s 2024-09-21T06:19:11.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:10 smithi107 ceph-mon[34380]: pgmap v550: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 5.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.4 MiB/s wr, 545 op/s 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 -- 172.21.15.31:0/2166086794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5401083d0 msgr2=0x7fc5401087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 --2- 172.21.15.31:0/2166086794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5401083d0 0x7fc5401087e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009900 tx=0x7fc52c02a410 comp rx=0 tx=0).stop 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 -- 172.21.15.31:0/2166086794 shutdown_connections 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 --2- 172.21.15.31:0/2166086794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc540108d20 0x7fc54010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 --2- 172.21.15.31:0/2166086794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5401083d0 0x7fc5401087e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 --2- 172.21.15.31:0/2166086794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 0x7fc540102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 -- 172.21.15.31:0/2166086794 >> 172.21.15.31:0/2166086794 conn(0x7fc540079790 msgr2=0x7fc540079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:11.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 -- 172.21.15.31:0/2166086794 shutdown_connections 2024-09-21T06:19:11.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.461+0000 7fc546bb1640 1 -- 172.21.15.31:0/2166086794 wait complete. 2024-09-21T06:19:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 Processor -- start 2024-09-21T06:19:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 -- start start 2024-09-21T06:19:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 0x7fc5401993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5401083d0 0x7fc540199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 0x7fc54019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401129e0 con 0x7fc540108d20 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc540112860 con 0x7fc5401083d0 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc546bb1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc540112b60 con 0x7fc540102ae0 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc54494d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 0x7fc5401993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc54514e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 0x7fc54019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc54514e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 0x7fc54019ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53278/0 (socket says 172.21.15.31:53278) 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5401083d0 0x7fc540199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc54494d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 0x7fc5401993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44584/0 (socket says 172.21.15.31:44584) 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.462+0000 7fc54514e640 1 -- 172.21.15.31:0/701633040 learned_addr learned my addr 172.21.15.31:0/701633040 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53ffff640 1 -- 172.21.15.31:0/701633040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 msgr2=0x7fc5401993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53ffff640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 0x7fc5401993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53ffff640 1 -- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 msgr2=0x7fc54019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53ffff640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 0x7fc54019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53ffff640 1 -- 172.21.15.31:0/701633040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54019f320 con 0x7fc5401083d0 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc54514e640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 0x7fc54019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53ffff640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5401083d0 0x7fc540199950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02a930 tx=0x7fc52c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53dffb640 1 -- 172.21.15.31:0/701633040 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c03f070 con 0x7fc5401083d0 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53dffb640 1 -- 172.21.15.31:0/701633040 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c004490 con 0x7fc5401083d0 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54019f5b0 con 0x7fc5401083d0 2024-09-21T06:19:11.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.463+0000 7fc53dffb640 1 -- 172.21.15.31:0/701633040 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c035070 con 0x7fc5401083d0 2024-09-21T06:19:11.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.464+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54006bb20 con 0x7fc5401083d0 2024-09-21T06:19:11.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.466+0000 7fc54d125740 1 Processor -- start 2024-09-21T06:19:11.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.466+0000 7fc54d125740 1 -- start start 2024-09-21T06:19:11.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.466+0000 7fc54d125740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 0x5637a849a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54d125740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a849a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54d125740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54d125740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637a8410e60 con 0x5637a80f3350 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54d125740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5637a8400290 con 0x5637a80ca490 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54d125740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637a84040a0 con 0x5637a80d2670 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 0x5637a849a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 0x5637a849a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53284/0 (socket says 172.21.15.31:53284) 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 -- 172.21.15.31:0/3106941934 learned_addr learned my addr 172.21.15.31:0/3106941934 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54494d640 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a849a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc53dffb640 1 -- 172.21.15.31:0/701633040 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc52c03b460 con 0x7fc5401083d0 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 -- 172.21.15.31:0/3106941934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 msgr2=0x5637a849aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 -- 172.21.15.31:0/3106941934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 msgr2=0x5637a849a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a849a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 -- 172.21.15.31:0/3106941934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a815e930 con 0x5637a80f3350 2024-09-21T06:19:11.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54494d640 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a849a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:11.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.467+0000 7fc54514e640 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 0x5637a849a430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e970 tx=0x7fc538009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fd9dd6ed14fe62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.468+0000 7fc52affd640 1 -- 172.21.15.31:0/3106941934 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800bae0 con 0x5637a80f3350 2024-09-21T06:19:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.468+0000 7fc52affd640 1 -- 172.21.15.31:0/3106941934 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53800bc80 con 0x5637a80f3350 2024-09-21T06:19:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.468+0000 7fc52affd640 1 -- 172.21.15.31:0/3106941934 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380175c0 con 0x5637a80f3350 2024-09-21T06:19:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.468+0000 7fc53dffb640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc5300b6700 0x7fc5300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.468+0000 7fc54494d640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc5300b6700 0x7fc5300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.468+0000 7fc53dffb640 1 -- 172.21.15.31:0/701633040 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7fc52c107b70 con 0x7fc5401083d0 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.469+0000 7fc54d125740 1 -- 172.21.15.31:0/3106941934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 msgr2=0x5637a849a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.469+0000 7fc54494d640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc5300b6700 0x7fc5300b8c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5637a84990b0 tx=0x7fc534005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.469+0000 7fc54d125740 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 0x5637a849a430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc53800e970 tx=0x7fc538009500 comp rx=0 tx=0).stop 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 -- 172.21.15.31:0/3106941934 shutdown_connections 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a849a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 --2- 172.21.15.31:0/3106941934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80f3350 0x5637a849a430 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 -- 172.21.15.31:0/3106941934 >> 172.21.15.31:0/3106941934 conn(0x5637a80e7740 msgr2=0x5637a8498320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 -- 172.21.15.31:0/3106941934 shutdown_connections 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 -- 172.21.15.31:0/3106941934 wait complete. 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 -- start start 2024-09-21T06:19:11.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.470+0000 7fc54d125740 1 Processor -- start 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a84a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80d2670 0x5637a84a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80f3350 0x5637a84b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5637a8261890 con 0x5637a80ca490 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5637a839ad40 con 0x5637a80d2670 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637a84a8c10 con 0x5637a80f3350 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54d125740 10 client.0 _ll_register_callbacks cb 0x5637a83fec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:11.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a84a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a84a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53294/0 (socket says 172.21.15.31:53294) 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 -- 172.21.15.31:0/3848489148 learned_addr learned my addr 172.21.15.31:0/3848489148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80d2670 0x5637a84a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54514e640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80f3350 0x5637a84b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 -- 172.21.15.31:0/3848489148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80f3350 msgr2=0x5637a84b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80f3350 0x5637a84b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 -- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80d2670 msgr2=0x5637a84a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80d2670 0x5637a84a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.471+0000 7fc54494d640 1 -- 172.21.15.31:0/3848489148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a84b24f0 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc54494d640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a84a6070 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc5340045f0 tx=0x7fc534004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18acd1bb1147ac04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc51ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc51ffff640 1 -- 172.21.15.31:0/3848489148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53401f640 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc51ffff640 1 -- 172.21.15.31:0/3848489148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7fc53401f7e0 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc53800bc80 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc51ffff640 1 -- 172.21.15.31:0/3848489148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc53401f9e0 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc54d125740 10 client.17103 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5637a84b4e80 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.472+0000 7fc51ffff640 1 -- 172.21.15.31:0/3848489148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53800bc80 con 0x5637a80ca490 2024-09-21T06:19:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc51ffff640 1 -- 172.21.15.31:0/3848489148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc534027070 con 0x5637a80ca490 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc54d125740 10 client.17103 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc54d125740 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc54d125740 20 client.17103 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc54d125740 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80b1cd0 0x5637a84b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc54d125740 4 client.17103 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a8412db0 con 0x5637a80b1cd0 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80b1cd0 0x5637a84b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc51ffff640 10 client.17103 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.473+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80b1cd0 0x5637a84b8520 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364a619d41748b04 server_cookie=9b005a04fc2cfefb in_seq=0 out_seq=0 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.474+0000 7fc51ffff640 1 -- 172.21.15.31:0/3848489148 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5637a8412db0 con 0x5637a80b1cd0 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.474+0000 7fc51ffff640 10 client.17103 handle_command_reply: tid=0 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.474+0000 7fc54d125740 1 client.17103 shutdown 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.474+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 msgr2=0x5637a84a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.474+0000 7fc54d125740 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a84a6070 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc5340045f0 tx=0x7fc534004730 comp rx=0 tx=0).stop 2024-09-21T06:19:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 shutdown_connections 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80b1cd0 0x5637a84b8520 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc51ffff640 0 client.17103 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80f3350 0x5637a84b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80d2670 0x5637a84a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 --2- 172.21.15.31:0/3848489148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a84a6070 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 >> 172.21.15.31:0/3848489148 conn(0x5637a80e7740 msgr2=0x5637a849d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 shutdown_connections 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 1 -- 172.21.15.31:0/3848489148 wait complete. 2024-09-21T06:19:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.475+0000 7fc54d125740 20 client.17103 trim_cache size 0 max 16384 2024-09-21T06:19:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 Processor -- start 2024-09-21T06:19:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 -- start start 2024-09-21T06:19:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80b1cd0 0x5637a84b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a849f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401ab5b0 con 0x5637a80ca490 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401ade10 con 0x5637a80b1cd0 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54d125740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401abc00 con 0x5637a80d2670 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a849f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a849f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53308/0 (socket says 172.21.15.31:53308) 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc53ffff640 1 -- 172.21.15.31:0/1286695524 learned_addr learned my addr 172.21.15.31:0/1286695524 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54514e640 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.500+0000 7fc54494d640 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80b1cd0 0x5637a84b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc53ffff640 1 -- 172.21.15.31:0/1286695524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 msgr2=0x5637a849fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc53ffff640 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc53ffff640 1 -- 172.21.15.31:0/1286695524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80b1cd0 msgr2=0x5637a84b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc53ffff640 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80b1cd0 0x5637a84b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc53ffff640 1 -- 172.21.15.31:0/1286695524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a849c720 con 0x5637a80ca490 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc53ffff640 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a849f510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc52c1052b0 tx=0x7fc52c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1df0ca20df45710 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc52a7fc640 1 -- 172.21.15.31:0/1286695524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c048040 con 0x5637a80ca490 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc52a7fc640 1 -- 172.21.15.31:0/1286695524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c0d2430 con 0x5637a80ca490 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.501+0000 7fc52a7fc640 1 -- 172.21.15.31:0/1286695524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0c9350 con 0x5637a80ca490 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 -- 172.21.15.31:0/1286695524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 msgr2=0x5637a849f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a849f510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc52c1052b0 tx=0x7fc52c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 -- 172.21.15.31:0/1286695524 shutdown_connections 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a849fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80ca490 0x5637a849f510 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 --2- 172.21.15.31:0/1286695524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80b1cd0 0x5637a84b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 -- 172.21.15.31:0/1286695524 >> 172.21.15.31:0/1286695524 conn(0x5637a80e7740 msgr2=0x5637a84a46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 -- 172.21.15.31:0/1286695524 shutdown_connections 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 -- 172.21.15.31:0/1286695524 wait complete. 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.502+0000 7fc54d125740 1 -- start start 2024-09-21T06:19:11.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 Processor -- start 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 0x5637a84ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a84ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a84a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401ac2d0 con 0x5637a80b1cd0 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401ad740 con 0x5637a80ca490 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5401ac9a0 con 0x5637a80d2670 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54d125740 10 client.0 _ll_register_callbacks cb 0x7fc5340044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:11.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54494d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 0x5637a84ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54514e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a84a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54494d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 0x5637a84ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53310/0 (socket says 172.21.15.31:53310) 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc54494d640 1 -- 172.21.15.31:0/3532282634 learned_addr learned my addr 172.21.15.31:0/3532282634 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.503+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a84ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54494d640 1 -- 172.21.15.31:0/3532282634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 msgr2=0x5637a84a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54494d640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a84a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54494d640 1 -- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 msgr2=0x5637a84ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54494d640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a84ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54494d640 1 -- 172.21.15.31:0/3532282634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a84a9360 con 0x5637a80b1cd0 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a84ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54514e640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a84a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54494d640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 0x5637a84ade70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5637a84a86c0 tx=0x7fc534032b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde070565c60202c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc51ffff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc51ffff640 1 -- 172.21.15.31:0/3532282634 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53403e060 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc51ffff640 1 -- 172.21.15.31:0/3532282634 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7fc534029440 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc52c0d2430 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc51ffff640 1 -- 172.21.15.31:0/3532282634 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc534003e50 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.504+0000 7fc54d125740 10 client.17109 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5637a8523570 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc51ffff640 1 -- 172.21.15.31:0/3532282634 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c0d2430 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc51ffff640 1 -- 172.21.15.31:0/3532282634 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc534027040 con 0x5637a80b1cd0 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 10 client.17109 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:11.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 20 client.17109 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80f3350 0x5637a8525d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 4 client.17109 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637a840e860 con 0x5637a80f3350 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.505+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80f3350 0x5637a8525d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.506+0000 7fc51ffff640 10 client.17109 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.506+0000 7fc53ffff640 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80f3350 0x5637a8525d90 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51120f51ad4b355 server_cookie=a0fc368698d148b3 in_seq=0 out_seq=0 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.507+0000 7fc51ffff640 1 -- 172.21.15.31:0/3532282634 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637a840e860 con 0x5637a80f3350 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.507+0000 7fc51ffff640 10 client.17109 handle_command_reply: tid=0 2024-09-21T06:19:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.507+0000 7fc54d125740 1 client.17109 shutdown 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 msgr2=0x5637a84ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 0x5637a84ade70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x5637a84a86c0 tx=0x7fc534032b40 comp rx=0 tx=0).stop 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 shutdown_connections 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5637a80f3350 0x5637a8525d90 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637a80d2670 0x5637a84a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5637a80ca490 0x5637a84ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 --2- 172.21.15.31:0/3532282634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5637a80b1cd0 0x5637a84ade70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 >> 172.21.15.31:0/3532282634 conn(0x5637a80e7740 msgr2=0x5637a849d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc51ffff640 0 client.17109 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 shutdown_connections 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 1 -- 172.21.15.31:0/3532282634 wait complete. 2024-09-21T06:19:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.508+0000 7fc54d125740 20 client.17109 trim_cache size 0 max 16384 2024-09-21T06:19:11.516 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:19:11.516 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:19:11.516 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "ababb4c4-14ff-4c13-9f92-e2c95075bbb2", 2024-09-21T06:19:11.516 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:19:11.516 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:19:11.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc5300b6700 msgr2=0x7fc5300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc5300b6700 0x7fc5300b8c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5637a84990b0 tx=0x7fc534005f90 comp rx=0 tx=0).stop 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5401083d0 msgr2=0x7fc540199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5401083d0 0x7fc540199950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc52c02a930 tx=0x7fc52c031a20 comp rx=0 tx=0).stop 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 shutdown_connections 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc5300b6700 0x7fc5300b8c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc540108d20 0x7fc54019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5401083d0 0x7fc540199950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 --2- 172.21.15.31:0/701633040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc540102ae0 0x7fc5401993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 >> 172.21.15.31:0/701633040 conn(0x7fc540079790 msgr2=0x7fc540106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:11.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 shutdown_connections 2024-09-21T06:19:11.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:11.509+0000 7fc546bb1640 1 -- 172.21.15.31:0/701633040 wait complete. 2024-09-21T06:19:12.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:19:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:13.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:12 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:19:13.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:12 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:13.123 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:12 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:19:13.124 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:12 smithi031 ceph-mon[31237]: pgmap v551: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-09-21T06:19:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:12 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:19:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:12 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:12 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:19:13.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:12 smithi110 ceph-mon[34550]: pgmap v551: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-09-21T06:19:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:12 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:19:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:12 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:12 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:19:13.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:12 smithi107 ceph-mon[34380]: pgmap v551: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-09-21T06:19:13.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.318+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2425564497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7011cba0 msgr2=0x7f2e7011efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.318+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2425564497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7011cba0 0x7f2e7011efc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009900 tx=0x7f2e6402a440 comp rx=0 tx=0).stop 2024-09-21T06:19:13.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.323+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2425564497 shutdown_connections 2024-09-21T06:19:13.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.323+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2425564497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7011cba0 0x7f2e7011efc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.323+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2425564497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e7010dbf0 0x7f2e7010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.323+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2425564497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e70111cd0 0x7f2e7010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.323+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2425564497 >> 172.21.15.31:0/2425564497 conn(0x7f2e7006e9f0 msgr2=0x7f2e7006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:13.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.328+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2425564497 shutdown_connections 2024-09-21T06:19:13.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.328+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2425564497 wait complete. 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.329+0000 7f2e74a2e640 1 Processor -- start 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 -- start start 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7010dbf0 0x7f2e7011c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70111cd0 0x7f2e70116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e7011cba0 0x7f2e70116d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70112c60 con 0x7f2e7010dbf0 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70112ae0 con 0x7f2e70111cd0 2024-09-21T06:19:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.330+0000 7f2e74a2e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70112de0 con 0x7f2e7011cba0 2024-09-21T06:19:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.333+0000 7f2e6f59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e7011cba0 0x7f2e70116d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.333+0000 7f2e6e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70111cd0 0x7f2e70116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.333+0000 7f2e6f59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e7011cba0 0x7f2e70116d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44656/0 (socket says 172.21.15.31:44656) 2024-09-21T06:19:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.333+0000 7f2e6f59e640 1 -- 172.21.15.31:0/2037661686 learned_addr learned my addr 172.21.15.31:0/2037661686 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.333+0000 7f2e6e59c640 1 -- 172.21.15.31:0/2037661686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e7011cba0 msgr2=0x7f2e70116d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.333+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e7011cba0 0x7f2e70116d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e6e59c640 1 -- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7010dbf0 msgr2=0x7f2e7011c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7010dbf0 0x7f2e7011c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e6e59c640 1 -- 172.21.15.31:0/2037661686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e701175b0 con 0x7f2e70111cd0 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70111cd0 0x7f2e70116780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2e7006c910 tx=0x7f2e680075f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e57fff640 1 -- 172.21.15.31:0/2037661686 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68004b20 con 0x7f2e70111cd0 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e57fff640 1 -- 172.21.15.31:0/2037661686 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e68004cc0 con 0x7f2e70111cd0 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.334+0000 7f2e57fff640 1 -- 172.21.15.31:0/2037661686 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e680059d0 con 0x7f2e70111cd0 2024-09-21T06:19:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.335+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7007b740 con 0x7f2e70111cd0 2024-09-21T06:19:13.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.335+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7007bc30 con 0x7f2e70111cd0 2024-09-21T06:19:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.339+0000 7f2e7b7a3740 1 Processor -- start 2024-09-21T06:19:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.339+0000 7f2e7b7a3740 1 -- start start 2024-09-21T06:19:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6841e50 con 0x55b6c6524070 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6831280 con 0x55b6c64fb1b0 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6835090 con 0x55b6c6503390 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e6ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e6ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41710/0 (socket says 172.21.15.31:41710) 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.340+0000 7f2e6ed9d640 1 -- 172.21.15.31:0/3505070065 learned_addr learned my addr 172.21.15.31:0/3505070065 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6f59e640 1 -- 172.21.15.31:0/3505070065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 msgr2=0x55b6c68cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6f59e640 1 -- 172.21.15.31:0/3505070065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 msgr2=0x55b6c68cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.341+0000 7f2e6f59e640 1 -- 172.21.15.31:0/3505070065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c658f650 con 0x55b6c6524070 2024-09-21T06:19:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.342+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cb810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009850 tx=0x7f2e640040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485da275d534674a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.343+0000 7f2e557fa640 1 -- 172.21.15.31:0/3505070065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64035030 con 0x55b6c6524070 2024-09-21T06:19:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.343+0000 7f2e557fa640 1 -- 172.21.15.31:0/3505070065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e64004470 con 0x55b6c6524070 2024-09-21T06:19:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.343+0000 7f2e557fa640 1 -- 172.21.15.31:0/3505070065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64004e60 con 0x55b6c6524070 2024-09-21T06:19:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.348+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3505070065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 msgr2=0x55b6c68cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.348+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cb810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009850 tx=0x7f2e640040d0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.357+0000 7f2e57fff640 1 -- 172.21.15.31:0/2037661686 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2e68005060 con 0x7f2e70111cd0 2024-09-21T06:19:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.359+0000 7f2e57fff640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e580b67b0 0x7f2e580b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.359+0000 7f2e57fff640 1 -- 172.21.15.31:0/2037661686 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7f2e680e7c40 con 0x7f2e70111cd0 2024-09-21T06:19:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.361+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3505070065 shutdown_connections 2024-09-21T06:19:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.361+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.361+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.361+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3505070065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cb810 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.361+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3505070065 >> 172.21.15.31:0/3505070065 conn(0x55b6c6518460 msgr2=0x55b6c68c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.362+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e580b67b0 0x7f2e580b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.366+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3505070065 shutdown_connections 2024-09-21T06:19:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.366+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3505070065 wait complete. 2024-09-21T06:19:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.366+0000 7f2e7b7a3740 1 -- start start 2024-09-21T06:19:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 Processor -- start 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c64fb1b0 0x55b6c68d6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68d7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c6524070 0x55b6c68e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b6c6692700 con 0x55b6c64fb1b0 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b6c67cc110 con 0x55b6c6524070 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b6c68d9f90 con 0x55b6c6503390 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.367+0000 7f2e7b7a3740 10 client.0 _ll_register_callbacks cb 0x55b6c67e5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68d7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68d7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44688/0 (socket says 172.21.15.31:44688) 2024-09-21T06:19:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 -- 172.21.15.31:0/3154293815 learned_addr learned my addr 172.21.15.31:0/3154293815 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6ed9d640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c64fb1b0 0x55b6c68d6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c6524070 0x55b6c68e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 -- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c6524070 msgr2=0x55b6c68e32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c6524070 0x55b6c68e32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 -- 172.21.15.31:0/3154293815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c64fb1b0 msgr2=0x55b6c68d6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.368+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c64fb1b0 0x55b6c68d6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.369+0000 7f2e6e59c640 1 -- 172.21.15.31:0/3154293815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c67f9c40 con 0x55b6c6503390 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.369+0000 7f2e52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.369+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68d7480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ad9b0 tx=0x7f2e680ad190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b715fe1c2d66d2bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.369+0000 7f2e52ff5640 1 -- 172.21.15.31:0/3154293815 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6801d070 con 0x55b6c6503390 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.369+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e64004470 con 0x55b6c6503390 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.369+0000 7f2e52ff5640 1 -- 172.21.15.31:0/3154293815 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7f2e680a9580 con 0x55b6c6503390 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.370+0000 7f2e52ff5640 1 -- 172.21.15.31:0/3154293815 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e6801d070 con 0x55b6c6503390 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.370+0000 7f2e7b7a3740 10 client.26923 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.370+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b6c68e5d00 con 0x55b6c6503390 2024-09-21T06:19:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.370+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e580b67b0 0x7f2e580b8cc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009340 tx=0x7f2e6402a440 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.371+0000 7f2e52ff5640 1 -- 172.21.15.31:0/3154293815 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e64004470 con 0x55b6c6503390 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e52ff5640 1 -- 172.21.15.31:0/3154293815 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2e680b2170 con 0x55b6c6503390 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e7b7a3740 10 client.26923 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e7b7a3740 10 client.26923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e7b7a3740 20 client.26923 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64e29f0 0x55b6c68e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e7b7a3740 4 client.26923 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6c6843da0 con 0x55b6c64e29f0 2024-09-21T06:19:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.372+0000 7f2e6ed9d640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64e29f0 0x55b6c68e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.373+0000 7f2e52ff5640 10 client.26923 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.373+0000 7f2e6ed9d640 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64e29f0 0x55b6c68e96b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3471b895da54d674 server_cookie=1114f448a91009a1 in_seq=0 out_seq=0 2024-09-21T06:19:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.375+0000 7f2e52ff5640 1 -- 172.21.15.31:0/3154293815 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6c6843da0 con 0x55b6c64e29f0 2024-09-21T06:19:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.375+0000 7f2e52ff5640 10 client.26923 handle_command_reply: tid=0 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.375+0000 7f2e7b7a3740 1 client.26923 shutdown 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.375+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 msgr2=0x55b6c68d7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.375+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68d7480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ad9b0 tx=0x7f2e680ad190 comp rx=0 tx=0).stop 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 shutdown_connections 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64e29f0 0x55b6c68e96b0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e52ff5640 0 client.26923 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c6524070 0x55b6c68e32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68d7480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/3154293815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c64fb1b0 0x55b6c68d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 >> 172.21.15.31:0/3154293815 conn(0x55b6c6518460 msgr2=0x55b6c68ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 shutdown_connections 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/3154293815 wait complete. 2024-09-21T06:19:13.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.376+0000 7f2e7b7a3740 20 client.26923 trim_cache size 0 max 16384 2024-09-21T06:19:13.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.417+0000 7f2e7b7a3740 1 Processor -- start 2024-09-21T06:19:13.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.417+0000 7f2e7b7a3740 1 -- start start 2024-09-21T06:19:13.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.418+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.418+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68e31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.418+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.419+0000 7f2e6e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68e31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.419+0000 7f2e6e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68e31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44698/0 (socket says 172.21.15.31:44698) 2024-09-21T06:19:13.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.419+0000 7f2e6e59c640 1 -- 172.21.15.31:0/100493009 learned_addr learned my addr 172.21.15.31:0/100493009 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:13.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.419+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70082a50 con 0x55b6c6524070 2024-09-21T06:19:13.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.419+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70085330 con 0x55b6c64fb1b0 2024-09-21T06:19:13.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.420+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70083120 con 0x55b6c6503390 2024-09-21T06:19:13.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.420+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.421+0000 7f2e6f59e640 1 -- 172.21.15.31:0/100493009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 msgr2=0x55b6c68e31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.421+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68e31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.421+0000 7f2e6f59e640 1 -- 172.21.15.31:0/100493009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 msgr2=0x55b6c68c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.421+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.421+0000 7f2e6f59e640 1 -- 172.21.15.31:0/100493009 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c68cdb00 con 0x55b6c6524070 2024-09-21T06:19:13.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.421+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cbd40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2e64004b50 tx=0x7f2e64005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6260bf31f209bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.422+0000 7f2e54ff9640 1 -- 172.21.15.31:0/100493009 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64035040 con 0x55b6c6524070 2024-09-21T06:19:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.422+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68e31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.422+0000 7f2e54ff9640 1 -- 172.21.15.31:0/100493009 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6405e070 con 0x55b6c6524070 2024-09-21T06:19:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.424+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 msgr2=0x55b6c68cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.424+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cbd40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2e64004b50 tx=0x7f2e64005e70 comp rx=0 tx=0).stop 2024-09-21T06:19:13.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.427+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 shutdown_connections 2024-09-21T06:19:13.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.427+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6524070 0x55b6c68cbd40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.427+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6503390 0x55b6c68e31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.427+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/100493009 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.427+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 >> 172.21.15.31:0/100493009 conn(0x55b6c6518460 msgr2=0x55b6c68c8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:13.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.428+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 shutdown_connections 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.428+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/100493009 wait complete. 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.428+0000 7f2e7b7a3740 1 -- start start 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.428+0000 7f2e7b7a3740 1 Processor -- start 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6503390 0x55b6c68df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6524070 0x55b6c68d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e700837f0 con 0x55b6c6503390 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70084c60 con 0x55b6c64fb1b0 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70083ec0 con 0x55b6c6524070 2024-09-21T06:19:13.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.429+0000 7f2e7b7a3740 10 client.0 _ll_register_callbacks cb 0x7f2e680aa070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:13.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.430+0000 7f2e6ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.430+0000 7f2e6ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68df230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41732/0 (socket says 172.21.15.31:41732) 2024-09-21T06:19:13.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.430+0000 7f2e6ed9d640 1 -- 172.21.15.31:0/980313152 learned_addr learned my addr 172.21.15.31:0/980313152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:13.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.431+0000 7f2e6f59e640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6524070 0x55b6c68d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.431+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6503390 0x55b6c68df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.432+0000 7f2e6e59c640 1 -- 172.21.15.31:0/980313152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6524070 msgr2=0x55b6c68d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.432+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6524070 0x55b6c68d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.432+0000 7f2e6e59c640 1 -- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 msgr2=0x55b6c68df230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:13.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.432+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68df230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.432+0000 7f2e6e59c640 1 -- 172.21.15.31:0/980313152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c68dfd00 con 0x55b6c6503390 2024-09-21T06:19:13.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.433+0000 7f2e6e59c640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6503390 0x55b6c68df770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ad9b0 tx=0x7f2e68007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6fd3efa5c88b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:13.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.434+0000 7f2e52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:19:13.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.434+0000 7f2e52ff5640 1 -- 172.21.15.31:0/980313152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6801d070 con 0x55b6c6503390 2024-09-21T06:19:13.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.435+0000 7f2e52ff5640 1 -- 172.21.15.31:0/980313152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8543+0+0 (secure 0 0 0) 0x7f2e680ae3b0 con 0x55b6c6503390 2024-09-21T06:19:13.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.435+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6405e070 con 0x55b6c6503390 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.438+0000 7f2e52ff5640 1 -- 172.21.15.31:0/980313152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e68005560 con 0x55b6c6503390 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.438+0000 7f2e7b7a3740 10 client.17121 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.438+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b6c68ee1a0 con 0x55b6c6503390 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e52ff5640 1 -- 172.21.15.31:0/980313152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6405e070 con 0x55b6c6503390 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e52ff5640 1 -- 172.21.15.31:0/980313152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2e680ae820 con 0x55b6c6503390 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e7b7a3740 10 client.17121 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e7b7a3740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e7b7a3740 20 client.17121 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:13.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64b7da0 0x55b6c69570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:13.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e7b7a3740 4 client.17121 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:19:13.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.439+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6c683f850 con 0x55b6c64b7da0 2024-09-21T06:19:13.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.440+0000 7f2e6ed9d640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64b7da0 0x55b6c69570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.451+0000 7f2e6ed9d640 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64b7da0 0x55b6c69570d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd7ce77c30cded8 server_cookie=9bfb581934beef6a in_seq=0 out_seq=0 2024-09-21T06:19:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.451+0000 7f2e52ff5640 10 client.17121 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.451+0000 7f2e52ff5640 1 -- 172.21.15.31:0/980313152 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6250 (crc 0 0 0) 0x55b6c6843da0 con 0x55b6c64b7da0 2024-09-21T06:19:13.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.451+0000 7f2e52ff5640 10 client.17121 handle_command_reply: tid=0 2024-09-21T06:19:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.451+0000 7f2e7b7a3740 1 client.17121 shutdown 2024-09-21T06:19:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6503390 msgr2=0x55b6c68df770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6503390 0x55b6c68df770 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ad9b0 tx=0x7f2e68007f00 comp rx=0 tx=0).stop 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 shutdown_connections 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b6c64b7da0 0x55b6c69570d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b6c6524070 0x55b6c68d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b6c6503390 0x55b6c68df770 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 --2- 172.21.15.31:0/980313152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b6c64fb1b0 0x55b6c68df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 >> 172.21.15.31:0/980313152 conn(0x55b6c6518460 msgr2=0x55b6c68cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.452+0000 7f2e52ff5640 0 client.17121 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.453+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 shutdown_connections 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.453+0000 7f2e7b7a3740 1 -- 172.21.15.31:0/980313152 wait complete. 2024-09-21T06:19:13.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.453+0000 7f2e7b7a3740 20 client.17121 trim_cache size 0 max 16384 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "ababb4c4-14ff-4c13-9f92-e2c95075bbb2", 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:19:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:19:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:19:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:19:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e580b67b0 msgr2=0x7f2e580b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e580b67b0 0x7f2e580b8cc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009340 tx=0x7f2e6402a440 comp rx=0 tx=0).stop 2024-09-21T06:19:13.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70111cd0 msgr2=0x7f2e70116780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70111cd0 0x7f2e70116780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2e7006c910 tx=0x7f2e680075f0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 shutdown_connections 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e7011cba0 0x7f2e70116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70111cd0 0x7f2e70116780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e580b67b0 0x7f2e580b8cc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 --2- 172.21.15.31:0/2037661686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e7010dbf0 0x7f2e7011c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.454+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 >> 172.21.15.31:0/2037661686 conn(0x7f2e7006e9f0 msgr2=0x7f2e70110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.455+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 shutdown_connections 2024-09-21T06:19:13.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:13.455+0000 7f2e74a2e640 1 -- 172.21.15.31:0/2037661686 wait complete. 2024-09-21T06:19:14.220 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:13 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:14.248 INFO:tasks.cephfs.filesystem:scrub status for tag:ababb4c4-14ff-4c13-9f92-e2c95075bbb2 - {'path': '/', 'tag': 'ababb4c4-14ff-4c13-9f92-e2c95075bbb2', 'options': 'recursive,force'} 2024-09-21T06:19:14.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:13 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:14.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:13 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:15.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 455.0 s, 91.2 tps, lat 1147.825 ms stddev 1270.388 2024-09-21T06:19:15.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:14 smithi110 ceph-mon[34550]: pgmap v552: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-09-21T06:19:15.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:14 smithi107 ceph-mon[34380]: pgmap v552: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-09-21T06:19:15.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:14 smithi031 ceph-mon[31237]: pgmap v552: 97 pgs: 13 active+clean+snaptrim_wait, 9 active+clean+snaptrim, 75 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-09-21T06:19:17.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:16 smithi110 ceph-mon[34550]: pgmap v553: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 27 MiB/s wr, 1.17k op/s 2024-09-21T06:19:17.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:16 smithi107 ceph-mon[34380]: pgmap v553: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 27 MiB/s wr, 1.17k op/s 2024-09-21T06:19:17.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:16 smithi031 ceph-mon[31237]: pgmap v553: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 27 MiB/s wr, 1.17k op/s 2024-09-21T06:19:18.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:17 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:19:18.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:17 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:19:18.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:17 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:19:19.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:19 smithi031 ceph-mon[31237]: pgmap v554: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2024-09-21T06:19:19.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:19 smithi031 ceph-mon[31237]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:19:19.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:19 smithi110 ceph-mon[34550]: pgmap v554: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2024-09-21T06:19:19.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:19 smithi110 ceph-mon[34550]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:19:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:19 smithi107 ceph-mon[34380]: pgmap v554: 97 pgs: 97 active+clean; 4.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 31 MiB/s wr, 1.26k op/s 2024-09-21T06:19:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:19 smithi107 ceph-mon[34380]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:19:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 460.0 s, 1430.9 tps, lat 101.662 ms stddev 429.224 2024-09-21T06:19:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 -- 172.21.15.31:0/1071695156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac10a5a0 msgr2=0x7fa0ac10a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/1071695156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac10a9b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009900 tx=0x7fa09c02a410 comp rx=0 tx=0).stop 2024-09-21T06:19:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 -- 172.21.15.31:0/1071695156 shutdown_connections 2024-09-21T06:19:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/1071695156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac1165a0 0x7fa0ac118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/1071695156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0ac10aef0 0x7fa0ac115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/1071695156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac10a9b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 -- 172.21.15.31:0/1071695156 >> 172.21.15.31:0/1071695156 conn(0x7fa0ac070dd0 msgr2=0x7fa0ac0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 -- 172.21.15.31:0/1071695156 shutdown_connections 2024-09-21T06:19:20.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.661+0000 7fa0b36cc640 1 -- 172.21.15.31:0/1071695156 wait complete. 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.662+0000 7fa0b36cc640 1 Processor -- start 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.662+0000 7fa0b36cc640 1 -- start start 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b36cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac1a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b36cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0ac10aef0 0x7fa0ac1a3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b36cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac1165a0 0x7fa0ac1a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b36cc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac11c270 con 0x7fa0ac1165a0 2024-09-21T06:19:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac1a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b36cc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac11c0f0 con 0x7fa0ac10aef0 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b36cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac11c3f0 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac1a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:56182/0 (socket says 172.21.15.31:56182) 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 -- 172.21.15.31:0/3956733645 learned_addr learned my addr 172.21.15.31:0/3956733645 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac1165a0 0x7fa0ac1a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0ac10aef0 0x7fa0ac1a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 -- 172.21.15.31:0/3956733645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0ac10aef0 msgr2=0x7fa0ac1a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0ac10aef0 0x7fa0ac1a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 -- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac1165a0 msgr2=0x7fa0ac1a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac1165a0 0x7fa0ac1a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.663+0000 7fa0b1468640 1 -- 172.21.15.31:0/3956733645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac1a8ba0 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.664+0000 7fa0b1468640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac1a2c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005ab0 tx=0x7fa09c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.664+0000 7fa0aa7fc640 1 -- 172.21.15.31:0/3956733645 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c03f070 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.664+0000 7fa0aa7fc640 1 -- 172.21.15.31:0/3956733645 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09c031b60 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.664+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac1a8e30 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.664+0000 7fa0aa7fc640 1 -- 172.21.15.31:0/3956733645 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c035030 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.664+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac10bd60 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.667+0000 7fa0ba638740 1 Processor -- start 2024-09-21T06:19:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.667+0000 7fa0ba638740 1 -- start start 2024-09-21T06:19:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.667+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071aff79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.667+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071aff7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.667+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.668+0000 7fa0ba638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56071af6c6f0 con 0x56071ac521b0 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.668+0000 7fa0ba638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56071af5bb20 con 0x56071ac292f0 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.668+0000 7fa0ba638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56071af5f930 con 0x56071ac314d0 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.668+0000 7fa0aa7fc640 1 -- 172.21.15.31:0/3956733645 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa09c042070 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.668+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b1c69640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071aff79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:56194/0 (socket says 172.21.15.31:56194) 2024-09-21T06:19:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b0c67640 1 -- 172.21.15.31:0/2350846964 learned_addr learned my addr 172.21.15.31:0/2350846964 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b1c69640 1 -- 172.21.15.31:0/2350846964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 msgr2=0x56071aff8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b1c69640 1 -- 172.21.15.31:0/2350846964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 msgr2=0x56071aff7ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:19:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071aff7ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.669+0000 7fa0b1c69640 1 -- 172.21.15.31:0/2350846964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071af21530 con 0x56071ac521b0 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0aa7fc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0940b6700 0x7fa0940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071aff79b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009b80 tx=0x7fa0a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9775196eb53c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0b1468640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0940b6700 0x7fa0940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0937fe640 1 -- 172.21.15.31:0/2350846964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004030 con 0x56071ac521b0 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0937fe640 1 -- 172.21.15.31:0/2350846964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a0005020 con 0x56071ac521b0 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0937fe640 1 -- 172.21.15.31:0/2350846964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004ca0 con 0x56071ac521b0 2024-09-21T06:19:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.670+0000 7fa0aa7fc640 1 -- 172.21.15.31:0/3956733645 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fa09c107ea0 con 0x7fa0ac10a5a0 2024-09-21T06:19:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0b1468640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0940b6700 0x7fa0940b8c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56071aff6630 tx=0x7fa09c02a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 -- 172.21.15.31:0/2350846964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 msgr2=0x56071aff79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071aff79b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009b80 tx=0x7fa0a0011810 comp rx=0 tx=0).stop 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 -- 172.21.15.31:0/2350846964 shutdown_connections 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071aff7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2350846964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071aff79b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 -- 172.21.15.31:0/2350846964 >> 172.21.15.31:0/2350846964 conn(0x56071ac46300 msgr2=0x56071aff58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 -- 172.21.15.31:0/2350846964 shutdown_connections 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.672+0000 7fa0ba638740 1 -- 172.21.15.31:0/2350846964 wait complete. 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.673+0000 7fa0ba638740 1 -- start start 2024-09-21T06:19:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.673+0000 7fa0ba638740 1 Processor -- start 2024-09-21T06:19:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071b0036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56071ae93ab0 con 0x56071ac521b0 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56071af01730 con 0x56071ac292f0 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56071b006130 con 0x56071ac314d0 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0ba638740 10 client.0 _ll_register_callbacks cb 0x56071af556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071b0036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071b0036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:56200/0 (socket says 172.21.15.31:56200) 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 -- 172.21.15.31:0/2953216126 learned_addr learned my addr 172.21.15.31:0/2953216126 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 -- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 msgr2=0x56071b003170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 -- 172.21.15.31:0/2953216126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 msgr2=0x56071b00f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b0c67640 1 -- 172.21.15.31:0/2953216126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071acbd790 con 0x56071ac314d0 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.674+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071b0036b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4011ce0 tx=0x7fa0a400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789aac5302e60cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa090ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa090ff9640 1 -- 172.21.15.31:0/2953216126 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400e5c0 con 0x56071ac314d0 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa090ff9640 1 -- 172.21.15.31:0/2953216126 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fa0a400e760 con 0x56071ac314d0 2024-09-21T06:19:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a0005020 con 0x56071ac314d0 2024-09-21T06:19:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa090ff9640 1 -- 172.21.15.31:0/2953216126 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa0a400e9a0 con 0x56071ac314d0 2024-09-21T06:19:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa0ba638740 10 client.26938 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56071ad36620 con 0x56071ac314d0 2024-09-21T06:19:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.675+0000 7fa090ff9640 1 -- 172.21.15.31:0/2953216126 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a0005020 con 0x56071ac314d0 2024-09-21T06:19:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa090ff9640 1 -- 172.21.15.31:0/2953216126 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa0a4019070 con 0x56071ac314d0 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0ba638740 10 client.26938 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0ba638740 10 client.26938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0ba638740 20 client.26938 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071ac10d80 0x56071b0152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0ba638740 4 client.26938 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56071af6e640 con 0x56071ac10d80 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071ac10d80 0x56071b0152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071ac10d80 0x56071b0152f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69af31cfb3234921 server_cookie=69d882a263709f4a in_seq=0 out_seq=0 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.676+0000 7fa090ff9640 10 client.26938 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa090ff9640 1 -- 172.21.15.31:0/2953216126 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56071af6e640 con 0x56071ac10d80 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa090ff9640 10 client.26938 handle_command_reply: tid=0 2024-09-21T06:19:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 client.26938 shutdown 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 msgr2=0x56071b0036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071b0036b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4011ce0 tx=0x7fa0a400f790 comp rx=0 tx=0).stop 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 shutdown_connections 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071ac10d80 0x56071b0152f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071b0036b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 --2- 172.21.15.31:0/2953216126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa090ff9640 0 client.26938 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 >> 172.21.15.31:0/2953216126 conn(0x56071ac46300 msgr2=0x56071aff89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 shutdown_connections 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.677+0000 7fa0ba638740 1 -- 172.21.15.31:0/2953216126 wait complete. 2024-09-21T06:19:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.678+0000 7fa0ba638740 20 client.26938 trim_cache size 0 max 16384 2024-09-21T06:19:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 Processor -- start 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 -- start start 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1b8120 con 0x56071ac521b0 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1ba980 con 0x56071ac292f0 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.699+0000 7fa0ba638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1b8770 con 0x56071ac314d0 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b1c69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b1c69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:37998/0 (socket says 172.21.15.31:37998) 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b1c69640 1 -- 172.21.15.31:0/1903931526 learned_addr learned my addr 172.21.15.31:0/1903931526 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1903931526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 msgr2=0x56071b003650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1903931526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 msgr2=0x56071b00f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1903931526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071aff9cb0 con 0x56071ac314d0 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff90f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4023c80 tx=0x7fa0a4023f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e00bce1acdeb02b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa092ffd640 1 -- 172.21.15.31:0/1903931526 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400e5c0 con 0x56071ac314d0 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa092ffd640 1 -- 172.21.15.31:0/1903931526 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a400e760 con 0x56071ac314d0 2024-09-21T06:19:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.700+0000 7fa092ffd640 1 -- 172.21.15.31:0/1903931526 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4003bd0 con 0x56071ac314d0 2024-09-21T06:19:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 -- 172.21.15.31:0/1903931526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 msgr2=0x56071aff90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff90f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4023c80 tx=0x7fa0a4023f20 comp rx=0 tx=0).stop 2024-09-21T06:19:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 -- 172.21.15.31:0/1903931526 shutdown_connections 2024-09-21T06:19:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac521b0 0x56071b00f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac314d0 0x56071aff90f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1903931526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac292f0 0x56071b003650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 -- 172.21.15.31:0/1903931526 >> 172.21.15.31:0/1903931526 conn(0x56071ac46300 msgr2=0x56071aff89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 -- 172.21.15.31:0/1903931526 shutdown_connections 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.701+0000 7fa0ba638740 1 -- 172.21.15.31:0/1903931526 wait complete. 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 -- start start 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 Processor -- start 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac292f0 0x56071b006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 0x56071b006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac521b0 0x56071b07dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1b8e40 con 0x56071ac314d0 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1ba2b0 con 0x56071ac521b0 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac1b9510 con 0x56071ac292f0 2024-09-21T06:19:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.702+0000 7fa0ba638740 10 client.0 _ll_register_callbacks cb 0x7fa0a4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.703+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 0x56071b006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.703+0000 7fa0b0c67640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 0x56071b006cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46694/0 (socket says 172.21.15.31:46694) 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.703+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1206964781 learned_addr learned my addr 172.21.15.31:0/1206964781 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac292f0 0x56071b006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac521b0 0x56071b07dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1206964781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac292f0 msgr2=0x56071b006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac292f0 0x56071b006780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac521b0 msgr2=0x56071b07dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac521b0 0x56071b07dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.704+0000 7fa0b0c67640 1 -- 172.21.15.31:0/1206964781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071b07e1e0 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.705+0000 7fa0b0c67640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 0x56071b006cc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400ffd0 tx=0x7fa0a4011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ece11ed3abeccf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.705+0000 7fa090ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.705+0000 7fa090ff9640 1 -- 172.21.15.31:0/1206964781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400e5c0 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.705+0000 7fa090ff9640 1 -- 172.21.15.31:0/1206964781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fa0a4017030 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.705+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0a400e760 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa090ff9640 1 -- 172.21.15.31:0/1206964781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa0a4042050 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 10 client.17139 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56071b07f310 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa090ff9640 1 -- 172.21.15.31:0/1206964781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a400e760 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa090ff9640 1 -- 172.21.15.31:0/1206964781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fa0a4019070 con 0x56071ac314d0 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 10 client.17139 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 20 client.17139 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071abe61c0 0x56071b0819d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 4 client.17139 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.706+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56071af6a0f0 con 0x56071abe61c0 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.707+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071abe61c0 0x56071b0819d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.707+0000 7fa090ff9640 10 client.17139 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.707+0000 7fa0b1c69640 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071abe61c0 0x56071b0819d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28134f9ff75490a8 server_cookie=43ade5258c87fb1 in_seq=0 out_seq=0 2024-09-21T06:19:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.728+0000 7fa090ff9640 1 -- 172.21.15.31:0/1206964781 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56071af6a0f0 con 0x56071abe61c0 2024-09-21T06:19:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.728+0000 7fa090ff9640 10 client.17139 handle_command_reply: tid=0 2024-09-21T06:19:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.728+0000 7fa0ba638740 1 client.17139 shutdown 2024-09-21T06:19:20.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 msgr2=0x56071b006cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 0x56071b006cc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400ffd0 tx=0x7fa0a4011980 comp rx=0 tx=0).stop 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 shutdown_connections 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56071abe61c0 0x56071b0819d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56071ac521b0 0x56071b07dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56071ac314d0 0x56071b006cc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 --2- 172.21.15.31:0/1206964781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56071ac292f0 0x56071b006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.729+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 >> 172.21.15.31:0/1206964781 conn(0x56071ac46300 msgr2=0x56071aff93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa090ff9640 0 client.17139 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 shutdown_connections 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0ba638740 1 -- 172.21.15.31:0/1206964781 wait complete. 2024-09-21T06:19:20.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0ba638740 20 client.17139 trim_cache size 0 max 16384 2024-09-21T06:19:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:19:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:19:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:19:20.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0940b6700 msgr2=0x7fa0940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0940b6700 0x7fa0940b8c10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56071aff6630 tx=0x7fa09c02a410 comp rx=0 tx=0).stop 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 msgr2=0x7fa0ac1a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac1a2c70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005ab0 tx=0x7fa09c003c10 comp rx=0 tx=0).stop 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 shutdown_connections 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa0940b6700 0x7fa0940b8c10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0ac1165a0 0x7fa0ac1a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0ac10aef0 0x7fa0ac1a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 --2- 172.21.15.31:0/3956733645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0ac10a5a0 0x7fa0ac1a2c70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.730+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 >> 172.21.15.31:0/3956733645 conn(0x7fa0ac070dd0 msgr2=0x7fa0ac073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.731+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 shutdown_connections 2024-09-21T06:19:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:20.731+0000 7fa0b36cc640 1 -- 172.21.15.31:0/3956733645 wait complete. 2024-09-21T06:19:21.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:21 smithi110 ceph-mon[34550]: pgmap v556: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 32 MiB/s wr, 1.33k op/s 2024-09-21T06:19:21.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:21 smithi107 ceph-mon[34380]: pgmap v556: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 32 MiB/s wr, 1.33k op/s 2024-09-21T06:19:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:21 smithi031 ceph-mon[31237]: pgmap v556: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 32 MiB/s wr, 1.33k op/s 2024-09-21T06:19:23.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:23 smithi110 ceph-mon[34550]: pgmap v557: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 36 MiB/s wr, 1.61k op/s 2024-09-21T06:19:23.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:23 smithi107 ceph-mon[34380]: pgmap v557: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 36 MiB/s wr, 1.61k op/s 2024-09-21T06:19:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:23 smithi031 ceph-mon[31237]: pgmap v557: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 36 MiB/s wr, 1.61k op/s 2024-09-21T06:19:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 465.0 s, 2086.8 tps, lat 48.344 ms stddev 52.026 2024-09-21T06:19:25.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:25 smithi110 ceph-mon[34550]: pgmap v558: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 36 MiB/s wr, 1.61k op/s 2024-09-21T06:19:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:25 smithi107 ceph-mon[34380]: pgmap v558: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 36 MiB/s wr, 1.61k op/s 2024-09-21T06:19:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:25 smithi031 ceph-mon[31237]: pgmap v558: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 36 MiB/s wr, 1.61k op/s 2024-09-21T06:19:27.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:27 smithi110 ceph-mon[34550]: pgmap v559: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-09-21T06:19:27.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:27 smithi107 ceph-mon[34380]: pgmap v559: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-09-21T06:19:27.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:27 smithi031 ceph-mon[31237]: pgmap v559: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 808 op/s 2024-09-21T06:19:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:29 smithi110 ceph-mon[34550]: pgmap v560: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 747 op/s 2024-09-21T06:19:29.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:29 smithi107 ceph-mon[34380]: pgmap v560: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 747 op/s 2024-09-21T06:19:29.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:29 smithi031 ceph-mon[31237]: pgmap v560: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.6 MiB/s wr, 747 op/s 2024-09-21T06:19:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 470.0 s, 2418.4 tps, lat 40.902 ms stddev 44.806 2024-09-21T06:19:31.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:31 smithi110 ceph-mon[34550]: pgmap v561: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 711 op/s 2024-09-21T06:19:31.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:31 smithi107 ceph-mon[34380]: pgmap v561: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 711 op/s 2024-09-21T06:19:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:31 smithi031 ceph-mon[31237]: pgmap v561: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.2 MiB/s wr, 711 op/s 2024-09-21T06:19:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:33 smithi110 ceph-mon[34550]: pgmap v562: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 816 op/s 2024-09-21T06:19:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:33 smithi107 ceph-mon[34380]: pgmap v562: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 816 op/s 2024-09-21T06:19:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:33 smithi031 ceph-mon[31237]: pgmap v562: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.3 MiB/s wr, 816 op/s 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 -- 172.21.15.31:0/2454720037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108c90 msgr2=0x7fc44810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/2454720037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108c90 0x7fc44810f4c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc440009be0 tx=0x7fc44002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 -- 172.21.15.31:0/2454720037 shutdown_connections 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/2454720037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108c90 0x7fc44810f4c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/2454720037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108340 0x7fc448108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/2454720037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc448102a10 0x7fc448102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 -- 172.21.15.31:0/2454720037 >> 172.21.15.31:0/2454720037 conn(0x7fc4480fe410 msgr2=0x7fc448100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 -- 172.21.15.31:0/2454720037 shutdown_connections 2024-09-21T06:19:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.877+0000 7fc44ec3f640 1 -- 172.21.15.31:0/2454720037 wait complete. 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.878+0000 7fc44ec3f640 1 Processor -- start 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.878+0000 7fc44ec3f640 1 -- start start 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44ec3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc448102a10 0x7fc448198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44ec3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108340 0x7fc4481994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44ec3f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 0x7fc448199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44ec3f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc448112950 con 0x7fc448102a10 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44ec3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481127d0 con 0x7fc448108340 2024-09-21T06:19:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44ec3f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc448112ad0 con 0x7fc448108c90 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44d1dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 0x7fc448199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc447fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108340 0x7fc4481994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44d1dc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 0x7fc448199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55374/0 (socket says 172.21.15.31:55374) 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.879+0000 7fc44d1dc640 1 -- 172.21.15.31:0/3301076240 learned_addr learned my addr 172.21.15.31:0/3301076240 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc44d1dc640 1 -- 172.21.15.31:0/3301076240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108340 msgr2=0x7fc4481994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc44d1dc640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108340 0x7fc4481994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc44d1dc640 1 -- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc448102a10 msgr2=0x7fc448198fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc44d1dc640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc448102a10 0x7fc448198fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc44d1dc640 1 -- 172.21.15.31:0/3301076240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44819e290 con 0x7fc448108c90 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc44d1dc640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 0x7fc448199a30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc440009b30 tx=0x7fc440003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.880+0000 7fc445ffb640 1 -- 172.21.15.31:0/3301076240 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440040020 con 0x7fc448108c90 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.881+0000 7fc445ffb640 1 -- 172.21.15.31:0/3301076240 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc440030970 con 0x7fc448108c90 2024-09-21T06:19:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.881+0000 7fc445ffb640 1 -- 172.21.15.31:0/3301076240 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440036040 con 0x7fc448108c90 2024-09-21T06:19:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.881+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44819e520 con 0x7fc448108c90 2024-09-21T06:19:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.881+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44819ea60 con 0x7fc448108c90 2024-09-21T06:19:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.885+0000 7fc455c8a740 1 Processor -- start 2024-09-21T06:19:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.885+0000 7fc445ffb640 1 -- 172.21.15.31:0/3301076240 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc440043020 con 0x7fc448108c90 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 -- start start 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 0x558ab9ea49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9ea4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9ea5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558ab9e196f0 con 0x558ab9ade4d0 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ab9e08b20 con 0x558ab9aff1b0 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc455c8a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558ab9e0c930 con 0x558ab9ad62f0 2024-09-21T06:19:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc447fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 0x558ab9ea49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc447fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 0x558ab9ea49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58354/0 (socket says 172.21.15.31:58354) 2024-09-21T06:19:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc447fff640 1 -- 172.21.15.31:0/362305678 learned_addr learned my addr 172.21.15.31:0/362305678 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc445ffb640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc4380b6700 0x7fc4380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.886+0000 7fc447fff640 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9ea5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc44c9db640 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9ea4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc447fff640 1 -- 172.21.15.31:0/362305678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 msgr2=0x558ab9ea4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc447fff640 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9ea4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc447fff640 1 -- 172.21.15.31:0/362305678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 msgr2=0x558ab9ea5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc447fff640 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9ea5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc44d1dc640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc4380b6700 0x7fc4380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc447fff640 1 -- 172.21.15.31:0/362305678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab9dce530 con 0x558ab9aff1b0 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc445ffb640 1 -- 172.21.15.31:0/3301076240 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fc440108370 con 0x7fc448108c90 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc447fff640 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 0x558ab9ea49b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4340117a0 tx=0x7fc434009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b96c58be0ac7ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc432ffd640 1 -- 172.21.15.31:0/362305678 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434004050 con 0x558ab9aff1b0 2024-09-21T06:19:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc44d1dc640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc4380b6700 0x7fc4380b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc440105b80 tx=0x7fc4400c2a30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc432ffd640 1 -- 172.21.15.31:0/362305678 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43400fc70 con 0x558ab9aff1b0 2024-09-21T06:19:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.887+0000 7fc432ffd640 1 -- 172.21.15.31:0/362305678 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340037f0 con 0x558ab9aff1b0 2024-09-21T06:19:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.888+0000 7fc455c8a740 1 -- 172.21.15.31:0/362305678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 msgr2=0x558ab9ea49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.888+0000 7fc455c8a740 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 0x558ab9ea49b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4340117a0 tx=0x7fc434009770 comp rx=0 tx=0).stop 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.889+0000 7fc455c8a740 1 -- 172.21.15.31:0/362305678 shutdown_connections 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.889+0000 7fc455c8a740 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9ea5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.889+0000 7fc455c8a740 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9ea4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.889+0000 7fc455c8a740 1 --2- 172.21.15.31:0/362305678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9aff1b0 0x558ab9ea49b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.889+0000 7fc455c8a740 1 -- 172.21.15.31:0/362305678 >> 172.21.15.31:0/362305678 conn(0x558ab9af3300 msgr2=0x558ab9ea28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.890+0000 7fc455c8a740 1 -- 172.21.15.31:0/362305678 shutdown_connections 2024-09-21T06:19:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.890+0000 7fc455c8a740 1 -- 172.21.15.31:0/362305678 wait complete. 2024-09-21T06:19:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.891+0000 7fc455c8a740 1 -- start start 2024-09-21T06:19:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.891+0000 7fc455c8a740 1 Processor -- start 2024-09-21T06:19:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.891+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 0x558ab9eb0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eb0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9aff1b0 0x558ab9ebc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc455c8a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558ab9d40ab0 con 0x558ab9ad62f0 2024-09-21T06:19:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc455c8a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ab9dae730 con 0x558ab9ade4d0 2024-09-21T06:19:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc455c8a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558ab9eb3130 con 0x558ab9aff1b0 2024-09-21T06:19:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc455c8a740 10 client.0 _ll_register_callbacks cb 0x558ab9e026b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc447fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9aff1b0 0x558ab9ebc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eb0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc447fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 0x558ab9eb0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eb0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58358/0 (socket says 172.21.15.31:58358) 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 -- 172.21.15.31:0/1723730667 learned_addr learned my addr 172.21.15.31:0/1723730667 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc447fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 0x558ab9eb0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43294/0 (socket says 172.21.15.31:43294) 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 -- 172.21.15.31:0/1723730667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9aff1b0 msgr2=0x558ab9ebc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9aff1b0 0x558ab9ebc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 -- 172.21.15.31:0/1723730667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 msgr2=0x558ab9eb0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 0x558ab9eb0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc44c9db640 1 -- 172.21.15.31:0/1723730667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab9b6a790 con 0x558ab9ade4d0 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.892+0000 7fc447fff640 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 0x558ab9eb0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.893+0000 7fc427fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.893+0000 7fc44c9db640 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eb0670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc43c010410 tx=0x7fc43c0108d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef43412813cc151f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.893+0000 7fc427fff640 1 -- 172.21.15.31:0/1723730667 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c005220 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.893+0000 7fc427fff640 1 -- 172.21.15.31:0/1723730667 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fc43c005020 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.893+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43400fc70 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.893+0000 7fc427fff640 1 -- 172.21.15.31:0/1723730667 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc43c005220 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 10 client.26900 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558ab9be3620 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc427fff640 1 -- 172.21.15.31:0/1723730667 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43400fc70 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc427fff640 1 -- 172.21.15.31:0/1723730667 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc43c00f040 con 0x558ab9ade4d0 2024-09-21T06:19:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 10 client.26900 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 20 client.26900 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9abdd80 0x558ab9ec2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 4 client.26900 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab9e1b640 con 0x558ab9abdd80 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.894+0000 7fc447fff640 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9abdd80 0x558ab9ec2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.895+0000 7fc427fff640 10 client.26900 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.895+0000 7fc447fff640 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9abdd80 0x558ab9ec2240 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1ee401fb9ca3ad server_cookie=43719a8a20ec95ab in_seq=0 out_seq=0 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.895+0000 7fc427fff640 1 -- 172.21.15.31:0/1723730667 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ab9e1b640 con 0x558ab9abdd80 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.895+0000 7fc427fff640 10 client.26900 handle_command_reply: tid=0 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.896+0000 7fc455c8a740 1 client.26900 shutdown 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.896+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 msgr2=0x558ab9eb0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.896+0000 7fc455c8a740 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eb0670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc43c010410 tx=0x7fc43c0108d0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.896+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 shutdown_connections 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9abdd80 0x558ab9ec2240 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc427fff640 0 client.26900 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9aff1b0 0x558ab9ebc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eb0670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 --2- 172.21.15.31:0/1723730667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ad62f0 0x558ab9eb0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 >> 172.21.15.31:0/1723730667 conn(0x558ab9af3300 msgr2=0x558ab9ea59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 shutdown_connections 2024-09-21T06:19:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.897+0000 7fc455c8a740 1 -- 172.21.15.31:0/1723730667 wait complete. 2024-09-21T06:19:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.898+0000 7fc455c8a740 20 client.26900 trim_cache size 0 max 16384 2024-09-21T06:19:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 Processor -- start 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 -- start start 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9abdd80 0x558ab9eb0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eaa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481b3c90 con 0x558ab9abdd80 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481b64f0 con 0x558ab9ade4d0 2024-09-21T06:19:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.920+0000 7fc455c8a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481b42e0 con 0x558ab9ad62f0 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc447fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9abdd80 0x558ab9eb0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc447fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9abdd80 0x558ab9eb0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43302/0 (socket says 172.21.15.31:43302) 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc447fff640 1 -- 172.21.15.31:0/511032217 learned_addr learned my addr 172.21.15.31:0/511032217 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc44c9db640 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc447fff640 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eaa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc44c9db640 1 -- 172.21.15.31:0/511032217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 msgr2=0x558ab9eaa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc44c9db640 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eaa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc44c9db640 1 -- 172.21.15.31:0/511032217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9abdd80 msgr2=0x558ab9eb0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc44c9db640 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9abdd80 0x558ab9eb0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.921+0000 7fc44c9db640 1 -- 172.21.15.31:0/511032217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab9eaf350 con 0x558ab9ad62f0 2024-09-21T06:19:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.922+0000 7fc44c9db640 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eab830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006010 tx=0x7fc43c013fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78030e48415c88e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.922+0000 7fc4327fc640 1 -- 172.21.15.31:0/511032217 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c027070 con 0x558ab9ad62f0 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.922+0000 7fc4327fc640 1 -- 172.21.15.31:0/511032217 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43c028040 con 0x558ab9ad62f0 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.922+0000 7fc4327fc640 1 -- 172.21.15.31:0/511032217 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c003dd0 con 0x558ab9ad62f0 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 -- 172.21.15.31:0/511032217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 msgr2=0x558ab9eab830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eab830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006010 tx=0x7fc43c013fb0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 -- 172.21.15.31:0/511032217 shutdown_connections 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9ade4d0 0x558ab9eaa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eab830 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 --2- 172.21.15.31:0/511032217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9abdd80 0x558ab9eb0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.923+0000 7fc455c8a740 1 -- 172.21.15.31:0/511032217 >> 172.21.15.31:0/511032217 conn(0x558ab9af3300 msgr2=0x558ab9ea9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.924+0000 7fc455c8a740 1 -- 172.21.15.31:0/511032217 shutdown_connections 2024-09-21T06:19:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.924+0000 7fc455c8a740 1 -- 172.21.15.31:0/511032217 wait complete. 2024-09-21T06:19:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.924+0000 7fc455c8a740 1 -- start start 2024-09-21T06:19:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 Processor -- start 2024-09-21T06:19:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 0x558ab9eb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eb4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9f24c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481b49b0 con 0x558ab9ade4d0 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481b5e20 con 0x558ab9abdd80 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc4481b5080 con 0x558ab9ad62f0 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc447fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 0x558ab9eb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc447fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 0x558ab9eb3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:58378/0 (socket says 172.21.15.31:58378) 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc447fff640 1 -- 172.21.15.31:0/2577625764 learned_addr learned my addr 172.21.15.31:0/2577625764 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.925+0000 7fc455c8a740 10 client.0 _ll_register_callbacks cb 0x7fc43c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9f24c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc44c9db640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eb4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 -- 172.21.15.31:0/2577625764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 msgr2=0x558ab9eb4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eb4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 -- 172.21.15.31:0/2577625764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 msgr2=0x558ab9f24c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9f24c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 -- 172.21.15.31:0/2577625764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab9f252f0 con 0x558ab9abdd80 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc44c9db640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eb4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc447fff640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 0x558ab9eb3c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4340059f0 tx=0x7fc434021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedf0360eb3501ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc427fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc427fff640 1 -- 172.21.15.31:0/2577625764 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434011250 con 0x558ab9abdd80 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc427fff640 1 -- 172.21.15.31:0/2577625764 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fc434005040 con 0x558ab9abdd80 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.926+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43c028040 con 0x558ab9abdd80 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc427fff640 1 -- 172.21.15.31:0/2577625764 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc434005240 con 0x558ab9abdd80 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc455c8a740 10 client.26906 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558ab9f2c620 con 0x558ab9abdd80 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc427fff640 1 -- 172.21.15.31:0/2577625764 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43c028040 con 0x558ab9abdd80 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc427fff640 1 -- 172.21.15.31:0/2577625764 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc434043470 con 0x558ab9abdd80 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc455c8a740 10 client.26906 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.927+0000 7fc455c8a740 10 client.26906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.928+0000 7fc455c8a740 20 client.26906 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.928+0000 7fc455c8a740 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9aff1b0 0x558ab9f2ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.928+0000 7fc455c8a740 4 client.26906 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.928+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ab9e170f0 con 0x558ab9aff1b0 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.928+0000 7fc44c9db640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9aff1b0 0x558ab9f2ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.929+0000 7fc427fff640 10 client.26906 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.930+0000 7fc44c9db640 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9aff1b0 0x558ab9f2ebf0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf990c71339e7f08 server_cookie=fb7c4f0ace654e47 in_seq=0 out_seq=0 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc427fff640 1 -- 172.21.15.31:0/2577625764 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ab9e170f0 con 0x558ab9aff1b0 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc427fff640 10 client.26906 handle_command_reply: tid=0 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 client.26906 shutdown 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 msgr2=0x558ab9eb3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 0x558ab9eb3c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4340059f0 tx=0x7fc434021040 comp rx=0 tx=0).stop 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 shutdown_connections 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ab9aff1b0 0x558ab9f2ebf0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ab9ade4d0 0x558ab9f24c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc427fff640 0 client.26906 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ab9ad62f0 0x558ab9eb4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.955+0000 7fc455c8a740 1 --2- 172.21.15.31:0/2577625764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ab9abdd80 0x558ab9eb3c80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.956+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 >> 172.21.15.31:0/2577625764 conn(0x558ab9af3300 msgr2=0x558ab9ea59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.956+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 shutdown_connections 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.956+0000 7fc455c8a740 1 -- 172.21.15.31:0/2577625764 wait complete. 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.956+0000 7fc455c8a740 20 client.26906 trim_cache size 0 max 16384 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:19:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:19:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.957+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc4380b6700 msgr2=0x7fc4380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.957+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc4380b6700 0x7fc4380b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc440105b80 tx=0x7fc4400c2a30 comp rx=0 tx=0).stop 2024-09-21T06:19:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.957+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 msgr2=0x7fc448199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.957+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 0x7fc448199a30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc440009b30 tx=0x7fc440003c10 comp rx=0 tx=0).stop 2024-09-21T06:19:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.964+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 shutdown_connections 2024-09-21T06:19:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.964+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc448108c90 0x7fc448199a30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.964+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc448108340 0x7fc4481994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.964+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc4380b6700 0x7fc4380b8c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.964+0000 7fc44ec3f640 1 --2- 172.21.15.31:0/3301076240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc448102a10 0x7fc448198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.964+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 >> 172.21.15.31:0/3301076240 conn(0x7fc4480fe410 msgr2=0x7fc448107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.971+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 shutdown_connections 2024-09-21T06:19:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:33.973+0000 7fc44ec3f640 1 -- 172.21.15.31:0/3301076240 wait complete. 2024-09-21T06:19:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 475.0 s, 2915.4 tps, lat 34.605 ms stddev 40.895 2024-09-21T06:19:35.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:35 smithi110 ceph-mon[34550]: pgmap v563: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-09-21T06:19:35.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:35 smithi107 ceph-mon[34380]: pgmap v563: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-09-21T06:19:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:35 smithi031 ceph-mon[31237]: pgmap v563: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-09-21T06:19:37.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:37 smithi110 ceph-mon[34550]: pgmap v564: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.8 MiB/s wr, 798 op/s 2024-09-21T06:19:37.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:37 smithi107 ceph-mon[34380]: pgmap v564: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.8 MiB/s wr, 798 op/s 2024-09-21T06:19:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:37 smithi031 ceph-mon[31237]: pgmap v564: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 7.8 MiB/s wr, 798 op/s 2024-09-21T06:19:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:39 smithi110 ceph-mon[34550]: pgmap v565: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-09-21T06:19:39.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:39 smithi107 ceph-mon[34380]: pgmap v565: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-09-21T06:19:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:39 smithi031 ceph-mon[31237]: pgmap v565: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 646 op/s 2024-09-21T06:19:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 480.0 s, 2582.8 tps, lat 38.404 ms stddev 43.935 2024-09-21T06:19:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:41 smithi110 ceph-mon[34550]: pgmap v566: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-09-21T06:19:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:41 smithi107 ceph-mon[34380]: pgmap v566: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-09-21T06:19:41.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:41 smithi031 ceph-mon[31237]: pgmap v566: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-09-21T06:19:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:43 smithi110 ceph-mon[34550]: pgmap v567: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.9 MiB/s wr, 816 op/s 2024-09-21T06:19:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:43 smithi107 ceph-mon[34380]: pgmap v567: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.9 MiB/s wr, 816 op/s 2024-09-21T06:19:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:43 smithi031 ceph-mon[31237]: pgmap v567: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.9 MiB/s wr, 816 op/s 2024-09-21T06:19:44.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:19:44.676 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 485.0 s, 2449.2 tps, lat 41.210 ms stddev 46.401 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 -- 172.21.15.31:0/63085222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c1083a0 msgr2=0x7f383c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 --2- 172.21.15.31:0/63085222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c1083a0 0x7f383c1087b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3830009900 tx=0x7f383002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 -- 172.21.15.31:0/63085222 shutdown_connections 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 --2- 172.21.15.31:0/63085222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f383c108cf0 0x7f383c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 --2- 172.21.15.31:0/63085222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c1083a0 0x7f383c1087b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 --2- 172.21.15.31:0/63085222 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c102a60 0x7f383c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 -- 172.21.15.31:0/63085222 >> 172.21.15.31:0/63085222 conn(0x7f383c079730 msgr2=0x7f383c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 -- 172.21.15.31:0/63085222 shutdown_connections 2024-09-21T06:19:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.293+0000 7f384094b640 1 -- 172.21.15.31:0/63085222 wait complete. 2024-09-21T06:19:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.294+0000 7f384094b640 1 Processor -- start 2024-09-21T06:19:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.295+0000 7f384094b640 1 -- start start 2024-09-21T06:19:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.295+0000 7f384094b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f383c102a60 0x7f383c10c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.295+0000 7f384094b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c1083a0 0x7f383c10cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f384094b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c108cf0 0x7f383c10d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f384094b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f383c111890 con 0x7f383c1083a0 2024-09-21T06:19:45.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f3839d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c1083a0 0x7f383c10cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f383ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c108cf0 0x7f383c10d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f383ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c108cf0 0x7f383c10d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55066/0 (socket says 172.21.15.31:55066) 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f383ad9d640 1 -- 172.21.15.31:0/4268622137 learned_addr learned my addr 172.21.15.31:0/4268622137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f383c111710 con 0x7f383c108cf0 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f383c111a10 con 0x7f383c102a60 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.296+0000 7f383a59c640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f383c102a60 0x7f383c10c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f3839d9b640 1 -- 172.21.15.31:0/4268622137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f383c102a60 msgr2=0x7f383c10c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f3839d9b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f383c102a60 0x7f383c10c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f3839d9b640 1 -- 172.21.15.31:0/4268622137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c108cf0 msgr2=0x7f383c10d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f3839d9b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c108cf0 0x7f383c10d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f3839d9b640 1 -- 172.21.15.31:0/4268622137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c1aca10 con 0x7f383c1083a0 2024-09-21T06:19:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f3839d9b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c1083a0 0x7f383c10cd90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3830009850 tx=0x7f3830030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f382b7fe640 1 -- 172.21.15.31:0/4268622137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383003f070 con 0x7f383c1083a0 2024-09-21T06:19:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c1acca0 con 0x7f383c1083a0 2024-09-21T06:19:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f382b7fe640 1 -- 172.21.15.31:0/4268622137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830004460 con 0x7f383c1083a0 2024-09-21T06:19:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f382b7fe640 1 -- 172.21.15.31:0/4268622137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830035030 con 0x7f383c1083a0 2024-09-21T06:19:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.297+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c1ad180 con 0x7f383c1083a0 2024-09-21T06:19:45.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.301+0000 7f3846ebf740 1 Processor -- start 2024-09-21T06:19:45.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.302+0000 7f382b7fe640 1 -- 172.21.15.31:0/4268622137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3830042070 con 0x7f383c1083a0 2024-09-21T06:19:45.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 -- start start 2024-09-21T06:19:45.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26281070 0x563a26628650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26628b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26260390 0x563a266290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563a2659ee50 con 0x563a26260390 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a2658e280 con 0x563a262581b0 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.304+0000 7f3846ebf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563a26592090 con 0x563a26281070 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26628b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26628b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55082/0 (socket says 172.21.15.31:55082) 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 -- 172.21.15.31:0/1799154890 learned_addr learned my addr 172.21.15.31:0/1799154890 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383ad9d640 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26281070 0x563a26628650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 -- 172.21.15.31:0/1799154890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26281070 msgr2=0x563a26628650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26281070 0x563a26628650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 -- 172.21.15.31:0/1799154890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26260390 msgr2=0x563a266290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26260390 0x563a266290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 -- 172.21.15.31:0/1799154890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a262ec650 con 0x563a262581b0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.305+0000 7f383a59c640 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26628b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f382c00a8e0 tx=0x7f382c00ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe2ad1bb61ed43c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.306+0000 7f3828ff9640 1 -- 172.21.15.31:0/1799154890 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c015070 con 0x563a262581b0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.306+0000 7f3828ff9640 1 -- 172.21.15.31:0/1799154890 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382c0113f0 con 0x563a262581b0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.306+0000 7f3828ff9640 1 -- 172.21.15.31:0/1799154890 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c0185a0 con 0x563a262581b0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.306+0000 7f382b7fe640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f38240b6700 0x7f38240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.306+0000 7f383ad9d640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f38240b6700 0x7f38240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.306+0000 7f382b7fe640 1 -- 172.21.15.31:0/4268622137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f38301081f0 con 0x7f383c1083a0 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.307+0000 7f3846ebf740 1 -- 172.21.15.31:0/1799154890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 msgr2=0x563a26628b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.307+0000 7f3846ebf740 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26628b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f382c00a8e0 tx=0x7f382c00ada0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.307+0000 7f3846ebf740 1 -- 172.21.15.31:0/1799154890 shutdown_connections 2024-09-21T06:19:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.307+0000 7f383ad9d640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f38240b6700 0x7f38240b8c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3834005c40 tx=0x7f3834009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:45.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.307+0000 7f3846ebf740 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26260390 0x563a266290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.308+0000 7f3846ebf740 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26628b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.308+0000 7f3846ebf740 1 --2- 172.21.15.31:0/1799154890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26281070 0x563a26628650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.308+0000 7f3846ebf740 1 -- 172.21.15.31:0/1799154890 >> 172.21.15.31:0/1799154890 conn(0x563a26275460 msgr2=0x563a26626540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.308+0000 7f3846ebf740 1 -- 172.21.15.31:0/1799154890 shutdown_connections 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.308+0000 7f3846ebf740 1 -- 172.21.15.31:0/1799154890 wait complete. 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.309+0000 7f3846ebf740 1 -- start start 2024-09-21T06:19:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.309+0000 7f3846ebf740 1 Processor -- start 2024-09-21T06:19:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.310+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a26633db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.310+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.310+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26281070 0x563a26640120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.310+0000 7f3846ebf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563a263ef700 con 0x563a26281070 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.310+0000 7f3846ebf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a26529110 con 0x563a26260390 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.310+0000 7f3839d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f3846ebf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563a26636e30 con 0x563a262581b0 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a26633db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26281070 0x563a26640120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a26633db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48108/0 (socket says 172.21.15.31:48108) 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26281070 0x563a26640120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44426/0 (socket says 172.21.15.31:44426) 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 -- 172.21.15.31:0/3873174952 learned_addr learned my addr 172.21.15.31:0/3873174952 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f3846ebf740 10 client.0 _ll_register_callbacks cb 0x563a26542990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 -- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 msgr2=0x563a266342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 -- 172.21.15.31:0/3873174952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26281070 msgr2=0x563a26640120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26281070 0x563a26640120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 -- 172.21.15.31:0/3873174952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a26556c40 con 0x563a262581b0 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f383a59c640 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a26633db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f382c00ce70 tx=0x7f382c0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c50eef0f9abb98b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.311+0000 7f3821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.312+0000 7f3821ffb640 1 -- 172.21.15.31:0/3873174952 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c015040 con 0x563a262581b0 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.312+0000 7f3821ffb640 1 -- 172.21.15.31:0/3873174952 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f382c01a450 con 0x563a262581b0 2024-09-21T06:19:45.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.312+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c0113f0 con 0x563a262581b0 2024-09-21T06:19:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.315+0000 7f3821ffb640 1 -- 172.21.15.31:0/3873174952 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f382c00a4b0 con 0x563a262581b0 2024-09-21T06:19:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.315+0000 7f3846ebf740 10 client.26968 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.315+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563a26642be0 con 0x563a262581b0 2024-09-21T06:19:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.316+0000 7f3821ffb640 1 -- 172.21.15.31:0/3873174952 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382c0113f0 con 0x563a262581b0 2024-09-21T06:19:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.316+0000 7f3821ffb640 1 -- 172.21.15.31:0/3873174952 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f382c025750 con 0x563a262581b0 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.316+0000 7f3846ebf740 10 client.26968 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.316+0000 7f3846ebf740 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.316+0000 7f3846ebf740 20 client.26968 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.317+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a2623f9f0 0x563a26645f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.317+0000 7f3846ebf740 4 client.26968 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.317+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a265a0da0 con 0x563a2623f9f0 2024-09-21T06:19:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.317+0000 7f3839d9b640 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a2623f9f0 0x563a26645f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.317+0000 7f3839d9b640 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a2623f9f0 0x563a26645f40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4630bc67ba80bb03 server_cookie=e44fb595c8f9ba0a in_seq=0 out_seq=0 2024-09-21T06:19:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.317+0000 7f3821ffb640 10 client.26968 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3821ffb640 1 -- 172.21.15.31:0/3873174952 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a265a0da0 con 0x563a2623f9f0 2024-09-21T06:19:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3821ffb640 10 client.26968 handle_command_reply: tid=0 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 client.26968 shutdown 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 msgr2=0x563a26633db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a26633db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f382c00ce70 tx=0x7f382c0047d0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 shutdown_connections 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a2623f9f0 0x563a26645f40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a26281070 0x563a26640120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3873174952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a26633db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3821ffb640 0 client.26968 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 >> 172.21.15.31:0/3873174952 conn(0x563a26275460 msgr2=0x563a2662b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 shutdown_connections 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.318+0000 7f3846ebf740 1 -- 172.21.15.31:0/3873174952 wait complete. 2024-09-21T06:19:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.319+0000 7f3846ebf740 20 client.26968 trim_cache size 0 max 16384 2024-09-21T06:19:45.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.348+0000 7f3846ebf740 1 Processor -- start 2024-09-21T06:19:45.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.349+0000 7f3846ebf740 1 -- start start 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.349+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a26635c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.349+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26629d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.349+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26260390 0x563a26640090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f3846ebf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b3d40 con 0x563a2623f9f0 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f3846ebf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b65a0 con 0x563a262581b0 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f3846ebf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b4390 con 0x563a26260390 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a26635c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26260390 0x563a26640090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a26635c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44430/0 (socket says 172.21.15.31:44430) 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f3839d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26629d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383a59c640 1 -- 172.21.15.31:0/3175805719 learned_addr learned my addr 172.21.15.31:0/3175805719 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:45.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383ad9d640 1 -- 172.21.15.31:0/3175805719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 msgr2=0x563a26629d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383ad9d640 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26629d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383ad9d640 1 -- 172.21.15.31:0/3175805719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 msgr2=0x563a26635c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383ad9d640 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a26635c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.350+0000 7f383ad9d640 1 -- 172.21.15.31:0/3175805719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a26635070 con 0x563a26260390 2024-09-21T06:19:45.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.351+0000 7f383ad9d640 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26260390 0x563a26640090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3834009fd0 tx=0x7f3834019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa1e6efddfae9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:45.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.351+0000 7f3823fff640 1 -- 172.21.15.31:0/3175805719 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383401e9a0 con 0x563a26260390 2024-09-21T06:19:45.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.351+0000 7f3823fff640 1 -- 172.21.15.31:0/3175805719 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3834003970 con 0x563a26260390 2024-09-21T06:19:45.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.351+0000 7f3823fff640 1 -- 172.21.15.31:0/3175805719 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340046d0 con 0x563a26260390 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 -- 172.21.15.31:0/3175805719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26260390 msgr2=0x563a26640090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26260390 0x563a26640090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3834009fd0 tx=0x7f3834019000 comp rx=0 tx=0).stop 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 -- 172.21.15.31:0/3175805719 shutdown_connections 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a26260390 0x563a26640090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a262581b0 0x563a26629d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 --2- 172.21.15.31:0/3175805719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a26635c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 -- 172.21.15.31:0/3175805719 >> 172.21.15.31:0/3175805719 conn(0x563a26275460 msgr2=0x563a26629680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 -- 172.21.15.31:0/3175805719 shutdown_connections 2024-09-21T06:19:45.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.353+0000 7f3846ebf740 1 -- 172.21.15.31:0/3175805719 wait complete. 2024-09-21T06:19:45.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.354+0000 7f3846ebf740 1 -- start start 2024-09-21T06:19:45.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.354+0000 7f3846ebf740 1 Processor -- start 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a266a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a266a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b4a60 con 0x563a2623f9f0 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b5ed0 con 0x563a26260390 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b5130 con 0x563a262581b0 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3846ebf740 10 client.0 _ll_register_callbacks cb 0x7f3834009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f383a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a266a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3839d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a266a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f383ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3839d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a266a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48136/0 (socket says 172.21.15.31:48136) 2024-09-21T06:19:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f383a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a266a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44436/0 (socket says 172.21.15.31:44436) 2024-09-21T06:19:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f3839d9b640 1 -- 172.21.15.31:0/2672730027 learned_addr learned my addr 172.21.15.31:0/2672730027 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:45.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.355+0000 7f383ad9d640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55106/0 (socket says 172.21.15.31:55106) 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f383ad9d640 1 -- 172.21.15.31:0/2672730027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 msgr2=0x563a266a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f383ad9d640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a266a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f383ad9d640 1 -- 172.21.15.31:0/2672730027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 msgr2=0x563a266a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f383ad9d640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a266a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f383ad9d640 1 -- 172.21.15.31:0/2672730027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a266afc60 con 0x563a26260390 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f3839d9b640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a266a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f383ad9d640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266af6d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3834009fd0 tx=0x7f3834003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48422f45c1538b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f3821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.356+0000 7f3821ffb640 1 -- 172.21.15.31:0/2672730027 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834004000 con 0x563a26260390 2024-09-21T06:19:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.357+0000 7f3821ffb640 1 -- 172.21.15.31:0/2672730027 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f3834004c50 con 0x563a26260390 2024-09-21T06:19:45.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.357+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3834003970 con 0x563a26260390 2024-09-21T06:19:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.359+0000 7f3821ffb640 1 -- 172.21.15.31:0/2672730027 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3834004e50 con 0x563a26260390 2024-09-21T06:19:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.359+0000 7f3846ebf740 10 client.26924 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.359+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563a266b0ef0 con 0x563a26260390 2024-09-21T06:19:45.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.359+0000 7f3821ffb640 1 -- 172.21.15.31:0/2672730027 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383401ee50 con 0x563a26260390 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3821ffb640 1 -- 172.21.15.31:0/2672730027 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f3834005150 con 0x563a26260390 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3846ebf740 10 client.26924 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3846ebf740 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3846ebf740 20 client.26924 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3846ebf740 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a26281070 0x563a266b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3846ebf740 4 client.26924 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3834003970 con 0x563a26281070 2024-09-21T06:19:45.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.360+0000 7f383a59c640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a26281070 0x563a266b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f383a59c640 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a26281070 0x563a266b34a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8d84f221f3532c server_cookie=b3e57ad6ee344cde in_seq=0 out_seq=0 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3821ffb640 10 client.26924 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3821ffb640 1 -- 172.21.15.31:0/2672730027 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x7f3834003970 con 0x563a26281070 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3821ffb640 10 client.26924 handle_command_reply: tid=0 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 client.26924 shutdown 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 msgr2=0x563a266af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266af6d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3834009fd0 tx=0x7f3834003d50 comp rx=0 tx=0).stop 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 shutdown_connections 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563a26281070 0x563a266b34a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a26260390 0x563a266af6d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a262581b0 0x563a266a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 --2- 172.21.15.31:0/2672730027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563a2623f9f0 0x563a266a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 >> 172.21.15.31:0/2672730027 conn(0x563a26275460 msgr2=0x563a2662a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3821ffb640 0 client.26924 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.361+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 shutdown_connections 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f3846ebf740 1 -- 172.21.15.31:0/2672730027 wait complete. 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f3846ebf740 20 client.26924 trim_cache size 0 max 16384 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:19:45.365 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:19:45.366 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.367 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.368 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:19:45.369 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.371 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:19:45.372 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.374 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:19:45.375 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.377 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:45.378 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f38240b6700 msgr2=0x7f38240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f384094b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f38240b6700 0x7f38240b8c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3834005c40 tx=0x7f3834009760 comp rx=0 tx=0).stop 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c1083a0 msgr2=0x7f383c10cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f384094b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c1083a0 0x7f383c10cd90 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3830009850 tx=0x7f3830030650 comp rx=0 tx=0).stop 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 shutdown_connections 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.362+0000 7f384094b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f383c108cf0 0x7f383c10d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.363+0000 7f384094b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f38240b6700 0x7f38240b8c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.363+0000 7f384094b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f383c1083a0 0x7f383c10cd90 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.363+0000 7f384094b640 1 --2- 172.21.15.31:0/4268622137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f383c102a60 0x7f383c10c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.363+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 >> 172.21.15.31:0/4268622137 conn(0x7f383c079730 msgr2=0x7f383c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:45.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.363+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 shutdown_connections 2024-09-21T06:19:45.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:45.363+0000 7f384094b640 1 -- 172.21.15.31:0/4268622137 wait complete. 2024-09-21T06:19:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:45 smithi110 ceph-mon[34550]: pgmap v568: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-09-21T06:19:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:45 smithi107 ceph-mon[34380]: pgmap v568: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-09-21T06:19:45.580 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:45 smithi031 ceph-mon[31237]: pgmap v568: 97 pgs: 97 active+clean; 4.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 591 op/s 2024-09-21T06:19:46.235 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:19:46.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:19:46.692 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.140+0000 7f9add28b640 1 -- 172.21.15.31:0/828844312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad8107bb0 msgr2=0x7f9ad8108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.140+0000 7f9add28b640 1 --2- 172.21.15.31:0/828844312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad8107bb0 0x7f9ad8108020 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009be0 tx=0x7f9ad002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 -- 172.21.15.31:0/828844312 shutdown_connections 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 --2- 172.21.15.31:0/828844312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad8107bb0 0x7f9ad8108020 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 --2- 172.21.15.31:0/828844312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8103040 0x7f9ad8107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 --2- 172.21.15.31:0/828844312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad81026f0 0x7f9ad8102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 -- 172.21.15.31:0/828844312 >> 172.21.15.31:0/828844312 conn(0x7f9ad807a740 msgr2=0x7f9ad807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 -- 172.21.15.31:0/828844312 shutdown_connections 2024-09-21T06:19:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.141+0000 7f9add28b640 1 -- 172.21.15.31:0/828844312 wait complete. 2024-09-21T06:19:47.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.142+0000 7f9add28b640 1 Processor -- start 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 -- start start 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad81026f0 0x7f9ad8074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 0x7f9ad806e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8107bb0 0x7f9ad806e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad811a070 con 0x7f9ad8103040 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8119ef0 con 0x7f9ad8107bb0 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9add28b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad811a1f0 con 0x7f9ad81026f0 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9ad67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 0x7f9ad806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9ad67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 0x7f9ad806e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44452/0 (socket says 172.21.15.31:44452) 2024-09-21T06:19:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.143+0000 7f9ad67fc640 1 -- 172.21.15.31:0/4095401656 learned_addr learned my addr 172.21.15.31:0/4095401656 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8107bb0 0x7f9ad806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad81026f0 0x7f9ad8074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad77fe640 1 -- 172.21.15.31:0/4095401656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad81026f0 msgr2=0x7f9ad8074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad81026f0 0x7f9ad8074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad77fe640 1 -- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 msgr2=0x7f9ad806e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 0x7f9ad806e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad77fe640 1 -- 172.21.15.31:0/4095401656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad806efa0 con 0x7f9ad8107bb0 2024-09-21T06:19:47.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.144+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 0x7f9ad806e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.145+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8107bb0 0x7f9ad806e780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0034040 tx=0x7f9ad0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.145+0000 7f9abffff640 1 -- 172.21.15.31:0/4095401656 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0040020 con 0x7f9ad8107bb0 2024-09-21T06:19:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.145+0000 7f9abffff640 1 -- 172.21.15.31:0/4095401656 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ad0030970 con 0x7f9ad8107bb0 2024-09-21T06:19:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.146+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad8073bf0 con 0x7f9ad8107bb0 2024-09-21T06:19:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.146+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad8074130 con 0x7f9ad8107bb0 2024-09-21T06:19:47.149 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.146+0000 7f9abffff640 1 -- 172.21.15.31:0/4095401656 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0040020 con 0x7f9ad8107bb0 2024-09-21T06:19:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.149+0000 7f9ae41fe740 1 Processor -- start 2024-09-21T06:19:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.152+0000 7f9abffff640 1 -- 172.21.15.31:0/4095401656 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9ad0043020 con 0x7f9ad8107bb0 2024-09-21T06:19:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.153+0000 7f9abffff640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9ac00b6ad0 0x7f9ac00b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.153+0000 7f9abffff640 1 -- 172.21.15.31:0/4095401656 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f9ad0031440 con 0x7f9ad8107bb0 2024-09-21T06:19:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9ac00b6ad0 0x7f9ac00b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 -- start start 2024-09-21T06:19:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2102f0 0x56548c5deef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56548c5536f0 con 0x56548c2102f0 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56548c542b20 con 0x56548c2391b0 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.154+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56548c546930 con 0x56548c2184d0 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.155+0000 7f9ad6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.155+0000 7f9ad6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5de9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55126/0 (socket says 172.21.15.31:55126) 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.155+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/3763430338 learned_addr learned my addr 172.21.15.31:0/3763430338 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:47.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.156+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2102f0 0x56548c5deef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.156+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/3763430338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 msgr2=0x56548c5df430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:19:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.156+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5df430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.156+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/3763430338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2102f0 msgr2=0x56548c5deef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.156+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2102f0 0x56548c5deef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.156+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/3763430338 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548c508530 con 0x56548c2391b0 2024-09-21T06:19:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.159+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9ac00b6ad0 0x7f9ac00b8fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x56548c5dd630 tx=0x7f9acc00b5f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.161+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5de9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b850 tx=0x7f9ac800bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe6dbdad340626b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.166+0000 7f9abd7fa640 1 -- 172.21.15.31:0/3763430338 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800d950 con 0x56548c2391b0 2024-09-21T06:19:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.166+0000 7f9abd7fa640 1 -- 172.21.15.31:0/3763430338 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ac8019070 con 0x56548c2391b0 2024-09-21T06:19:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.166+0000 7f9abd7fa640 1 -- 172.21.15.31:0/3763430338 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8014e40 con 0x56548c2391b0 2024-09-21T06:19:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.170+0000 7f9ae41fe740 1 -- 172.21.15.31:0/3763430338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 msgr2=0x56548c5de9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.170+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5de9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b850 tx=0x7f9ac800bd10 comp rx=0 tx=0).stop 2024-09-21T06:19:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.172+0000 7f9ae41fe740 1 -- 172.21.15.31:0/3763430338 shutdown_connections 2024-09-21T06:19:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.172+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.172+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2102f0 0x56548c5deef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.172+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/3763430338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5de9b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.172+0000 7f9ae41fe740 1 -- 172.21.15.31:0/3763430338 >> 172.21.15.31:0/3763430338 conn(0x56548c22d300 msgr2=0x56548c5dc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.174+0000 7f9ae41fe740 1 -- 172.21.15.31:0/3763430338 shutdown_connections 2024-09-21T06:19:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.176+0000 7f9ae41fe740 1 -- 172.21.15.31:0/3763430338 wait complete. 2024-09-21T06:19:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.177+0000 7f9ae41fe740 1 -- start start 2024-09-21T06:19:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.177+0000 7f9ae41fe740 1 Processor -- start 2024-09-21T06:19:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5ea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 0x56548c5ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 0x56548c5f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56548c47aab0 con 0x56548c2184d0 2024-09-21T06:19:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56548c4e8730 con 0x56548c2102f0 2024-09-21T06:19:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56548c5ed130 con 0x56548c2391b0 2024-09-21T06:19:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.181+0000 7f9ae41fe740 10 client.0 _ll_register_callbacks cb 0x56548c53c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.182+0000 7f9ad77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 0x56548c5f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.182+0000 7f9ad77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 0x56548c5f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48168/0 (socket says 172.21.15.31:48168) 2024-09-21T06:19:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.182+0000 7f9ad77fe640 1 -- 172.21.15.31:0/1651356882 learned_addr learned my addr 172.21.15.31:0/1651356882 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9ad77fe640 1 -- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 msgr2=0x56548c5ea160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:19:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5ea160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9ad77fe640 1 -- 172.21.15.31:0/1651356882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 msgr2=0x56548c5ea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 0x56548c5ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9ad77fe640 1 -- 172.21.15.31:0/1651356882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548c2a4790 con 0x56548c2391b0 2024-09-21T06:19:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9ad77fe640 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 0x56548c5f6420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00ccf70 tx=0x7f9ad00c8f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bf74d58c2125c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9abaff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.183+0000 7f9abaff5640 1 -- 172.21.15.31:0/1651356882 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0040070 con 0x56548c2391b0 2024-09-21T06:19:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.184+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ac8019070 con 0x56548c2391b0 2024-09-21T06:19:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.184+0000 7f9abaff5640 1 -- 172.21.15.31:0/1651356882 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f9ad00b4110 con 0x56548c2391b0 2024-09-21T06:19:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.184+0000 7f9abaff5640 1 -- 172.21.15.31:0/1651356882 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9ad00ccaf0 con 0x56548c2391b0 2024-09-21T06:19:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 10 client.26983 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56548c31d620 con 0x56548c2391b0 2024-09-21T06:19:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9abaff5640 1 -- 172.21.15.31:0/1651356882 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ac8019070 con 0x56548c2391b0 2024-09-21T06:19:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9abaff5640 1 -- 172.21.15.31:0/1651356882 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f9ad00cbcb0 con 0x56548c2391b0 2024-09-21T06:19:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 10 client.26983 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 10 client.26983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 20 client.26983 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1f7d80 0x56548c60d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 4 client.26983 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.186+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56548c555640 con 0x56548c1f7d80 2024-09-21T06:19:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.187+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1f7d80 0x56548c60d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.194 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.193+0000 7f9abaff5640 10 client.26983 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:47.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.194+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1f7d80 0x56548c60d2e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62265ecf819d4367 server_cookie=668b88c80b55933e in_seq=0 out_seq=0 2024-09-21T06:19:47.199 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.198+0000 7f9abaff5640 1 -- 172.21.15.31:0/1651356882 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56548c555640 con 0x56548c1f7d80 2024-09-21T06:19:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.198+0000 7f9abaff5640 10 client.26983 handle_command_reply: tid=0 2024-09-21T06:19:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.198+0000 7f9ae41fe740 1 client.26983 shutdown 2024-09-21T06:19:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.198+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 msgr2=0x56548c5f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.198+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 0x56548c5f6420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00ccf70 tx=0x7f9ad00c8f40 comp rx=0 tx=0).stop 2024-09-21T06:19:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.199+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 shutdown_connections 2024-09-21T06:19:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.199+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1f7d80 0x56548c60d2e0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.199+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2391b0 0x56548c5f6420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.199+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 0x56548c5ea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.199+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/1651356882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5ea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.199+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 >> 172.21.15.31:0/1651356882 conn(0x56548c22d300 msgr2=0x56548c5df9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.202 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.200+0000 7f9abaff5640 0 client.26983 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.202+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 shutdown_connections 2024-09-21T06:19:47.203 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.202+0000 7f9ae41fe740 1 -- 172.21.15.31:0/1651356882 wait complete. 2024-09-21T06:19:47.204 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.203+0000 7f9ae41fe740 20 client.26983 trim_cache size 0 max 16384 2024-09-21T06:19:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.237+0000 7f9ae41fe740 1 Processor -- start 2024-09-21T06:19:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.237+0000 7f9ae41fe740 1 -- start start 2024-09-21T06:19:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.237+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 0x56548c5f6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.237+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 0x56548c5ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.237+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.237+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad81b73a0 con 0x56548c2184d0 2024-09-21T06:19:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad81b6600 con 0x56548c2391b0 2024-09-21T06:19:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8110330 con 0x56548c2102f0 2024-09-21T06:19:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 0x56548c5f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 0x56548c5f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48174/0 (socket says 172.21.15.31:48174) 2024-09-21T06:19:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/701244131 learned_addr learned my addr 172.21.15.31:0/701244131 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/701244131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 msgr2=0x56548c5dbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/701244131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 msgr2=0x56548c5ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 0x56548c5ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.238+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/701244131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56548c5df790 con 0x56548c2102f0 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.239+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 0x56548c5f6320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b740 tx=0x7f9ac800f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1228e5642d6dfed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.239+0000 7f9abcff9640 1 -- 172.21.15.31:0/701244131 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8014a10 con 0x56548c2102f0 2024-09-21T06:19:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.239+0000 7f9abcff9640 1 -- 172.21.15.31:0/701244131 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ac802b070 con 0x56548c2102f0 2024-09-21T06:19:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.239+0000 7f9abcff9640 1 -- 172.21.15.31:0/701244131 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8014e60 con 0x56548c2102f0 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 -- 172.21.15.31:0/701244131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 msgr2=0x56548c5f6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 0x56548c5f6320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800b740 tx=0x7f9ac800f040 comp rx=0 tx=0).stop 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 -- 172.21.15.31:0/701244131 shutdown_connections 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2391b0 0x56548c5dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2184d0 0x56548c5ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/701244131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2102f0 0x56548c5f6320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 -- 172.21.15.31:0/701244131 >> 172.21.15.31:0/701244131 conn(0x56548c22d300 msgr2=0x56548c5eb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.240+0000 7f9ae41fe740 1 -- 172.21.15.31:0/701244131 shutdown_connections 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 -- 172.21.15.31:0/701244131 wait complete. 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 -- start start 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 Processor -- start 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2391b0 0x56548c675cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8110a00 con 0x56548c2391b0 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad8111e70 con 0x56548c2102f0 2024-09-21T06:19:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.241+0000 7f9ae41fe740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad81110d0 con 0x56548c2184d0 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ae41fe740 10 client.0 _ll_register_callbacks cb 0x7f9ad00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2391b0 0x56548c675cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5edb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55164/0 (socket says 172.21.15.31:55164) 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad6ffd640 1 -- 172.21.15.31:0/2918071273 learned_addr learned my addr 172.21.15.31:0/2918071273 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 -- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 msgr2=0x56548c5edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 -- 172.21.15.31:0/2918071273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2391b0 msgr2=0x56548c675cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2391b0 0x56548c675cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 -- 172.21.15.31:0/2918071273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56548c676210 con 0x56548c2184d0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9ad67fc640 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5ee0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56548c5ecd00 tx=0x7f9acc00de80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43ce48da2912f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.242+0000 7f9abaff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.243+0000 7f9abaff5640 1 -- 172.21.15.31:0/2918071273 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc01fe10 con 0x56548c2184d0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.243+0000 7f9abaff5640 1 -- 172.21.15.31:0/2918071273 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f9acc01d420 con 0x56548c2184d0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.243+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ac802b070 con 0x56548c2184d0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.243+0000 7f9abaff5640 1 -- 172.21.15.31:0/2918071273 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9acc01d620 con 0x56548c2184d0 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.243+0000 7f9ae41fe740 10 client.26989 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.243+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56548c677430 con 0x56548c2184d0 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9abaff5640 1 -- 172.21.15.31:0/2918071273 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9acc01dc70 con 0x56548c2184d0 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9abaff5640 1 -- 172.21.15.31:0/2918071273 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f9acc029980 con 0x56548c2184d0 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9ae41fe740 10 client.26989 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9ae41fe740 10 client.26989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9ae41fe740 20 client.26989 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1cd1c0 0x56548c679c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.244+0000 7f9ae41fe740 4 client.26989 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.245+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9ac802b070 con 0x56548c1cd1c0 2024-09-21T06:19:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.245+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1cd1c0 0x56548c679c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.245+0000 7f9abaff5640 10 client.26989 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.245+0000 7f9ad6ffd640 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1cd1c0 0x56548c679c20 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a9c00cc85886bc server_cookie=8108ad6abd41f02f in_seq=0 out_seq=0 2024-09-21T06:19:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9abaff5640 1 -- 172.21.15.31:0/2918071273 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9ac802b070 con 0x56548c1cd1c0 2024-09-21T06:19:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9abaff5640 10 client.26989 handle_command_reply: tid=0 2024-09-21T06:19:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 client.26989 shutdown 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 msgr2=0x56548c5ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5ee0c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56548c5ecd00 tx=0x7f9acc00de80 comp rx=0 tx=0).stop 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 shutdown_connections 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56548c1cd1c0 0x56548c679c20 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56548c2391b0 0x56548c675cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56548c2184d0 0x56548c5ee0c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9abaff5640 0 client.26989 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 --2- 172.21.15.31:0/2918071273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56548c2102f0 0x56548c5edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 >> 172.21.15.31:0/2918071273 conn(0x56548c22d300 msgr2=0x56548c5e0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 shutdown_connections 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 1 -- 172.21.15.31:0/2918071273 wait complete. 2024-09-21T06:19:47.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.273+0000 7f9ae41fe740 20 client.26989 trim_cache size 0 max 16384 2024-09-21T06:19:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:19:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:19:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:19:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:19:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9ac00b6ad0 msgr2=0x7f9ac00b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9ac00b6ad0 0x7f9ac00b8fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x56548c5dd630 tx=0x7f9acc00b5f0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8107bb0 msgr2=0x7f9ad806e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8107bb0 0x7f9ad806e780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0034040 tx=0x7f9ad0003f10 comp rx=0 tx=0).stop 2024-09-21T06:19:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 shutdown_connections 2024-09-21T06:19:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad8107bb0 0x7f9ad806e780 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9ac00b6ad0 0x7f9ac00b8fe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ad8103040 0x7f9ad806e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 --2- 172.21.15.31:0/4095401656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ad81026f0 0x7f9ad8074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 >> 172.21.15.31:0/4095401656 conn(0x7f9ad807a740 msgr2=0x7f9ad8105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 shutdown_connections 2024-09-21T06:19:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:19:47.274+0000 7f9add28b640 1 -- 172.21.15.31:0/4095401656 wait complete. 2024-09-21T06:19:47.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:47 smithi031 ceph-mon[31237]: pgmap v569: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.4 MiB/s wr, 771 op/s 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 -- 172.21.15.31:0/2761064943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58102730 msgr2=0x7fec58110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 --2- 172.21.15.31:0/2761064943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58102730 0x7fec58110560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fec50009ca0 tx=0x7fec5002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 -- 172.21.15.31:0/2761064943 shutdown_connections 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 --2- 172.21.15.31:0/2761064943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58102730 0x7fec58110560 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 --2- 172.21.15.31:0/2761064943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec581021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 --2- 172.21.15.31:0/2761064943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec581086f0 0x7fec58108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 -- 172.21.15.31:0/2761064943 >> 172.21.15.31:0/2761064943 conn(0x7fec58077c60 msgr2=0x7fec580780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 -- 172.21.15.31:0/2761064943 shutdown_connections 2024-09-21T06:19:47.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.442+0000 7fec5d109640 1 -- 172.21.15.31:0/2761064943 wait complete. 2024-09-21T06:19:47.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.443+0000 7fec5d109640 1 Processor -- start 2024-09-21T06:19:47.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.443+0000 7fec5d109640 1 -- start start 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5d109640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec5819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5d109640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58102730 0x7fec5819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5d109640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec581086f0 0x7fec581a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5d109640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec58113bf0 con 0x7fec58101de0 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec56d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec5819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec56d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec5819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44492/0 (socket says 172.21.15.31:44492) 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5d109640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec58113a70 con 0x7fec581086f0 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec56d9d640 1 -- 172.21.15.31:0/1267251769 learned_addr learned my addr 172.21.15.31:0/1267251769 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:47.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec58113d70 con 0x7fec58102730 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5759e640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec581086f0 0x7fec581a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.444+0000 7fec5659c640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58102730 0x7fec5819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec56d9d640 1 -- 172.21.15.31:0/1267251769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58102730 msgr2=0x7fec5819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec56d9d640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58102730 0x7fec5819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec56d9d640 1 -- 172.21.15.31:0/1267251769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec581086f0 msgr2=0x7fec581a2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec56d9d640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec581086f0 0x7fec581a2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec56d9d640 1 -- 172.21.15.31:0/1267251769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581a2750 con 0x7fec58101de0 2024-09-21T06:19:47.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec5759e640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec581086f0 0x7fec581a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.445+0000 7fec56d9d640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec5819c7e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fec4800b4a0 tx=0x7fec4800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.446+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48014070 con 0x7fec58101de0 2024-09-21T06:19:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.446+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec581a2a40 con 0x7fec58101de0 2024-09-21T06:19:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.446+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec48010ac0 con 0x7fec58101de0 2024-09-21T06:19:47.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.446+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec581a91a0 con 0x7fec58101de0 2024-09-21T06:19:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.446+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48017c30 con 0x7fec58101de0 2024-09-21T06:19:47.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.447+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58102270 con 0x7fec58101de0 2024-09-21T06:19:47.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.450+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fec48017070 con 0x7fec58101de0 2024-09-21T06:19:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.451+0000 7fec3ffff640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec400b6700 0x7fec400b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.451+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fec480e6d30 con 0x7fec58101de0 2024-09-21T06:19:47.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.451+0000 7fec5659c640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec400b6700 0x7fec400b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:47.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.452+0000 7fec5659c640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec400b6700 0x7fec400b8c10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fec58101b00 tx=0x7fec4c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:47.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.457+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec480b1670 con 0x7fec58101de0 2024-09-21T06:19:47.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:47 smithi107 ceph-mon[34380]: pgmap v569: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.4 MiB/s wr, 771 op/s 2024-09-21T06:19:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:47 smithi110 ceph-mon[34550]: pgmap v569: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.4 MiB/s wr, 771 op/s 2024-09-21T06:19:47.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.831+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec5819d910 con 0x7fec58101de0 2024-09-21T06:19:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.833+0000 7fec3ffff640 1 -- 172.21.15.31:0/1267251769 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fec480b65b0 con 0x7fec58101de0 2024-09-21T06:19:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:19:47.835 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:19:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:19:47.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.835+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec400b6700 msgr2=0x7fec400b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.835+0000 7fec5d109640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec400b6700 0x7fec400b8c10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fec58101b00 tx=0x7fec4c009110 comp rx=0 tx=0).stop 2024-09-21T06:19:47.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.835+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 msgr2=0x7fec5819c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:47.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.835+0000 7fec5d109640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec5819c7e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fec4800b4a0 tx=0x7fec4800b960 comp rx=0 tx=0).stop 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.836+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 shutdown_connections 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.836+0000 7fec5d109640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec581086f0 0x7fec581a2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.836+0000 7fec5d109640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec58102730 0x7fec5819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.836+0000 7fec5d109640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec400b6700 0x7fec400b8c10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.836+0000 7fec5d109640 1 --2- 172.21.15.31:0/1267251769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec58101de0 0x7fec5819c7e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.836+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 >> 172.21.15.31:0/1267251769 conn(0x7fec58077c60 msgr2=0x7fec581073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.837+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 shutdown_connections 2024-09-21T06:19:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:47.837+0000 7fec5d109640 1 -- 172.21.15.31:0/1267251769 wait complete. 2024-09-21T06:19:48.566 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:48 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1267251769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:48.609 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:19:48.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:48 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1267251769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:48.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:48 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1267251769' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:19:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:49.497 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:49 smithi031 ceph-mon[31237]: pgmap v570: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 634 op/s 2024-09-21T06:19:49.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 -- 172.21.15.31:0/1621477877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10381026d0 msgr2=0x7f1038102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 --2- 172.21.15.31:0/1621477877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10381026d0 0x7f1038102ae0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1030009ca0 tx=0x7f103002a9e0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 -- 172.21.15.31:0/1621477877 shutdown_connections 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 --2- 172.21.15.31:0/1621477877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038110280 0x7f10381126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 --2- 172.21.15.31:0/1621477877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1038103020 0x7f103810fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 --2- 172.21.15.31:0/1621477877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10381026d0 0x7f1038102ae0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 -- 172.21.15.31:0/1621477877 >> 172.21.15.31:0/1621477877 conn(0x7f1038079760 msgr2=0x7f1038079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 -- 172.21.15.31:0/1621477877 shutdown_connections 2024-09-21T06:19:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.647+0000 7f103cd72640 1 -- 172.21.15.31:0/1621477877 wait complete. 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.648+0000 7f103cd72640 1 Processor -- start 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 -- start start 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10381026d0 0x7f103819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038103020 0x7f103819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 0x7f10381a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1038115c90 con 0x7f10381026d0 2024-09-21T06:19:49.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1038115b10 con 0x7f1038103020 2024-09-21T06:19:49.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.649+0000 7f103cd72640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1038115e10 con 0x7f1038110280 2024-09-21T06:19:49.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.651+0000 7f103659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10381026d0 0x7f103819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.651+0000 7f1035d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038103020 0x7f103819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.651+0000 7f1036d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 0x7f10381a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.651+0000 7f1036d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 0x7f10381a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33722/0 (socket says 172.21.15.31:33722) 2024-09-21T06:19:49.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.652+0000 7f1036d9d640 1 -- 172.21.15.31:0/2992445892 learned_addr learned my addr 172.21.15.31:0/2992445892 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.652+0000 7f1035d9b640 1 -- 172.21.15.31:0/2992445892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 msgr2=0x7f10381a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.652+0000 7f1035d9b640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 0x7f10381a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.652+0000 7f1035d9b640 1 -- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10381026d0 msgr2=0x7f103819c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f1035d9b640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10381026d0 0x7f103819c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f1035d9b640 1 -- 172.21.15.31:0/2992445892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10381a23a0 con 0x7f1038103020 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f1036d9d640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 0x7f10381a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f103659c640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10381026d0 0x7f103819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f1035d9b640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038103020 0x7f103819c9d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1028005fd0 tx=0x7f102800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:49.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f10277fe640 1 -- 172.21.15.31:0/2992445892 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028009c30 con 0x7f1038103020 2024-09-21T06:19:49.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f10277fe640 1 -- 172.21.15.31:0/2992445892 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1028009dd0 con 0x7f1038103020 2024-09-21T06:19:49.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f10277fe640 1 -- 172.21.15.31:0/2992445892 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028010460 con 0x7f1038103020 2024-09-21T06:19:49.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10381a2690 con 0x7f1038103020 2024-09-21T06:19:49.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.653+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10381a8df0 con 0x7f1038103020 2024-09-21T06:19:49.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.657+0000 7f10432e6740 1 Processor -- start 2024-09-21T06:19:49.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10277fe640 1 -- 172.21.15.31:0/2992445892 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1028003e80 con 0x7f1038103020 2024-09-21T06:19:49.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 -- start start 2024-09-21T06:19:49.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 0x55b73f1d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b73f14bea0 con 0x55b73ee051b0 2024-09-21T06:19:49.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b73f13b2d0 con 0x55b73ee2e070 2024-09-21T06:19:49.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f10432e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b73f13f0e0 con 0x55b73ee0d390 2024-09-21T06:19:49.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f1036d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 0x55b73f1d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f1036d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 0x55b73f1d56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42292/0 (socket says 172.21.15.31:42292) 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.658+0000 7f1036d9d640 1 -- 172.21.15.31:0/1670766277 learned_addr learned my addr 172.21.15.31:0/1670766277 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f103659c640 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f1036d9d640 1 -- 172.21.15.31:0/1670766277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 msgr2=0x55b73f1d6120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f1036d9d640 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f1036d9d640 1 -- 172.21.15.31:0/1670766277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 msgr2=0x55b73f1d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f1036d9d640 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f1036d9d640 1 -- 172.21.15.31:0/1670766277 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73ee99650 con 0x55b73ee2e070 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f10277fe640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f10200b6700 0x7f10200b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.659+0000 7f103659c640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f10200b6700 0x7f10200b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.660+0000 7f1036d9d640 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 0x55b73f1d56a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f102c011570 tx=0x7f102c011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad99cecce34b6d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:49.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.660+0000 7f10277fe640 1 -- 172.21.15.31:0/2992445892 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f10280e7250 con 0x7f1038103020 2024-09-21T06:19:49.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.660+0000 7f1024ff9640 1 -- 172.21.15.31:0/1670766277 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c01ab70 con 0x55b73ee2e070 2024-09-21T06:19:49.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.660+0000 7f103659c640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f10200b6700 0x7f10200b8c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1030009a60 tx=0x7f103003b040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:49.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.660+0000 7f1024ff9640 1 -- 172.21.15.31:0/1670766277 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f102c01e070 con 0x55b73ee2e070 2024-09-21T06:19:49.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.660+0000 7f1024ff9640 1 -- 172.21.15.31:0/1670766277 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c022670 con 0x55b73ee2e070 2024-09-21T06:19:49.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 -- 172.21.15.31:0/1670766277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 msgr2=0x55b73f1d56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 0x55b73f1d56a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f102c011570 tx=0x7f102c011a30 comp rx=0 tx=0).stop 2024-09-21T06:19:49.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 -- 172.21.15.31:0/1670766277 shutdown_connections 2024-09-21T06:19:49.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 --2- 172.21.15.31:0/1670766277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee2e070 0x55b73f1d56a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.661+0000 7f10432e6740 1 -- 172.21.15.31:0/1670766277 >> 172.21.15.31:0/1670766277 conn(0x55b73ee22460 msgr2=0x55b73f1d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:49.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.663+0000 7f10432e6740 1 -- 172.21.15.31:0/1670766277 shutdown_connections 2024-09-21T06:19:49.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.663+0000 7f10432e6740 1 -- 172.21.15.31:0/1670766277 wait complete. 2024-09-21T06:19:49.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.664+0000 7f10432e6740 1 -- start start 2024-09-21T06:19:49.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.664+0000 7f10432e6740 1 Processor -- start 2024-09-21T06:19:49.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee051b0 0x55b73f1e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee2e070 0x55b73f1ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b73ef9c700 con 0x55b73ee2e070 2024-09-21T06:19:49.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b73f0d6160 con 0x55b73ee051b0 2024-09-21T06:19:49.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b73f1e3e80 con 0x55b73ee0d390 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f10432e6740 10 client.0 _ll_register_callbacks cb 0x55b73f0ef9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f1035d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f103659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee051b0 0x55b73f1e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f1036d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee2e070 0x55b73f1ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f1035d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1e13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33740/0 (socket says 172.21.15.31:33740) 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f1035d9b640 1 -- 172.21.15.31:0/573709980 learned_addr learned my addr 172.21.15.31:0/573709980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:49.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f1036d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee2e070 0x55b73f1ed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36260/0 (socket says 172.21.15.31:36260) 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f103659c640 1 -- 172.21.15.31:0/573709980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 msgr2=0x55b73f1e13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f103659c640 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1e13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f103659c640 1 -- 172.21.15.31:0/573709980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee2e070 msgr2=0x55b73f1ed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f103659c640 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee2e070 0x55b73f1ed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.665+0000 7f103659c640 1 -- 172.21.15.31:0/573709980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73f103c90 con 0x55b73ee051b0 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.666+0000 7f103659c640 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee051b0 0x55b73f1e0e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55b73f1d4320 tx=0x7f103002a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f05f583445c0675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.666+0000 7f101dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.666+0000 7f101dffb640 1 -- 172.21.15.31:0/573709980 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030053060 con 0x55b73ee051b0 2024-09-21T06:19:49.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.666+0000 7f101dffb640 1 -- 172.21.15.31:0/573709980 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f103002f640 con 0x55b73ee051b0 2024-09-21T06:19:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.666+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f102c01e070 con 0x55b73ee051b0 2024-09-21T06:19:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.667+0000 7f101dffb640 1 -- 172.21.15.31:0/573709980 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1030033040 con 0x55b73ee051b0 2024-09-21T06:19:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.667+0000 7f10432e6740 10 client.26954 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.667+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b73f1efc50 con 0x55b73ee051b0 2024-09-21T06:19:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.667+0000 7f101dffb640 1 -- 172.21.15.31:0/573709980 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1030004820 con 0x55b73ee051b0 2024-09-21T06:19:49.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.667+0000 7f101dffb640 1 -- 172.21.15.31:0/573709980 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1030038930 con 0x55b73ee051b0 2024-09-21T06:19:49.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.669+0000 7f10432e6740 10 client.26954 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:49.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.669+0000 7f10432e6740 10 client.26954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:49.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.669+0000 7f10432e6740 20 client.26954 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:49.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.669+0000 7f10432e6740 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73edec9f0 0x55b73f1f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.669+0000 7f10432e6740 4 client.26954 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:49.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.669+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f102c01e070 con 0x55b73edec9f0 2024-09-21T06:19:49.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.673+0000 7f1035d9b640 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73edec9f0 0x55b73f1f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.676+0000 7f1035d9b640 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73edec9f0 0x55b73f1f3070 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8165d84d05223d88 server_cookie=900294f2f2f9c681 in_seq=0 out_seq=0 2024-09-21T06:19:49.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.676+0000 7f101dffb640 10 client.26954 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:49.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.677+0000 7f101dffb640 1 -- 172.21.15.31:0/573709980 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f102c01e070 con 0x55b73edec9f0 2024-09-21T06:19:49.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.677+0000 7f101dffb640 10 client.26954 handle_command_reply: tid=0 2024-09-21T06:19:49.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.677+0000 7f10432e6740 1 client.26954 shutdown 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.677+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee051b0 msgr2=0x55b73f1e0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee051b0 0x55b73f1e0e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55b73f1d4320 tx=0x7f103002a9e0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 shutdown_connections 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73edec9f0 0x55b73f1f3070 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f101dffb640 0 client.26954 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee2e070 0x55b73f1ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1e13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 --2- 172.21.15.31:0/573709980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee051b0 0x55b73f1e0e80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 >> 172.21.15.31:0/573709980 conn(0x55b73ee22460 msgr2=0x55b73f1d8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:49.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 shutdown_connections 2024-09-21T06:19:49.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.678+0000 7f10432e6740 1 -- 172.21.15.31:0/573709980 wait complete. 2024-09-21T06:19:49.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.680+0000 7f10432e6740 20 client.26954 trim_cache size 0 max 16384 2024-09-21T06:19:49.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.707+0000 7f10432e6740 1 Processor -- start 2024-09-21T06:19:49.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 -- start start 2024-09-21T06:19:49.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 0x55b73f1d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10381ab410 con 0x55b73ee051b0 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10381adc50 con 0x55b73edec9f0 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.708+0000 7f10432e6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10381aba60 con 0x55b73ee0d390 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.709+0000 7f103659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 0x55b73f1d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.709+0000 7f103659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 0x55b73f1d8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42318/0 (socket says 172.21.15.31:42318) 2024-09-21T06:19:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.709+0000 7f103659c640 1 -- 172.21.15.31:0/3138626560 learned_addr learned my addr 172.21.15.31:0/3138626560 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f1035d9b640 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f1036d9d640 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f103659c640 1 -- 172.21.15.31:0/3138626560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 msgr2=0x55b73f1de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f103659c640 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f103659c640 1 -- 172.21.15.31:0/3138626560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 msgr2=0x55b73f1de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f103659c640 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f103659c640 1 -- 172.21.15.31:0/3138626560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b73f1d7c80 con 0x55b73edec9f0 2024-09-21T06:19:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f103659c640 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 0x55b73f1d8eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1030009bf0 tx=0x7f103002e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ada887c77058376 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:49.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.710+0000 7f101ffff640 1 -- 172.21.15.31:0/3138626560 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030053030 con 0x55b73edec9f0 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.711+0000 7f101ffff640 1 -- 172.21.15.31:0/3138626560 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103004ec80 con 0x55b73edec9f0 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.711+0000 7f101ffff640 1 -- 172.21.15.31:0/3138626560 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103005d080 con 0x55b73edec9f0 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 -- 172.21.15.31:0/3138626560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 msgr2=0x55b73f1d8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 0x55b73f1d8eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1030009bf0 tx=0x7f103002e5a0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 -- 172.21.15.31:0/3138626560 shutdown_connections 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73ee0d390 0x55b73f1de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 --2- 172.21.15.31:0/3138626560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73edec9f0 0x55b73f1d8eb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 -- 172.21.15.31:0/3138626560 >> 172.21.15.31:0/3138626560 conn(0x55b73ee22460 msgr2=0x55b73f1d87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 -- 172.21.15.31:0/3138626560 shutdown_connections 2024-09-21T06:19:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.712+0000 7f10432e6740 1 -- 172.21.15.31:0/3138626560 wait complete. 2024-09-21T06:19:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.713+0000 7f10432e6740 1 -- start start 2024-09-21T06:19:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.714+0000 7f10432e6740 1 Processor -- start 2024-09-21T06:19:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.714+0000 7f10432e6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 0x55b73f1e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f10432e6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f10432e6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee0d390 0x55b73f25ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f103659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 0x55b73f1e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f103659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 0x55b73f1e5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33746/0 (socket says 172.21.15.31:33746) 2024-09-21T06:19:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f103659c640 1 -- 172.21.15.31:0/3355734324 learned_addr learned my addr 172.21.15.31:0/3355734324 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10381ac110 con 0x55b73ee051b0 2024-09-21T06:19:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10381ad580 con 0x55b73ee0d390 2024-09-21T06:19:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10381ac7e0 con 0x55b73edec9f0 2024-09-21T06:19:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f10432e6740 10 client.0 _ll_register_callbacks cb 0x7f1030009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f1036d9d640 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee0d390 0x55b73f25ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.715+0000 7f103659c640 1 -- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee0d390 msgr2=0x55b73f25ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f103659c640 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee0d390 0x55b73f25ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f103659c640 1 -- 172.21.15.31:0/3355734324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 msgr2=0x55b73f1e5d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f103659c640 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1e5d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f103659c640 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73f25cf40 con 0x55b73edec9f0 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f103659c640 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 0x55b73f1e5840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f103002af00 tx=0x7f1030005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a45ee16a90c95e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f101dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.716+0000 7f101dffb640 1 -- 172.21.15.31:0/3355734324 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030065070 con 0x55b73edec9f0 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.717+0000 7f101dffb640 1 -- 172.21.15.31:0/3355734324 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f10300388e0 con 0x55b73edec9f0 2024-09-21T06:19:49.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.717+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103004ec80 con 0x55b73edec9f0 2024-09-21T06:19:49.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.717+0000 7f101dffb640 1 -- 172.21.15.31:0/3355734324 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f103006e050 con 0x55b73edec9f0 2024-09-21T06:19:49.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.718+0000 7f10432e6740 10 client.27007 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:49.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.718+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b73f25e1a0 con 0x55b73edec9f0 2024-09-21T06:19:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f101dffb640 1 -- 172.21.15.31:0/3355734324 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103004ec80 con 0x55b73edec9f0 2024-09-21T06:19:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f101dffb640 1 -- 172.21.15.31:0/3355734324 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f103002fc10 con 0x55b73edec9f0 2024-09-21T06:19:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f10432e6740 10 client.27007 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f10432e6740 10 client.27007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f10432e6740 20 client.27007 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f10432e6740 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73ee2e070 0x55b73f260860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f10432e6740 4 client.27007 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:19:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.719+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b73f14ddf0 con 0x55b73ee2e070 2024-09-21T06:19:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.720+0000 7f1035d9b640 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73ee2e070 0x55b73f260860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.720+0000 7f1035d9b640 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73ee2e070 0x55b73f260860 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfdc124e5311a4b server_cookie=e74bf8cdbc4ad34a in_seq=0 out_seq=0 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.720+0000 7f101dffb640 10 client.27007 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f101dffb640 1 -- 172.21.15.31:0/3355734324 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b73f14ddf0 con 0x55b73ee2e070 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f101dffb640 10 client.27007 handle_command_reply: tid=0 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 client.27007 shutdown 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 msgr2=0x55b73f1e5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 0x55b73f1e5840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f103002af00 tx=0x7f1030005dc0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 shutdown_connections 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b73ee2e070 0x55b73f260860 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b73ee0d390 0x55b73f25ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b73ee051b0 0x55b73f1e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 --2- 172.21.15.31:0/3355734324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b73edec9f0 0x55b73f1e5840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 >> 172.21.15.31:0/3355734324 conn(0x55b73ee22460 msgr2=0x55b73f1d66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f101dffb640 0 client.27007 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 shutdown_connections 2024-09-21T06:19:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.721+0000 7f10432e6740 1 -- 172.21.15.31:0/3355734324 wait complete. 2024-09-21T06:19:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.722+0000 7f10432e6740 20 client.27007 trim_cache size 0 max 16384 2024-09-21T06:19:49.726 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.724+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f10200b6700 msgr2=0x7f10200b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.724+0000 7f103cd72640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f10200b6700 0x7f10200b8c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1030009a60 tx=0x7f103003b040 comp rx=0 tx=0).stop 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.724+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038103020 msgr2=0x7f103819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.724+0000 7f103cd72640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038103020 0x7f103819c9d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1028005fd0 tx=0x7f102800dde0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.724+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 shutdown_connections 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1038110280 0x7f10381a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1038103020 0x7f103819c9d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f10200b6700 0x7f10200b8c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 --2- 172.21.15.31:0/2992445892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10381026d0 0x7f103819c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 >> 172.21.15.31:0/2992445892 conn(0x7f1038079760 msgr2=0x7f1038101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 shutdown_connections 2024-09-21T06:19:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:49.725+0000 7f103cd72640 1 -- 172.21.15.31:0/2992445892 wait complete. 2024-09-21T06:19:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:49 smithi110 ceph-mon[34550]: pgmap v570: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 634 op/s 2024-09-21T06:19:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:49 smithi107 ceph-mon[34380]: pgmap v570: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 634 op/s 2024-09-21T06:19:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 490.0 s, 1836.6 tps, lat 54.381 ms stddev 61.351 2024-09-21T06:19:51.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:19:51.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:19:51.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:51 smithi031 ceph-mon[31237]: pgmap v571: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 625 op/s 2024-09-21T06:19:51.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:19:51.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:19:51.602 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:19:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:51 smithi110 ceph-mon[34550]: pgmap v571: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 625 op/s 2024-09-21T06:19:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:19:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:19:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:19:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:51 smithi107 ceph-mon[34380]: pgmap v571: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 625 op/s 2024-09-21T06:19:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:19:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:19:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:19:51.981 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:52.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:52 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:19:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:52 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:19:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:52 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:19:53.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 -- 172.21.15.31:0/3066519318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a811cc30 msgr2=0x7fe0a811f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 --2- 172.21.15.31:0/3066519318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a811cc30 0x7fe0a811f140 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe098009900 tx=0x7fe09802a440 comp rx=0 tx=0).stop 2024-09-21T06:19:53.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 -- 172.21.15.31:0/3066519318 shutdown_connections 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 --2- 172.21.15.31:0/3066519318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a811cc30 0x7fe0a811f140 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 --2- 172.21.15.31:0/3066519318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a810a9e0 0x7fe0a810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 --2- 172.21.15.31:0/3066519318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0a8111f40 0x7fe0a810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.877+0000 7fe0ae740640 1 -- 172.21.15.31:0/3066519318 >> 172.21.15.31:0/3066519318 conn(0x7fe0a80712f0 msgr2=0x7fe0a8073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.882+0000 7fe0ae740640 1 -- 172.21.15.31:0/3066519318 shutdown_connections 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.883+0000 7fe0ae740640 1 -- 172.21.15.31:0/3066519318 wait complete. 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 Processor -- start 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 -- start start 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0a810a9e0 0x7fe0a811c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a8111f40 0x7fe0a811c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a8115330 0x7fe0a8115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8122910 con 0x7fe0a8111f40 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8122790 con 0x7fe0a8115330 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.884+0000 7fe0ae740640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8122a90 con 0x7fe0a810a9e0 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.885+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a8111f40 0x7fe0a811c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.885+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a8111f40 0x7fe0a811c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36282/0 (socket says 172.21.15.31:36282) 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.885+0000 7fe0a77fe640 1 -- 172.21.15.31:0/839323145 learned_addr learned my addr 172.21.15.31:0/839323145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a8115330 0x7fe0a8115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0a810a9e0 0x7fe0a811c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 -- 172.21.15.31:0/839323145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0a810a9e0 msgr2=0x7fe0a811c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0a810a9e0 0x7fe0a811c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 -- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a8111f40 msgr2=0x7fe0a811c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a8111f40 0x7fe0a811c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 -- 172.21.15.31:0/839323145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a8116050 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.886+0000 7fe0accdd640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a8115330 0x7fe0a8115830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe098033040 tx=0x7fe098031ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.888+0000 7fe0a57fa640 1 -- 172.21.15.31:0/839323145 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098040030 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.888+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a811acc0 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.888+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a811b1a0 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.888+0000 7fe0a57fa640 1 -- 172.21.15.31:0/839323145 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe098031e30 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.889+0000 7fe0a57fa640 1 -- 172.21.15.31:0/839323145 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098044850 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.893+0000 7fe0a57fa640 1 -- 172.21.15.31:0/839323145 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe098043030 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.894+0000 7fe0a57fa640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0900b66f0 0x7fe0900b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.895+0000 7fe0a57fa640 1 -- 172.21.15.31:0/839323145 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fe098107ad0 con 0x7fe0a8115330 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.896+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0900b66f0 0x7fe0900b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.900+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0900b66f0 0x7fe0900b8c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00bbe0 tx=0x7fe09c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.903+0000 7fe0b4cb4740 1 Processor -- start 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 -- start start 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141fd350 0x5607145a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145a4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141dc670 0x5607145a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56071451ae60 con 0x5607141dc670 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56071450a290 con 0x5607141d4490 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.905+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56071450e0a0 con 0x5607141fd350 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0accdd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141dc670 0x5607145a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42326/0 (socket says 172.21.15.31:42326) 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3253497440 learned_addr learned my addr 172.21.15.31:0/3253497440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141fd350 0x5607145a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3253497440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141fd350 msgr2=0x5607145a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141fd350 0x5607145a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3253497440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141dc670 msgr2=0x5607145a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141dc670 0x5607145a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.906+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3253497440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560714268930 con 0x5607141d4490 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.907+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145a4970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000e240 tx=0x7fe0a0009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d5c082594fdd28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.908+0000 7fe08e7fc640 1 -- 172.21.15.31:0/3253497440 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0018020 con 0x5607141d4490 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.908+0000 7fe08e7fc640 1 -- 172.21.15.31:0/3253497440 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a000ee60 con 0x5607141d4490 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.908+0000 7fe08e7fc640 1 -- 172.21.15.31:0/3253497440 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0013bd0 con 0x5607141d4490 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.909+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3253497440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 msgr2=0x5607145a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.909+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145a4970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000e240 tx=0x7fe0a0009730 comp rx=0 tx=0).stop 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.911+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3253497440 shutdown_connections 2024-09-21T06:19:53.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.911+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141dc670 0x5607145a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.911+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145a4970 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.911+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3253497440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141fd350 0x5607145a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.911+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3253497440 >> 172.21.15.31:0/3253497440 conn(0x5607141f1740 msgr2=0x5607145a2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.911+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3253497440 shutdown_connections 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.912+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3253497440 wait complete. 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.912+0000 7fe0b4cb4740 1 -- start start 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.912+0000 7fe0b4cb4740 1 Processor -- start 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145b0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141fd350 0x5607145bbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56071436b890 con 0x5607141fd350 2024-09-21T06:19:53.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607144a4d40 con 0x5607141d4490 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5607145b2c10 con 0x5607141dc670 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.914+0000 7fe0a7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.914+0000 7fe0a7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42332/0 (socket says 172.21.15.31:42332) 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.914+0000 7fe0a7fff640 1 -- 172.21.15.31:0/3781693959 learned_addr learned my addr 172.21.15.31:0/3781693959 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.914+0000 7fe0accdd640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141fd350 0x5607145bbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.913+0000 7fe0b4cb4740 10 client.0 _ll_register_callbacks cb 0x560714508c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.917+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.917+0000 7fe0accdd640 1 -- 172.21.15.31:0/3781693959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 msgr2=0x5607145b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.917+0000 7fe0accdd640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.917+0000 7fe0accdd640 1 -- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 msgr2=0x5607145b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.917+0000 7fe0accdd640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.917+0000 7fe0accdd640 1 -- 172.21.15.31:0/3781693959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607145bc520 con 0x5607141fd350 2024-09-21T06:19:53.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.918+0000 7fe0accdd640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141fd350 0x5607145bbf90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe098105230 tx=0x7fe0980d7280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0db2d2ec77cdf30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.918+0000 7fe0877fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.919+0000 7fe0877fe640 1 -- 172.21.15.31:0/3781693959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980b7030 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.919+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0a000ee60 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.919+0000 7fe0877fe640 1 -- 172.21.15.31:0/3781693959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fe0980cce50 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.919+0000 7fe0877fe640 1 -- 172.21.15.31:0/3781693959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe0980d2070 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.920+0000 7fe0b4cb4740 10 client.17205 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.920+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5607145bee30 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0877fe640 1 -- 172.21.15.31:0/3781693959 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a000ee60 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0877fe640 1 -- 172.21.15.31:0/3781693959 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe0980c9330 con 0x5607141fd350 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0b4cb4740 10 client.17205 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0b4cb4740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0b4cb4740 20 client.17205 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:53.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5607141bbcd0 0x5607145c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0b4cb4740 4 client.17205 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56071451cdb0 con 0x5607141bbcd0 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.923+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5607141bbcd0 0x5607145c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.924+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5607141bbcd0 0x5607145c2600 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aecbae574119aaa2 server_cookie=82d906e2e1453d6 in_seq=0 out_seq=0 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.924+0000 7fe0877fe640 10 client.17205 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.925+0000 7fe0877fe640 1 -- 172.21.15.31:0/3781693959 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56071451cdb0 con 0x5607141bbcd0 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.925+0000 7fe0877fe640 10 client.17205 handle_command_reply: tid=0 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.925+0000 7fe0b4cb4740 1 client.17205 shutdown 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.926+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141fd350 msgr2=0x5607145bbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.926+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141fd350 0x5607145bbf90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe098105230 tx=0x7fe0980d7280 comp rx=0 tx=0).stop 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 shutdown_connections 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5607141bbcd0 0x5607145c2600 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141fd350 0x5607145bbf90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3781693959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141d4490 0x5607145b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 >> 172.21.15.31:0/3781693959 conn(0x5607141f1740 msgr2=0x5607145a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0877fe640 0 client.17205 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 shutdown_connections 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3781693959 wait complete. 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.930+0000 7fe0b4cb4740 20 client.17205 trim_cache size 0 max 16384 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 Processor -- start 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 -- start start 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145bbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe09400c8e0 con 0x5607141d4490 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe09400f140 con 0x5607141fd350 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0b4cb4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe09400cf30 con 0x5607141dc670 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.973+0000 7fe0a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145bbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145a7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33788/0 (socket says 172.21.15.31:33788) 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3318610576 learned_addr learned my addr 172.21.15.31:0/3318610576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145bbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36302/0 (socket says 172.21.15.31:36302) 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3318610576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 msgr2=0x5607145a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0accdd640 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3318610576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 msgr2=0x5607145bbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145bbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 -- 172.21.15.31:0/3318610576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607145ab6b0 con 0x5607141dc670 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145a7530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009bc0 tx=0x7fe0a00137b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f0b0b55601b83d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.974+0000 7fe08dffb640 1 -- 172.21.15.31:0/3318610576 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a002c070 con 0x5607141dc670 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.975+0000 7fe08dffb640 1 -- 172.21.15.31:0/3318610576 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a00135f0 con 0x5607141dc670 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.975+0000 7fe08dffb640 1 -- 172.21.15.31:0/3318610576 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0018030 con 0x5607141dc670 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.976+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3318610576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 msgr2=0x5607145a7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.976+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145a7530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009bc0 tx=0x7fe0a00137b0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3318610576 shutdown_connections 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145a7530 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/3318610576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145bbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3318610576 >> 172.21.15.31:0/3318610576 conn(0x5607141f1740 msgr2=0x5607145ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3318610576 shutdown_connections 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/3318610576 wait complete. 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 -- start start 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.977+0000 7fe0b4cb4740 1 Processor -- start 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36312/0 (socket says 172.21.15.31:36312) 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0a7fff640 1 -- 172.21.15.31:0/15277540 learned_addr learned my addr 172.21.15.31:0/15277540 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:53.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe09400d600 con 0x5607141d4490 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe09400ea70 con 0x5607141fd350 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe09400dcd0 con 0x5607141dc670 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0accdd640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.978+0000 7fe0b4cb4740 10 client.0 _ll_register_callbacks cb 0x7fe098009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0a7fff640 1 -- 172.21.15.31:0/15277540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 msgr2=0x5607145b8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0a7fff640 1 -- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 msgr2=0x5607145b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0a7fff640 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607145c2b30 con 0x5607141d4490 2024-09-21T06:19:53.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0a7fff640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145b7f00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe09c004910 tx=0x7fe09c013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25062258d784306 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0877fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.979+0000 7fe0877fe640 1 -- 172.21.15.31:0/15277540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c020250 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.980+0000 7fe0877fe640 1 -- 172.21.15.31:0/15277540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fe09c0205e0 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.980+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560714518860 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.980+0000 7fe0877fe640 1 -- 172.21.15.31:0/15277540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe09c005750 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.980+0000 7fe0b4cb4740 10 client.17211 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.980+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56071462d200 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0877fe640 1 -- 172.21.15.31:0/15277540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560714518860 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0877fe640 1 -- 172.21.15.31:0/15277540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe09c011880 con 0x5607141d4490 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0b4cb4740 10 client.17211 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0b4cb4740 10 client.17211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0b4cb4740 20 client.17211 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560714191080 0x56071462fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:53.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0b4cb4740 4 client.17211 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560714515c90 con 0x560714191080 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560714191080 0x56071462fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.981+0000 7fe0877fe640 10 client.17211 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.982+0000 7fe0a77fe640 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560714191080 0x56071462fa70 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6ef112f36f28c1 server_cookie=c0d51426c6eadcea in_seq=0 out_seq=0 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0877fe640 1 -- 172.21.15.31:0/15277540 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560714515c90 con 0x560714191080 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0877fe640 10 client.17211 handle_command_reply: tid=0 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 client.17211 shutdown 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 msgr2=0x5607145b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145b7f00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe09c004910 tx=0x7fe09c013070 comp rx=0 tx=0).stop 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 shutdown_connections 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560714191080 0x56071462fa70 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607141fd350 0x5607145b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "0176f173-9d5c-4b70-866d-7257c518cda6", 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607141dc670 0x5607145b8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 --2- 172.21.15.31:0/15277540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5607141d4490 0x5607145b7f00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 >> 172.21.15.31:0/15277540 conn(0x5607141f1740 msgr2=0x5607145a1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0877fe640 0 client.17211 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 shutdown_connections 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.984+0000 7fe0b4cb4740 1 -- 172.21.15.31:0/15277540 wait complete. 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0b4cb4740 20 client.17211 trim_cache size 0 max 16384 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0900b66f0 msgr2=0x7fe0900b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0900b66f0 0x7fe0900b8c00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00bbe0 tx=0x7fe09c00b390 comp rx=0 tx=0).stop 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a8115330 msgr2=0x7fe0a8115830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a8115330 0x7fe0a8115830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe098033040 tx=0x7fe098031ab0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 shutdown_connections 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a8115330 0x7fe0a8115830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe0900b66f0 0x7fe0900b8c00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0a8111f40 0x7fe0a811c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 --2- 172.21.15.31:0/839323145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0a810a9e0 0x7fe0a811c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.985+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 >> 172.21.15.31:0/839323145 conn(0x7fe0a80712f0 msgr2=0x7fe0a8073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.986+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 shutdown_connections 2024-09-21T06:19:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:52.986+0000 7fe0ae740640 1 -- 172.21.15.31:0/839323145 wait complete. 2024-09-21T06:19:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:53 smithi031 ceph-mon[31237]: pgmap v572: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.4 MiB/s wr, 858 op/s 2024-09-21T06:19:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:53 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:19:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:53 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:53.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:53 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:19:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:53 smithi110 ceph-mon[34550]: pgmap v572: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.4 MiB/s wr, 858 op/s 2024-09-21T06:19:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:53 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:19:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:53 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:53 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:19:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:53 smithi107 ceph-mon[34380]: pgmap v572: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.4 MiB/s wr, 858 op/s 2024-09-21T06:19:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:53 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:19:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:53 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:53 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:19:54.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:19:54.522 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:19:54.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:54 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:54.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:54 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:54.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:54 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:19:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.126+0000 7fc5490c4640 1 -- 172.21.15.31:0/4125046445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544103120 msgr2=0x7fc544107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 --2- 172.21.15.31:0/4125046445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544103120 0x7fc544107650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc538009860 tx=0x7fc53802a340 comp rx=0 tx=0).stop 2024-09-21T06:19:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 -- 172.21.15.31:0/4125046445 shutdown_connections 2024-09-21T06:19:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 --2- 172.21.15.31:0/4125046445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc544107c10 0x7fc5441080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 --2- 172.21.15.31:0/4125046445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544103120 0x7fc544107650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 --2- 172.21.15.31:0/4125046445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5441027d0 0x7fc544102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 -- 172.21.15.31:0/4125046445 >> 172.21.15.31:0/4125046445 conn(0x7fc544079750 msgr2=0x7fc544079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 -- 172.21.15.31:0/4125046445 shutdown_connections 2024-09-21T06:19:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.127+0000 7fc5490c4640 1 -- 172.21.15.31:0/4125046445 wait complete. 2024-09-21T06:19:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.128+0000 7fc5490c4640 1 Processor -- start 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.128+0000 7fc5490c4640 1 -- start start 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc5490c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5441027d0 0x7fc5441a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc5490c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc544103120 0x7fc5441a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc5490c4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 0x7fc5441a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc5490c4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc54411a090 con 0x7fc544103120 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc5490c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc544119f10 con 0x7fc5441027d0 2024-09-21T06:19:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc5490c4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc54411a210 con 0x7fc544107c10 2024-09-21T06:19:55.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc54359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 0x7fc5441a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc54359e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 0x7fc5441a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33822/0 (socket says 172.21.15.31:33822) 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.129+0000 7fc54359e640 1 -- 172.21.15.31:0/3982431458 learned_addr learned my addr 172.21.15.31:0/3982431458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54259c640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc544103120 0x7fc5441a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc542d9d640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5441027d0 0x7fc5441a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54359e640 1 -- 172.21.15.31:0/3982431458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5441027d0 msgr2=0x7fc5441a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54359e640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5441027d0 0x7fc5441a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54359e640 1 -- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc544103120 msgr2=0x7fc5441a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54359e640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc544103120 0x7fc5441a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54359e640 1 -- 172.21.15.31:0/3982431458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5441a6a10 con 0x7fc544107c10 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc542d9d640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5441027d0 0x7fc5441a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc54359e640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 0x7fc5441a62e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00eaa0 tx=0x7fc53c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc52bfff640 1 -- 172.21.15.31:0/3982431458 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00cdb0 con 0x7fc544107c10 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.130+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5441a6d00 con 0x7fc544107c10 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.131+0000 7fc52bfff640 1 -- 172.21.15.31:0/3982431458 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53c005020 con 0x7fc544107c10 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.131+0000 7fc52bfff640 1 -- 172.21.15.31:0/3982431458 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0116d0 con 0x7fc544107c10 2024-09-21T06:19:55.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.131+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5441ad460 con 0x7fc544107c10 2024-09-21T06:19:55.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.133+0000 7fc54f638740 1 Processor -- start 2024-09-21T06:19:55.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc52bfff640 1 -- 172.21.15.31:0/3982431458 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc53c004b20 con 0x7fc544107c10 2024-09-21T06:19:55.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 -- start start 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75502650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75502b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b755030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556b75478e50 con 0x556b7513a390 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556b75468280 con 0x556b7515b070 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54f638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b7546c090 con 0x556b751321b0 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b755030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc542d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75502b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b755030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36346/0 (socket says 172.21.15.31:36346) 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.135+0000 7fc54259c640 1 -- 172.21.15.31:0/392748441 learned_addr learned my addr 172.21.15.31:0/392748441 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc54259c640 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75502650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc542d9d640 1 -- 172.21.15.31:0/392748441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 msgr2=0x556b75502650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc542d9d640 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75502650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc542d9d640 1 -- 172.21.15.31:0/392748441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 msgr2=0x556b755030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc542d9d640 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b755030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc542d9d640 1 -- 172.21.15.31:0/392748441 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b751c6650 con 0x556b751321b0 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc52bfff640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc52c0b6680 0x7fc52c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc54259c640 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b755030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc542d9d640 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75502b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc534005fd0 tx=0x7fc5340117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc509a0c4e240d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc52bfff640 1 -- 172.21.15.31:0/3982431458 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fc53c0e78e0 con 0x7fc544107c10 2024-09-21T06:19:55.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc54259c640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc52c0b6680 0x7fc52c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc5297fa640 1 -- 172.21.15.31:0/392748441 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004020 con 0x556b751321b0 2024-09-21T06:19:55.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc5297fa640 1 -- 172.21.15.31:0/392748441 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5340041c0 con 0x556b751321b0 2024-09-21T06:19:55.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.136+0000 7fc5297fa640 1 -- 172.21.15.31:0/392748441 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004020 con 0x556b751321b0 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 -- 172.21.15.31:0/392748441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 msgr2=0x556b75502b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54259c640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc52c0b6680 0x7fc52c0b8b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc53802cfd0 tx=0x7fc538000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75502b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc534005fd0 tx=0x7fc5340117b0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 -- 172.21.15.31:0/392748441 shutdown_connections 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b755030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75502b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 --2- 172.21.15.31:0/392748441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75502650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 -- 172.21.15.31:0/392748441 >> 172.21.15.31:0/392748441 conn(0x556b7514f460 msgr2=0x556b75500540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.138+0000 7fc54f638740 1 -- 172.21.15.31:0/392748441 shutdown_connections 2024-09-21T06:19:55.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.139+0000 7fc54f638740 1 -- 172.21.15.31:0/392748441 wait complete. 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.139+0000 7fc54f638740 1 -- start start 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.139+0000 7fc54f638740 1 Processor -- start 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b7550dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b7550e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b7550e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556b752c9700 con 0x556b7513a390 2024-09-21T06:19:55.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556b75403110 con 0x556b7515b070 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b75510e30 con 0x556b751321b0 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54f638740 10 client.0 _ll_register_callbacks cb 0x556b7541c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b7550e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b7550e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b7550e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36360/0 (socket says 172.21.15.31:36360) 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54259c640 1 -- 172.21.15.31:0/527572633 learned_addr learned my addr 172.21.15.31:0/527572633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc542d9d640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b7550dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 -- 172.21.15.31:0/527572633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 msgr2=0x556b7550dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b7550dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 -- 172.21.15.31:0/527572633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 msgr2=0x556b7550e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b7550e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 -- 172.21.15.31:0/527572633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b75430c40 con 0x556b7515b070 2024-09-21T06:19:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc542d9d640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b7550dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc54359e640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b7550e750 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0e5040 tx=0x7fc53c0ad630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2f2efac8aeb941 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.140+0000 7fc526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc526ff5640 1 -- 172.21.15.31:0/527572633 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c01f030 con 0x556b7515b070 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5340041c0 con 0x556b7515b070 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc526ff5640 1 -- 172.21.15.31:0/527572633 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fc53c0a9090 con 0x556b7515b070 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc526ff5640 1 -- 172.21.15.31:0/527572633 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc53c0ade40 con 0x556b7515b070 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc54f638740 10 client.26984 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:55.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556b7551cb20 con 0x556b7515b070 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc526ff5640 1 -- 172.21.15.31:0/527572633 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53c0b2220 con 0x556b7515b070 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.141+0000 7fc526ff5640 1 -- 172.21.15.31:0/527572633 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc53c00f040 con 0x556b7515b070 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc54f638740 10 client.26984 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc54f638740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc54f638740 20 client.26984 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc54f638740 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b751199f0 0x556b755204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc54f638740 4 client.26984 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:19:55.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc5340041c0 con 0x556b751199f0 2024-09-21T06:19:55.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.143+0000 7fc542d9d640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b751199f0 0x556b755204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.144+0000 7fc542d9d640 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b751199f0 0x556b755204b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745ea71692de8f9d server_cookie=4eaeef52c309bca7 in_seq=0 out_seq=0 2024-09-21T06:19:55.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.144+0000 7fc526ff5640 10 client.26984 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc526ff5640 1 -- 172.21.15.31:0/527572633 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc5340041c0 con 0x556b751199f0 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc526ff5640 10 client.26984 handle_command_reply: tid=0 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 client.26984 shutdown 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 msgr2=0x556b7550e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b7550e750 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0e5040 tx=0x7fc53c0ad630 comp rx=0 tx=0).stop 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 shutdown_connections 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b751199f0 0x556b755204b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b7550e750 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b7550e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc526ff5640 0 client.26984 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 --2- 172.21.15.31:0/527572633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b7550dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 >> 172.21.15.31:0/527572633 conn(0x556b7514f460 msgr2=0x556b75505830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 shutdown_connections 2024-09-21T06:19:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 1 -- 172.21.15.31:0/527572633 wait complete. 2024-09-21T06:19:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.145+0000 7fc54f638740 20 client.26984 trim_cache size 0 max 16384 2024-09-21T06:19:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.168+0000 7fc54f638740 1 Processor -- start 2024-09-21T06:19:55.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.168+0000 7fc54f638740 1 -- start start 2024-09-21T06:19:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.169+0000 7fc54f638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 0x556b75504040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.169+0000 7fc54f638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b7513a390 0x556b75519720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.169+0000 7fc54f638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7515b070 0x556b754ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.169+0000 7fc54f638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441b3cf0 con 0x556b7515b070 2024-09-21T06:19:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.169+0000 7fc54f638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441b6550 con 0x556b751321b0 2024-09-21T06:19:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.169+0000 7fc54f638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441b4340 con 0x556b7513a390 2024-09-21T06:19:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 0x556b75504040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 0x556b75504040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42394/0 (socket says 172.21.15.31:42394) 2024-09-21T06:19:55.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 -- 172.21.15.31:0/1730045233 learned_addr learned my addr 172.21.15.31:0/1730045233 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc54359e640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7515b070 0x556b754ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 -- 172.21.15.31:0/1730045233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b7513a390 msgr2=0x556b75519720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc54259c640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b7513a390 0x556b75519720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b7513a390 0x556b75519720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 -- 172.21.15.31:0/1730045233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7515b070 msgr2=0x556b754ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7515b070 0x556b754ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc542d9d640 1 -- 172.21.15.31:0/1730045233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b75504b70 con 0x556b751321b0 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc54359e640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7515b070 0x556b754ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.170+0000 7fc54259c640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b7513a390 0x556b75519720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.171+0000 7fc542d9d640 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 0x556b75504040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc534011cb0 tx=0x7fc534003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844ec5680dcb9b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.171+0000 7fc528ff9640 1 -- 172.21.15.31:0/1730045233 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53401f040 con 0x556b751321b0 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.171+0000 7fc528ff9640 1 -- 172.21.15.31:0/1730045233 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc534030070 con 0x556b751321b0 2024-09-21T06:19:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.171+0000 7fc528ff9640 1 -- 172.21.15.31:0/1730045233 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534004020 con 0x556b751321b0 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 -- 172.21.15.31:0/1730045233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 msgr2=0x556b75504040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 0x556b75504040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc534011cb0 tx=0x7fc534003de0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 -- 172.21.15.31:0/1730045233 shutdown_connections 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7515b070 0x556b754ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b7513a390 0x556b75519720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 --2- 172.21.15.31:0/1730045233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b751321b0 0x556b75504040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 -- 172.21.15.31:0/1730045233 >> 172.21.15.31:0/1730045233 conn(0x556b7514f460 msgr2=0x556b75519010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 -- 172.21.15.31:0/1730045233 shutdown_connections 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.172+0000 7fc54f638740 1 -- 172.21.15.31:0/1730045233 wait complete. 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 -- start start 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 Processor -- start 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75516740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b75511bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75512130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441b4a10 con 0x556b7513a390 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441b5e80 con 0x556b7515b070 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5441b50e0 con 0x556b751321b0 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54f638740 10 client.0 _ll_register_callbacks cb 0x7fc53c005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b75511bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc542d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75516740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b75511bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36374/0 (socket says 172.21.15.31:36374) 2024-09-21T06:19:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54259c640 1 -- 172.21.15.31:0/2441459367 learned_addr learned my addr 172.21.15.31:0/2441459367 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc542d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75516740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33850/0 (socket says 172.21.15.31:33850) 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.173+0000 7fc54359e640 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75512130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.174+0000 7fc542d9d640 1 -- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 msgr2=0x556b75512130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.174+0000 7fc542d9d640 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75512130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.174+0000 7fc542d9d640 1 -- 172.21.15.31:0/2441459367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 msgr2=0x556b75511bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.174+0000 7fc542d9d640 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b75511bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.174+0000 7fc542d9d640 1 -- 172.21.15.31:0/2441459367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b755127a0 con 0x556b751321b0 2024-09-21T06:19:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.175+0000 7fc542d9d640 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75516740 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc534009600 tx=0x7fc534004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3cfd437d3afcc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:19:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.175+0000 7fc526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:19:55.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.175+0000 7fc526ff5640 1 -- 172.21.15.31:0/2441459367 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003e50 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.175+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc534030070 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.175+0000 7fc526ff5640 1 -- 172.21.15.31:0/2441459367 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7fc53402d040 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc526ff5640 1 -- 172.21.15.31:0/2441459367 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc534003ff0 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 10 client.27037 fetch_fsmap learned FSMap version 19 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556b7558abd0 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc526ff5640 1 -- 172.21.15.31:0/2441459367 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc534030070 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc526ff5640 1 -- 172.21.15.31:0/2441459367 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc5340127f0 con 0x556b751321b0 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 10 client.27037 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 10 client.27037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 20 client.27037 populate_metadata read hostname 'smithi031' 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b750eeda0 0x556b7558d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 4 client.27037 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:19:55.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b7547ada0 con 0x556b750eeda0 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.176+0000 7fc54259c640 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b750eeda0 0x556b7558d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc526ff5640 10 client.27037 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54259c640 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b750eeda0 0x556b7558d150 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be808b5ad89c004 server_cookie=7bf1407747bb906c in_seq=0 out_seq=0 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc526ff5640 1 -- 172.21.15.31:0/2441459367 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x556b7547ada0 con 0x556b750eeda0 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc526ff5640 10 client.27037 handle_command_reply: tid=0 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 client.27037 shutdown 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 msgr2=0x556b75516740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75516740 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc534009600 tx=0x7fc534004570 comp rx=0 tx=0).stop 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 shutdown_connections 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556b750eeda0 0x556b7558d150 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556b7515b070 0x556b75512130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556b7513a390 0x556b75511bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 --2- 172.21.15.31:0/2441459367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b751321b0 0x556b75516740 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.177+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 >> 172.21.15.31:0/2441459367 conn(0x556b7514f460 msgr2=0x556b755199d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.178+0000 7fc526ff5640 0 client.27037 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.178+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 shutdown_connections 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.178+0000 7fc54f638740 1 -- 172.21.15.31:0/2441459367 wait complete. 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.178+0000 7fc54f638740 20 client.27037 trim_cache size 0 max 16384 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:19:55.183 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "0176f173-9d5c-4b70-866d-7257c518cda6", 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:19:55.185 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.187 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:19:55.188 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.189 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.190 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:19:55.191 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.192 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.194 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:19:55.195 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:19:55.197 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:19:55.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc52c0b6680 msgr2=0x7fc52c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc52c0b6680 0x7fc52c0b8b90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc53802cfd0 tx=0x7fc538000bc0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 msgr2=0x7fc5441a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:19:55.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 0x7fc5441a62e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00eaa0 tx=0x7fc53c00ef60 comp rx=0 tx=0).stop 2024-09-21T06:19:55.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 shutdown_connections 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc544107c10 0x7fc5441a62e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fc52c0b6680 0x7fc52c0b8b90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc544103120 0x7fc5441a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 --2- 172.21.15.31:0/3982431458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5441027d0 0x7fc5441a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.179+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 >> 172.21.15.31:0/3982431458 conn(0x7fc544079750 msgr2=0x7fc544105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.180+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 shutdown_connections 2024-09-21T06:19:55.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:19:55.180+0000 7fc5490c4640 1 -- 172.21.15.31:0/3982431458 wait complete. 2024-09-21T06:19:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 495.0 s, 1933.6 tps, lat 51.575 ms stddev 60.995 2024-09-21T06:19:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:55 smithi031 ceph-mon[31237]: pgmap v573: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-21T06:19:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:55 smithi110 ceph-mon[34550]: pgmap v573: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-21T06:19:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:55 smithi107 ceph-mon[34380]: pgmap v573: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.9 MiB/s wr, 606 op/s 2024-09-21T06:19:55.956 INFO:tasks.cephfs.filesystem:scrub status for tag:0176f173-9d5c-4b70-866d-7257c518cda6 - {'path': '/', 'tag': '0176f173-9d5c-4b70-866d-7257c518cda6', 'options': 'recursive,force'} 2024-09-21T06:19:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:57 smithi031 ceph-mon[31237]: pgmap v574: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.3 MiB/s wr, 843 op/s 2024-09-21T06:19:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:57 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:19:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:57 smithi110 ceph-mon[34550]: pgmap v574: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.3 MiB/s wr, 843 op/s 2024-09-21T06:19:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:57 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:19:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:57 smithi107 ceph-mon[34380]: pgmap v574: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.3 MiB/s wr, 843 op/s 2024-09-21T06:19:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:57 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:19:59.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:19:59 smithi031 ceph-mon[31237]: pgmap v575: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 6.4 MiB/s wr, 750 op/s 2024-09-21T06:19:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:19:59 smithi110 ceph-mon[34550]: pgmap v575: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 6.4 MiB/s wr, 750 op/s 2024-09-21T06:19:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:19:59 smithi107 ceph-mon[34380]: pgmap v575: 97 pgs: 97 active+clean; 4.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 6.4 MiB/s wr, 750 op/s 2024-09-21T06:20:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 500.0 s, 1684.6 tps, lat 59.146 ms stddev 62.543 2024-09-21T06:20:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 -- 172.21.15.31:0/2237428160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 msgr2=0x7f0acc10fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 --2- 172.21.15.31:0/2237428160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc10fb40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009900 tx=0x7f0abc02a410 comp rx=0 tx=0).stop 2024-09-21T06:20:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 -- 172.21.15.31:0/2237428160 shutdown_connections 2024-09-21T06:20:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 --2- 172.21.15.31:0/2237428160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 0x7f0acc1124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 --2- 172.21.15.31:0/2237428160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc10fb40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 --2- 172.21.15.31:0/2237428160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 0x7f0acc075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.423+0000 7f0ad4064640 1 -- 172.21.15.31:0/2237428160 >> 172.21.15.31:0/2237428160 conn(0x7f0acc0ff510 msgr2=0x7f0acc101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.424+0000 7f0ad4064640 1 -- 172.21.15.31:0/2237428160 shutdown_connections 2024-09-21T06:20:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.424+0000 7f0ad4064640 1 -- 172.21.15.31:0/2237428160 wait complete. 2024-09-21T06:20:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.424+0000 7f0ad4064640 1 Processor -- start 2024-09-21T06:20:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.424+0000 7f0ad4064640 1 -- start start 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad4064640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 0x7f0acc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad4064640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad4064640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 0x7f0acc1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad4064640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc115c60 con 0x7f0acc075750 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad4064640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc115ae0 con 0x7f0acc110080 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad4064640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc115de0 con 0x7f0acc0760a0 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 0x7f0acc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad15ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 0x7f0acc19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34574/0 (socket says 172.21.15.31:34574) 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad15ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48350/0 (socket says 172.21.15.31:48350) 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad1e00640 1 -- 172.21.15.31:0/362897110 learned_addr learned my addr 172.21.15.31:0/362897110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.425+0000 7f0ad2601640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 0x7f0acc1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad2601640 1 -- 172.21.15.31:0/362897110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 msgr2=0x7f0acc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad2601640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad2601640 1 -- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 msgr2=0x7f0acc19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad2601640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 0x7f0acc19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad2601640 1 -- 172.21.15.31:0/362897110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc1a25f0 con 0x7f0acc110080 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad2601640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 0x7f0acc1a1ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009fd0 tx=0x7f0ac400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ac2ffd640 1 -- 172.21.15.31:0/362897110 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400ce20 con 0x7f0acc110080 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ac2ffd640 1 -- 172.21.15.31:0/362897110 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac4005020 con 0x7f0acc110080 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ac2ffd640 1 -- 172.21.15.31:0/362897110 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4010690 con 0x7f0acc110080 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc1a28e0 con 0x7f0acc110080 2024-09-21T06:20:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.426+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1a9070 con 0x7f0acc110080 2024-09-21T06:20:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.429+0000 7f0ac2ffd640 1 -- 172.21.15.31:0/362897110 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0ac40054d0 con 0x7f0acc110080 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.429+0000 7f0adafce740 1 Processor -- start 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.429+0000 7f0adafce740 1 -- start start 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.429+0000 7f0adafce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 0x55f6bb74d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.429+0000 7f0adafce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb74def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.429+0000 7f0adafce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb3874d0 0x55f6bb74e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0adafce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb6c26f0 con 0x55f6bb37f2f0 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0adafce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb6b1b20 con 0x55f6bb3874d0 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0adafce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb6b5930 con 0x55f6bb3a81b0 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 0x55f6bb74d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 0x55f6bb74d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48352/0 (socket says 172.21.15.31:48352) 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 -- 172.21.15.31:0/3561382152 learned_addr learned my addr 172.21.15.31:0/3561382152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb74def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb3874d0 0x55f6bb74e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 -- 172.21.15.31:0/3561382152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb3874d0 msgr2=0x55f6bb74e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb3874d0 0x55f6bb74e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 -- 172.21.15.31:0/3561382152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 msgr2=0x55f6bb74def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb74def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 -- 172.21.15.31:0/3561382152 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6bb677530 con 0x55f6bb3a81b0 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ac2ffd640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ab80b6680 0x7f0ab80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 0x55f6bb74d9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b003dfee5cb8882e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ac2ffd640 1 -- 172.21.15.31:0/362897110 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f0ac40e7de0 con 0x7f0acc110080 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.430+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ab80b6680 0x7f0ab80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.431+0000 7f0ab7fff640 1 -- 172.21.15.31:0/3561382152 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc035070 con 0x55f6bb3a81b0 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.431+0000 7f0ab7fff640 1 -- 172.21.15.31:0/3561382152 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0abc003be0 con 0x55f6bb3a81b0 2024-09-21T06:20:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.431+0000 7f0ab7fff640 1 -- 172.21.15.31:0/3561382152 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc031bb0 con 0x55f6bb3a81b0 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 -- 172.21.15.31:0/3561382152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 msgr2=0x55f6bb74d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 0x55f6bb74d9b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc005780 comp rx=0 tx=0).stop 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ab80b6680 0x7f0ab80b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55f6bb74c630 tx=0x7f0ac80091c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 -- 172.21.15.31:0/3561382152 shutdown_connections 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb3874d0 0x55f6bb74e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb74def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 --2- 172.21.15.31:0/3561382152 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb3a81b0 0x55f6bb74d9b0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc005780 comp rx=0 tx=0).stop 2024-09-21T06:20:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.433+0000 7f0adafce740 1 -- 172.21.15.31:0/3561382152 >> 172.21.15.31:0/3561382152 conn(0x55f6bb39c300 msgr2=0x55f6bb74b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.434+0000 7f0adafce740 1 -- 172.21.15.31:0/3561382152 shutdown_connections 2024-09-21T06:20:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.434+0000 7f0adafce740 1 -- 172.21.15.31:0/3561382152 wait complete. 2024-09-21T06:20:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.435+0000 7f0adafce740 1 -- start start 2024-09-21T06:20:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.435+0000 7f0adafce740 1 Processor -- start 2024-09-21T06:20:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.435+0000 7f0adafce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb37f2f0 0x55f6bb759160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.435+0000 7f0adafce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb3874d0 0x55f6bb7596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0adafce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb366d80 0x55f6bb7653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0adafce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb5e9ab0 con 0x55f6bb3874d0 2024-09-21T06:20:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0adafce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb657730 con 0x55f6bb37f2f0 2024-09-21T06:20:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0adafce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f6bb75c0d0 con 0x55f6bb366d80 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0adafce740 10 client.0 _ll_register_callbacks cb 0x55f6bb6ab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad15ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb3874d0 0x55f6bb7596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad15ff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb3874d0 0x55f6bb7596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34590/0 (socket says 172.21.15.31:34590) 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad15ff640 1 -- 172.21.15.31:0/2956565767 learned_addr learned my addr 172.21.15.31:0/2956565767 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb366d80 0x55f6bb7653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb37f2f0 0x55f6bb759160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 -- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb37f2f0 msgr2=0x55f6bb759160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb37f2f0 0x55f6bb759160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 -- 172.21.15.31:0/2956565767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb3874d0 msgr2=0x55f6bb7596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb3874d0 0x55f6bb7596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 -- 172.21.15.31:0/2956565767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bb413790 con 0x55f6bb366d80 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ad2601640 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb366d80 0x55f6bb7653c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40e5520 tx=0x7f0ac40ad450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebccd8c07c24ed27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.436+0000 7f0ab57fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2956565767 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac40b73b0 con 0x55f6bb366d80 2024-09-21T06:20:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2956565767 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f0ac40b2250 con 0x55f6bb366d80 2024-09-21T06:20:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0abc003be0 con 0x55f6bb366d80 2024-09-21T06:20:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2956565767 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0ac40b2700 con 0x55f6bb366d80 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 10 client.27049 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f6bb48c620 con 0x55f6bb366d80 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2956565767 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac40b7640 con 0x55f6bb366d80 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2956565767 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0ac40a9570 con 0x55f6bb366d80 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 10 client.27049 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 10 client.27049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 20 client.27049 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.437+0000 7f0adafce740 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb33c1c0 0x55f6bb76b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.438+0000 7f0adafce740 4 client.27049 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.438+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0abc003be0 con 0x55f6bb33c1c0 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.438+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb33c1c0 0x55f6bb76b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.438+0000 7f0ab57fa640 10 client.27049 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.438+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb33c1c0 0x55f6bb76b210 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e881bcad1910d1a3 server_cookie=dbae6a07c0b03024 in_seq=0 out_seq=0 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2956565767 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0abc003be0 con 0x55f6bb33c1c0 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0ab57fa640 10 client.27049 handle_command_reply: tid=0 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 client.27049 shutdown 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb366d80 msgr2=0x55f6bb7653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb366d80 0x55f6bb7653c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40e5520 tx=0x7f0ac40ad450 comp rx=0 tx=0).stop 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 shutdown_connections 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb33c1c0 0x55f6bb76b210 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0ab57fa640 0 client.27049 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb366d80 0x55f6bb7653c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb3874d0 0x55f6bb7596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 --2- 172.21.15.31:0/2956565767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb37f2f0 0x55f6bb759160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 >> 172.21.15.31:0/2956565767 conn(0x55f6bb39c300 msgr2=0x55f6bb74e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 shutdown_connections 2024-09-21T06:20:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 1 -- 172.21.15.31:0/2956565767 wait complete. 2024-09-21T06:20:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.439+0000 7f0adafce740 20 client.27049 trim_cache size 0 max 16384 2024-09-21T06:20:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.458+0000 7f0adafce740 1 Processor -- start 2024-09-21T06:20:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.458+0000 7f0adafce740 1 -- start start 2024-09-21T06:20:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.458+0000 7f0adafce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 0x55f6bb753180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0adafce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb7652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 0x55f6bb753180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 0x55f6bb753180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48364/0 (socket says 172.21.15.31:48364) 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0ad1e00640 1 -- 172.21.15.31:0/974162778 learned_addr learned my addr 172.21.15.31:0/974162778 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0adafce740 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb765800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1afa20 con 0x55f6bb37f2f0 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1b2280 con 0x55f6bb366d80 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1b0070 con 0x55f6bb33c1c0 2024-09-21T06:20:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.459+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb7652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad15ff640 1 -- 172.21.15.31:0/974162778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 msgr2=0x55f6bb753180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 0x55f6bb753180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad15ff640 1 -- 172.21.15.31:0/974162778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 msgr2=0x55f6bb765800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:20:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb765800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad15ff640 1 -- 172.21.15.31:0/974162778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6bb757640 con 0x55f6bb366d80 2024-09-21T06:20:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 0x55f6bb753180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb7652c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0abc02a450 tx=0x7f0abc031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1a4c7888c02ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ab77fe640 1 -- 172.21.15.31:0/974162778 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc035040 con 0x55f6bb366d80 2024-09-21T06:20:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ab77fe640 1 -- 172.21.15.31:0/974162778 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0abc0054e0 con 0x55f6bb366d80 2024-09-21T06:20:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.460+0000 7f0ab77fe640 1 -- 172.21.15.31:0/974162778 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0abc003730 con 0x55f6bb366d80 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.461+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 msgr2=0x55f6bb7652c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.461+0000 7f0adafce740 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb7652c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0abc02a450 tx=0x7f0abc031850 comp rx=0 tx=0).stop 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 shutdown_connections 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb37f2f0 0x55f6bb765800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb7652c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 --2- 172.21.15.31:0/974162778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb33c1c0 0x55f6bb753180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 >> 172.21.15.31:0/974162778 conn(0x55f6bb39c300 msgr2=0x55f6bb752a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:00.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 shutdown_connections 2024-09-21T06:20:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 -- 172.21.15.31:0/974162778 wait complete. 2024-09-21T06:20:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 -- start start 2024-09-21T06:20:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.462+0000 7f0adafce740 1 Processor -- start 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 0x55f6bb761c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb75d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb37f2f0 0x55f6bb75d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1b0740 con 0x55f6bb33c1c0 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 0x55f6bb761c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 0x55f6bb761c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34594/0 (socket says 172.21.15.31:34594) 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 -- 172.21.15.31:0/2641164336 learned_addr learned my addr 172.21.15.31:0/2641164336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1b1bb0 con 0x55f6bb366d80 2024-09-21T06:20:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc1b0e10 con 0x55f6bb37f2f0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0adafce740 10 client.0 _ll_register_callbacks cb 0x7f0ac40aa270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad2601640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb37f2f0 0x55f6bb75d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb75d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 -- 172.21.15.31:0/2641164336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb37f2f0 msgr2=0x55f6bb75d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb37f2f0 0x55f6bb75d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 -- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 msgr2=0x55f6bb75d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.463+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb75d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0ad1e00640 1 -- 172.21.15.31:0/2641164336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6bb7d4b40 con 0x55f6bb33c1c0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0ad1e00640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 0x55f6bb761c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8023850 tx=0x7f0ac8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df98f215e63f1c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0ab57fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2641164336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8014650 con 0x55f6bb33c1c0 2024-09-21T06:20:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2641164336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8499+0+0 (secure 0 0 0) 0x7f0ac80147f0 con 0x55f6bb33c1c0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0abc0054e0 con 0x55f6bb33c1c0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2641164336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0ac80149f0 con 0x55f6bb33c1c0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0adafce740 10 client.17238 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.464+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f6bb7d5c70 con 0x55f6bb33c1c0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2641164336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac801b040 con 0x55f6bb33c1c0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2641164336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0ac8020040 con 0x55f6bb33c1c0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0adafce740 10 client.17238 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0adafce740 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0adafce740 20 client.17238 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0adafce740 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb3874d0 0x55f6bb7d83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0adafce740 4 client.17238 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0abc0054e0 con 0x55f6bb3874d0 2024-09-21T06:20:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.465+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb3874d0 0x55f6bb7d83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.466+0000 7f0ab57fa640 10 client.17238 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.466+0000 7f0ad15ff640 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb3874d0 0x55f6bb7d83f0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9c0585d1ee8f4d server_cookie=fb61ac8aa58ad52c in_seq=0 out_seq=0 2024-09-21T06:20:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.483+0000 7f0ab57fa640 1 -- 172.21.15.31:0/2641164336 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0abc0054e0 con 0x55f6bb3874d0 2024-09-21T06:20:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.483+0000 7f0ab57fa640 10 client.17238 handle_command_reply: tid=0 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.483+0000 7f0adafce740 1 client.17238 shutdown 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.483+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 msgr2=0x55f6bb761c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.483+0000 7f0adafce740 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 0x55f6bb761c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8023850 tx=0x7f0ac8018000 comp rx=0 tx=0).stop 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 shutdown_connections 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f6bb3874d0 0x55f6bb7d83f0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f6bb37f2f0 0x55f6bb75d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6bb366d80 0x55f6bb75d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 --2- 172.21.15.31:0/2641164336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bb33c1c0 0x55f6bb761c80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0ab57fa640 0 client.17238 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 >> 172.21.15.31:0/2641164336 conn(0x55f6bb39c300 msgr2=0x55f6bb74e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 shutdown_connections 2024-09-21T06:20:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 1 -- 172.21.15.31:0/2641164336 wait complete. 2024-09-21T06:20:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0adafce740 20 client.17238 trim_cache size 0 max 16384 2024-09-21T06:20:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:20:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:20:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:20:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:20:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ab80b6680 msgr2=0x7f0ab80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0ad4064640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ab80b6680 0x7f0ab80b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55f6bb74c630 tx=0x7f0ac80091c0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 msgr2=0x7f0acc1a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0ad4064640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 0x7f0acc1a1ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009fd0 tx=0x7f0ac400ef10 comp rx=0 tx=0).stop 2024-09-21T06:20:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.484+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 shutdown_connections 2024-09-21T06:20:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0acc110080 0x7f0acc1a1ef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0acc0760a0 0x7f0acc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ab80b6680 0x7f0ab80b8b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 --2- 172.21.15.31:0/362897110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0acc075750 0x7f0acc19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 >> 172.21.15.31:0/362897110 conn(0x7f0acc0ff510 msgr2=0x7f0acc109c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 shutdown_connections 2024-09-21T06:20:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:00.485+0000 7f0ad4064640 1 -- 172.21.15.31:0/362897110 wait complete. 2024-09-21T06:20:00.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:00 smithi031 ceph-mon[31237]: overall HEALTH_OK 2024-09-21T06:20:00.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:00 smithi110 ceph-mon[34550]: overall HEALTH_OK 2024-09-21T06:20:00.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:00 smithi107 ceph-mon[34380]: overall HEALTH_OK 2024-09-21T06:20:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:01 smithi031 ceph-mon[31237]: pgmap v576: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 5.7 MiB/s wr, 692 op/s 2024-09-21T06:20:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:01 smithi110 ceph-mon[34550]: pgmap v576: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 5.7 MiB/s wr, 692 op/s 2024-09-21T06:20:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:01 smithi107 ceph-mon[34380]: pgmap v576: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 5.7 MiB/s wr, 692 op/s 2024-09-21T06:20:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:03 smithi031 ceph-mon[31237]: pgmap v577: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 8.4 MiB/s wr, 858 op/s 2024-09-21T06:20:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:03 smithi110 ceph-mon[34550]: pgmap v577: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 8.4 MiB/s wr, 858 op/s 2024-09-21T06:20:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:03 smithi107 ceph-mon[34380]: pgmap v577: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 8.4 MiB/s wr, 858 op/s 2024-09-21T06:20:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 505.0 s, 450.0 tps, lat 213.367 ms stddev 232.118 2024-09-21T06:20:05.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:05 smithi031 ceph-mon[31237]: pgmap v578: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-21T06:20:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:05 smithi110 ceph-mon[34550]: pgmap v578: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-21T06:20:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:05 smithi107 ceph-mon[34380]: pgmap v578: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 5.9 MiB/s wr, 596 op/s 2024-09-21T06:20:07.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:07 smithi031 ceph-mon[31237]: pgmap v579: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 745 op/s 2024-09-21T06:20:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:07 smithi110 ceph-mon[34550]: pgmap v579: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 745 op/s 2024-09-21T06:20:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:07 smithi107 ceph-mon[34380]: pgmap v579: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.7 MiB/s wr, 745 op/s 2024-09-21T06:20:08.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:08 smithi031 ceph-mon[31237]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:20:08.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:08 smithi110 ceph-mon[34550]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:20:08.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:08 smithi107 ceph-mon[34380]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:20:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:09 smithi031 ceph-mon[31237]: pgmap v581: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 531 op/s 2024-09-21T06:20:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:09 smithi110 ceph-mon[34550]: pgmap v581: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 531 op/s 2024-09-21T06:20:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:09 smithi107 ceph-mon[34380]: pgmap v581: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.7 MiB/s wr, 531 op/s 2024-09-21T06:20:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 510.0 s, 110.0 tps, lat 505.292 ms stddev 563.487 2024-09-21T06:20:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:11 smithi110 ceph-mon[34550]: pgmap v582: 97 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 94 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-09-21T06:20:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:11 smithi107 ceph-mon[34380]: pgmap v582: 97 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 94 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-09-21T06:20:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:11 smithi031 ceph-mon[31237]: pgmap v582: 97 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 94 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.0 MiB/s wr, 471 op/s 2024-09-21T06:20:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.615+0000 7f28e0362640 1 -- 172.21.15.31:0/3995608395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 msgr2=0x7f28d8102de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.615+0000 7f28e0362640 1 --2- 172.21.15.31:0/3995608395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d8102de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009900 tx=0x7f28d402a410 comp rx=0 tx=0).stop 2024-09-21T06:20:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 -- 172.21.15.31:0/3995608395 shutdown_connections 2024-09-21T06:20:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 --2- 172.21.15.31:0/3995608395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108c50 0x7f28d810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 --2- 172.21.15.31:0/3995608395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108300 0x7f28d8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 --2- 172.21.15.31:0/3995608395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d8102de0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 -- 172.21.15.31:0/3995608395 >> 172.21.15.31:0/3995608395 conn(0x7f28d807a6f0 msgr2=0x7f28d807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 -- 172.21.15.31:0/3995608395 shutdown_connections 2024-09-21T06:20:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.616+0000 7f28e0362640 1 -- 172.21.15.31:0/3995608395 wait complete. 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 Processor -- start 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 -- start start 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108300 0x7f28d81a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108c50 0x7f28d81a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8112910 con 0x7f28d81029d0 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8112790 con 0x7f28d8108300 2024-09-21T06:20:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28e0362640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8112a90 con 0x7f28d8108c50 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28dd8fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108300 0x7f28d81a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d81a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57694/0 (socket says 172.21.15.31:57694) 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28de0fe640 1 -- 172.21.15.31:0/1689467694 learned_addr learned my addr 172.21.15.31:0/1689467694 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.618+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108c50 0x7f28d81a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28dd8fd640 1 -- 172.21.15.31:0/1689467694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108c50 msgr2=0x7f28d81a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108c50 0x7f28d81a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28dd8fd640 1 -- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 msgr2=0x7f28d81a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d81a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28dd8fd640 1 -- 172.21.15.31:0/1689467694 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d81a69e0 con 0x7f28d8108300 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108c50 0x7f28d81a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108300 0x7f28d81a1050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f28d806c980 tx=0x7f28c800db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28cf7fe640 1 -- 172.21.15.31:0/1689467694 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8017070 con 0x7f28d8108300 2024-09-21T06:20:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28cf7fe640 1 -- 172.21.15.31:0/1689467694 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c8009820 con 0x7f28d8108300 2024-09-21T06:20:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d81a6cd0 con 0x7f28d8108300 2024-09-21T06:20:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28cf7fe640 1 -- 172.21.15.31:0/1689467694 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004ba0 con 0x7f28d8108300 2024-09-21T06:20:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.619+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81ad460 con 0x7f28d8108300 2024-09-21T06:20:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.622+0000 7f28e72ce740 1 Processor -- start 2024-09-21T06:20:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.622+0000 7f28e72ce740 1 -- start start 2024-09-21T06:20:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.622+0000 7f28cf7fe640 1 -- 172.21.15.31:0/1689467694 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f28c8010460 con 0x7f28d8108300 2024-09-21T06:20:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392004d0 0x558d395c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28e72ce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558d3953b6f0 con 0x558d391f82f0 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28e72ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d3952ab20 con 0x558d392211b0 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28e72ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d3952e930 con 0x558d392004d0 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28de8ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28de8ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49794/0 (socket says 172.21.15.31:49794) 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57700/0 (socket says 172.21.15.31:57700) 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28de8ff640 1 -- 172.21.15.31:0/2222328051 learned_addr learned my addr 172.21.15.31:0/2222328051 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.623+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392004d0 0x558d395c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28de0fe640 1 -- 172.21.15.31:0/2222328051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392004d0 msgr2=0x558d395c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28de0fe640 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392004d0 0x558d395c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28de0fe640 1 -- 172.21.15.31:0/2222328051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 msgr2=0x558d395c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28de0fe640 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28cf7fe640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f28c40b6680 0x7f28c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28de0fe640 1 -- 172.21.15.31:0/2222328051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d394f0530 con 0x558d391f82f0 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392004d0 0x558d395c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28cf7fe640 1 -- 172.21.15.31:0/1689467694 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8543+0+0 (secure 0 0 0) 0x7f28c80e7080 con 0x7f28d8108300 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.624+0000 7f28de0fe640 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c6ef0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f28d4004770 tx=0x7f28d4031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf6ed7af0ea08ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.625+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f28c40b6680 0x7f28c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.625+0000 7f28ccbfe640 1 -- 172.21.15.31:0/2222328051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4040070 con 0x558d391f82f0 2024-09-21T06:20:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.625+0000 7f28ccbfe640 1 -- 172.21.15.31:0/2222328051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d4004030 con 0x558d391f82f0 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.626+0000 7f28ccbfe640 1 -- 172.21.15.31:0/2222328051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4035040 con 0x558d391f82f0 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.626+0000 7f28e72ce740 1 -- 172.21.15.31:0/2222328051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 msgr2=0x558d395c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.626+0000 7f28e72ce740 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c6ef0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f28d4004770 tx=0x7f28d4031b90 comp rx=0 tx=0).stop 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.626+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f28c40b6680 0x7f28c40b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005c40 tx=0x7f28d0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 -- 172.21.15.31:0/2222328051 shutdown_connections 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392004d0 0x558d395c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c6ef0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 --2- 172.21.15.31:0/2222328051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 -- 172.21.15.31:0/2222328051 >> 172.21.15.31:0/2222328051 conn(0x558d39215300 msgr2=0x558d395c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 -- 172.21.15.31:0/2222328051 shutdown_connections 2024-09-21T06:20:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 -- 172.21.15.31:0/2222328051 wait complete. 2024-09-21T06:20:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 -- start start 2024-09-21T06:20:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.627+0000 7f28e72ce740 1 Processor -- start 2024-09-21T06:20:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d391f82f0 0x558d395d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d392004d0 0x558d395d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558d39462ab0 con 0x558d392004d0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558d394d0730 con 0x558d392211b0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d395d5170 con 0x558d391f82f0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28e72ce740 10 client.0 _ll_register_callbacks cb 0x558d395246b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28de8ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d391f82f0 0x558d395d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28de8ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395de4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:49804/0 (socket says 172.21.15.31:49804) 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28de8ff640 1 -- 172.21.15.31:0/1427723166 learned_addr learned my addr 172.21.15.31:0/1427723166 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.628+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d392004d0 0x558d395d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28de8ff640 1 -- 172.21.15.31:0/1427723166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d391f82f0 msgr2=0x558d395d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d391f82f0 0x558d395d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28de8ff640 1 -- 172.21.15.31:0/1427723166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d392004d0 msgr2=0x558d395d2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d392004d0 0x558d395d2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28de8ff640 1 -- 172.21.15.31:0/1427723166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d3928c790 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28de8ff640 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395de4b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28d0010b60 tx=0x7f28d0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5c92c0dbdea257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28c1ffb640 1 -- 172.21.15.31:0/1427723166 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d001fc00 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28c1ffb640 1 -- 172.21.15.31:0/1427723166 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8543+0+0 (secure 0 0 0) 0x7f28d0017030 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d4004030 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28c1ffb640 1 -- 172.21.15.31:0/1427723166 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f28d001fda0 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28e72ce740 10 client.27020 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.629+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558d39305620 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28c1ffb640 1 -- 172.21.15.31:0/1427723166 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d4004030 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28c1ffb640 1 -- 172.21.15.31:0/1427723166 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f28d001c5b0 con 0x558d392211b0 2024-09-21T06:20:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28e72ce740 10 client.27020 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28e72ce740 10 client.27020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28e72ce740 20 client.27020 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391dfd80 0x558d395e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28e72ce740 4 client.27020 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d3953d640 con 0x558d391dfd80 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28de0fe640 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391dfd80 0x558d395e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28c1ffb640 10 client.27020 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.630+0000 7f28de0fe640 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391dfd80 0x558d395e41f0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a052577afa93aa8c server_cookie=fef9d6fc8d3f73d0 in_seq=0 out_seq=0 2024-09-21T06:20:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28c1ffb640 1 -- 172.21.15.31:0/1427723166 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d3953d640 con 0x558d391dfd80 2024-09-21T06:20:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28c1ffb640 10 client.27020 handle_command_reply: tid=0 2024-09-21T06:20:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 client.27020 shutdown 2024-09-21T06:20:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 msgr2=0x558d395de4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395de4b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f28d0010b60 tx=0x7f28d0004ab0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 shutdown_connections 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391dfd80 0x558d395e41f0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392211b0 0x558d395de4b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d392004d0 0x558d395d2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1427723166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d391f82f0 0x558d395d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 >> 172.21.15.31:0/1427723166 conn(0x558d39215300 msgr2=0x558d395c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28c1ffb640 0 client.27020 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 shutdown_connections 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.631+0000 7f28e72ce740 1 -- 172.21.15.31:0/1427723166 wait complete. 2024-09-21T06:20:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.632+0000 7f28e72ce740 20 client.27020 trim_cache size 0 max 16384 2024-09-21T06:20:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 Processor -- start 2024-09-21T06:20:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 -- start start 2024-09-21T06:20:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d395de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d395cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b3e40 con 0x558d391f82f0 2024-09-21T06:20:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b66a0 con 0x558d392004d0 2024-09-21T06:20:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.654+0000 7f28e72ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b4490 con 0x558d392211b0 2024-09-21T06:20:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57714/0 (socket says 172.21.15.31:57714) 2024-09-21T06:20:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 -- 172.21.15.31:0/1109525508 learned_addr learned my addr 172.21.15.31:0/1109525508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 -- 172.21.15.31:0/1109525508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 msgr2=0x558d395cf210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d395cf210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 -- 172.21.15.31:0/1109525508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 msgr2=0x558d395de3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d395de3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.655+0000 7f28de0fe640 1 -- 172.21.15.31:0/1109525508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d395d3300 con 0x558d391f82f0 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.656+0000 7f28de0fe640 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c80f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009a20 tx=0x7f28d402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32864746d14aa6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.656+0000 7f28c3fff640 1 -- 172.21.15.31:0/1109525508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4040070 con 0x558d391f82f0 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.656+0000 7f28c3fff640 1 -- 172.21.15.31:0/1109525508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d40318e0 con 0x558d391f82f0 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.657+0000 7f28e72ce740 1 -- 172.21.15.31:0/1109525508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 msgr2=0x558d395c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.657+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c80f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009a20 tx=0x7f28d402a410 comp rx=0 tx=0).stop 2024-09-21T06:20:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.671+0000 7f28e72ce740 1 -- 172.21.15.31:0/1109525508 shutdown_connections 2024-09-21T06:20:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.671+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d395cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.671+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d395de3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.671+0000 7f28e72ce740 1 --2- 172.21.15.31:0/1109525508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d395c80f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.671+0000 7f28e72ce740 1 -- 172.21.15.31:0/1109525508 >> 172.21.15.31:0/1109525508 conn(0x558d39215300 msgr2=0x558d395c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.679+0000 7f28e72ce740 1 -- 172.21.15.31:0/1109525508 shutdown_connections 2024-09-21T06:20:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.683+0000 7f28e72ce740 1 -- 172.21.15.31:0/1109525508 wait complete. 2024-09-21T06:20:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.683+0000 7f28e72ce740 1 -- start start 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.683+0000 7f28e72ce740 1 Processor -- start 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d396595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d39657ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d396581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b4b60 con 0x558d391f82f0 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b5fd0 con 0x558d392004d0 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b5230 con 0x558d392211b0 2024-09-21T06:20:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.684+0000 7f28e72ce740 10 client.0 _ll_register_callbacks cb 0x7f28d0010a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.686+0000 7f28de8ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d396581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.688+0000 7f28de8ff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d396581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33724/0 (socket says 172.21.15.31:33724) 2024-09-21T06:20:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.688+0000 7f28de8ff640 1 -- 172.21.15.31:0/955066020 learned_addr learned my addr 172.21.15.31:0/955066020 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.689+0000 7f28de0fe640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d396595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.689+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d39657ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.693+0000 7f28dd8fd640 1 -- 172.21.15.31:0/955066020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 msgr2=0x558d396581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.693+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d396581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.693+0000 7f28dd8fd640 1 -- 172.21.15.31:0/955066020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 msgr2=0x558d396595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.693+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d396595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.693+0000 7f28dd8fd640 1 -- 172.21.15.31:0/955066020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d395d9410 con 0x558d392004d0 2024-09-21T06:20:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.694+0000 7f28de8ff640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d396581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.699+0000 7f28dd8fd640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d39657ac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f28c80e47c0 tx=0x7f28c80acd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22df0f6e3987e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.699+0000 7f28c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.707+0000 7f28c1ffb640 1 -- 172.21.15.31:0/955066020 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8020070 con 0x558d392004d0 2024-09-21T06:20:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.707+0000 7f28c1ffb640 1 -- 172.21.15.31:0/955066020 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8543+0+0 (secure 0 0 0) 0x7f28c80940b0 con 0x558d392004d0 2024-09-21T06:20:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.707+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d40318e0 con 0x558d392004d0 2024-09-21T06:20:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.712+0000 7f28c1ffb640 1 -- 172.21.15.31:0/955066020 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f28c8009e60 con 0x558d392004d0 2024-09-21T06:20:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.712+0000 7f28e72ce740 10 client.27023 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.712+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x558d395da6a0 con 0x558d392004d0 2024-09-21T06:20:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28c1ffb640 1 -- 172.21.15.31:0/955066020 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d40318e0 con 0x558d392004d0 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28c1ffb640 1 -- 172.21.15.31:0/955066020 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f28c80e90d0 con 0x558d392004d0 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28e72ce740 10 client.27023 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28e72ce740 10 client.27023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28e72ce740 20 client.27023 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28e72ce740 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391b51c0 0x558d39672c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28e72ce740 4 client.27023 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.713+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d395390f0 con 0x558d391b51c0 2024-09-21T06:20:13.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.714+0000 7f28de0fe640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391b51c0 0x558d39672c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.719+0000 7f28de0fe640 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391b51c0 0x558d39672c50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285b147dc5d0d178 server_cookie=e1afb1dc24d0b73a in_seq=0 out_seq=0 2024-09-21T06:20:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.720+0000 7f28c1ffb640 10 client.27023 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28c1ffb640 1 -- 172.21.15.31:0/955066020 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d395390f0 con 0x558d391b51c0 2024-09-21T06:20:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28c1ffb640 10 client.27023 handle_command_reply: tid=0 2024-09-21T06:20:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 client.27023 shutdown 2024-09-21T06:20:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 msgr2=0x558d39657ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d39657ac0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f28c80e47c0 tx=0x7f28c80acd50 comp rx=0 tx=0).stop 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 shutdown_connections 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558d391b51c0 0x558d39672c50 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d392211b0 0x558d396581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558d392004d0 0x558d39657ac0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 --2- 172.21.15.31:0/955066020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558d391f82f0 0x558d396595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28c1ffb640 0 client.27023 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 >> 172.21.15.31:0/955066020 conn(0x558d39215300 msgr2=0x558d395c79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 shutdown_connections 2024-09-21T06:20:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 1 -- 172.21.15.31:0/955066020 wait complete. 2024-09-21T06:20:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:20:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:20:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:20:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.748+0000 7f28e72ce740 20 client.27023 trim_cache size 0 max 16384 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f28c40b6680 msgr2=0x7f28c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f28c40b6680 0x7f28c40b8b90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005c40 tx=0x7f28d0009500 comp rx=0 tx=0).stop 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108300 msgr2=0x7f28d81a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108300 0x7f28d81a1050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f28d806c980 tx=0x7f28c800db30 comp rx=0 tx=0).stop 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 shutdown_connections 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f28d8108c50 0x7f28d81a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d8108300 0x7f28d81a1050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f28c40b6680 0x7f28c40b8b90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 --2- 172.21.15.31:0/1689467694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d81029d0 0x7f28d81a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 >> 172.21.15.31:0/1689467694 conn(0x7f28d807a6f0 msgr2=0x7f28d806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 shutdown_connections 2024-09-21T06:20:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:13.749+0000 7f28e0362640 1 -- 172.21.15.31:0/1689467694 wait complete. 2024-09-21T06:20:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:13 smithi110 ceph-mon[34550]: pgmap v583: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-21T06:20:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:13 smithi107 ceph-mon[34380]: pgmap v583: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-21T06:20:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:13 smithi031 ceph-mon[31237]: pgmap v583: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-21T06:20:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 515.0 s, 1326.4 tps, lat 111.695 ms stddev 493.903 2024-09-21T06:20:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:15 smithi110 ceph-mon[34550]: pgmap v584: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-21T06:20:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:15 smithi107 ceph-mon[34380]: pgmap v584: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-21T06:20:15.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:15 smithi031 ceph-mon[31237]: pgmap v584: 97 pgs: 11 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 65 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-21T06:20:17.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:17 smithi031 ceph-mon[31237]: pgmap v585: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.66k op/s 2024-09-21T06:20:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:17 smithi110 ceph-mon[34550]: pgmap v585: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.66k op/s 2024-09-21T06:20:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:17 smithi107 ceph-mon[34380]: pgmap v585: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 MiB/s wr, 1.66k op/s 2024-09-21T06:20:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:19 smithi110 ceph-mon[34550]: pgmap v586: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.63k op/s 2024-09-21T06:20:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:19 smithi107 ceph-mon[34380]: pgmap v586: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.63k op/s 2024-09-21T06:20:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:19 smithi031 ceph-mon[31237]: pgmap v586: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 MiB/s wr, 1.63k op/s 2024-09-21T06:20:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 520.0 s, 2127.2 tps, lat 46.896 ms stddev 55.460 2024-09-21T06:20:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:21 smithi110 ceph-mon[34550]: pgmap v587: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 1.47k op/s 2024-09-21T06:20:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:21 smithi107 ceph-mon[34380]: pgmap v587: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 1.47k op/s 2024-09-21T06:20:21.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:21 smithi031 ceph-mon[31237]: pgmap v587: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 1.47k op/s 2024-09-21T06:20:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:23 smithi110 ceph-mon[34550]: pgmap v588: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-09-21T06:20:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:23 smithi110 ceph-mon[34550]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:20:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:23 smithi107 ceph-mon[34380]: pgmap v588: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-09-21T06:20:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:23 smithi107 ceph-mon[34380]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:20:23.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:23 smithi031 ceph-mon[31237]: pgmap v588: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 1.79k op/s 2024-09-21T06:20:23.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:23 smithi031 ceph-mon[31237]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:20:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 525.0 s, 3242.0 tps, lat 31.207 ms stddev 36.867 2024-09-21T06:20:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:25 smithi110 ceph-mon[34550]: pgmap v590: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-21T06:20:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:25 smithi107 ceph-mon[34380]: pgmap v590: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-21T06:20:25.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:25 smithi031 ceph-mon[31237]: pgmap v590: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-21T06:20:25.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:20:26.407 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.921+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/560497606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae81027f0 msgr2=0x7f3ae8102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.921+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/560497606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae81027f0 0x7f3ae8102c00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009900 tx=0x7f3ad802a440 comp rx=0 tx=0).stop 2024-09-21T06:20:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.922+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/560497606 shutdown_connections 2024-09-21T06:20:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.922+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/560497606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae8108a70 0x7f3ae806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.922+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/560497606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108120 0x7f3ae8108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.922+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/560497606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae81027f0 0x7f3ae8102c00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.922+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/560497606 >> 172.21.15.31:0/560497606 conn(0x7f3ae80fe230 msgr2=0x7f3ae8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.924+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/560497606 shutdown_connections 2024-09-21T06:20:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.925+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/560497606 wait complete. 2024-09-21T06:20:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.926+0000 7f3aeeb9d640 1 Processor -- start 2024-09-21T06:20:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.928+0000 7f3aeeb9d640 1 -- start start 2024-09-21T06:20:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.929+0000 7f3aeeb9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae81027f0 0x7f3ae819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.929+0000 7f3aeeb9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae8108120 0x7f3ae819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.929+0000 7f3aeeb9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 0x7f3ae819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.929+0000 7f3aeeb9d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8113900 con 0x7f3ae81027f0 2024-09-21T06:20:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.929+0000 7f3aeeb9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8113780 con 0x7f3ae8108a70 2024-09-21T06:20:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.929+0000 7f3aeeb9d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae8113a80 con 0x7f3ae8108120 2024-09-21T06:20:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.931+0000 7f3aee39c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 0x7f3ae819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.931+0000 7f3aee39c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 0x7f3ae819fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41706/0 (socket says 172.21.15.31:41706) 2024-09-21T06:20:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.931+0000 7f3aee39c640 1 -- 172.21.15.31:0/2943613589 learned_addr learned my addr 172.21.15.31:0/2943613589 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.933+0000 7f3aee39c640 1 -- 172.21.15.31:0/2943613589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae8108120 msgr2=0x7f3ae819a840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:20:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.933+0000 7f3aee39c640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae8108120 0x7f3ae819a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.933+0000 7f3aee39c640 1 -- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae81027f0 msgr2=0x7f3ae819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.933+0000 7f3aee39c640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae81027f0 0x7f3ae819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.933+0000 7f3aee39c640 1 -- 172.21.15.31:0/2943613589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae81a0010 con 0x7f3ae8108a70 2024-09-21T06:20:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.933+0000 7f3aee39c640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 0x7f3ae819fad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000b970 tx=0x7f3ae000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.935+0000 7f3ae6ffd640 1 -- 172.21.15.31:0/2943613589 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0019070 con 0x7f3ae8108a70 2024-09-21T06:20:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.936+0000 7f3ae6ffd640 1 -- 172.21.15.31:0/2943613589 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ae0014440 con 0x7f3ae8108a70 2024-09-21T06:20:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.937+0000 7f3ae6ffd640 1 -- 172.21.15.31:0/2943613589 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00075b0 con 0x7f3ae8108a70 2024-09-21T06:20:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.938+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae81a0300 con 0x7f3ae8108a70 2024-09-21T06:20:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.938+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae81a07d0 con 0x7f3ae8108a70 2024-09-21T06:20:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.945+0000 7f3ae6ffd640 1 -- 172.21.15.31:0/2943613589 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3ae000cb40 con 0x7f3ae8108a70 2024-09-21T06:20:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.947+0000 7f3ae6ffd640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3ad00b66f0 0x7f3ad00b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.948+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3ad00b66f0 0x7f3ad00b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.948+0000 7f3ae6ffd640 1 -- 172.21.15.31:0/2943613589 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f3ae00e6aa0 con 0x7f3ae8108a70 2024-09-21T06:20:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.951+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3ad00b66f0 0x7f3ad00b8c00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009230 tx=0x7f3ad8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.952+0000 7f3af6e45740 1 Processor -- start 2024-09-21T06:20:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.952+0000 7f3af6e45740 1 -- start start 2024-09-21T06:20:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 0x5606db316ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf482f0 0x5606db3170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3aedb9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 0x5606db316ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3aedb9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 0x5606db316ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54516/0 (socket says 172.21.15.31:54516) 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3aedb9b640 1 -- 172.21.15.31:0/4161621665 learned_addr learned my addr 172.21.15.31:0/4161621665 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db317620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.953+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5606db28b6f0 con 0x5606daf711b0 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aed39a640 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf482f0 0x5606db3170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5606db27ab20 con 0x5606daf504d0 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5606db27e930 con 0x5606daf482f0 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db317620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 -- 172.21.15.31:0/4161621665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf482f0 msgr2=0x5606db3170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf482f0 0x5606db3170e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 -- 172.21.15.31:0/4161621665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 msgr2=0x5606db316ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 0x5606db316ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 -- 172.21.15.31:0/4161621665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606db240530 con 0x5606daf504d0 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 0x5606db316ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3aee39c640 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db317620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00e41d0 tx=0x7f3ae000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8060b7629d3fda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.954+0000 7f3acffff640 1 -- 172.21.15.31:0/4161621665 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00b13e0 con 0x5606daf504d0 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.955+0000 7f3acffff640 1 -- 172.21.15.31:0/4161621665 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ae00b6320 con 0x5606daf504d0 2024-09-21T06:20:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.955+0000 7f3acffff640 1 -- 172.21.15.31:0/4161621665 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00ab830 con 0x5606daf504d0 2024-09-21T06:20:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.956+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 msgr2=0x5606db317620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.956+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db317620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00e41d0 tx=0x7f3ae000ce30 comp rx=0 tx=0).stop 2024-09-21T06:20:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 shutdown_connections 2024-09-21T06:20:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db317620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf482f0 0x5606db3170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4161621665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf711b0 0x5606db316ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 >> 172.21.15.31:0/4161621665 conn(0x5606daf65300 msgr2=0x5606db314a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 shutdown_connections 2024-09-21T06:20:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.957+0000 7f3af6e45740 1 -- 172.21.15.31:0/4161621665 wait complete. 2024-09-21T06:20:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.958+0000 7f3af6e45740 1 -- start start 2024-09-21T06:20:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.958+0000 7f3af6e45740 1 Processor -- start 2024-09-21T06:20:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.959+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db322340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3aed39a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3aed39a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41726/0 (socket says 172.21.15.31:41726) 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3aee39c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3aee39c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43934/0 (socket says 172.21.15.31:43934) 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.960+0000 7f3aed39a640 1 -- 172.21.15.31:0/3799628012 learned_addr learned my addr 172.21.15.31:0/3799628012 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5606db1b2ab0 con 0x5606daf482f0 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5606db220730 con 0x5606daf504d0 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db322340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5606db3252c0 con 0x5606daf711b0 2024-09-21T06:20:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3aed39a640 1 -- 172.21.15.31:0/3799628012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 msgr2=0x5606db32e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3aed39a640 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3aed39a640 1 -- 172.21.15.31:0/3799628012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 msgr2=0x5606db322340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.961+0000 7f3aed39a640 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db322340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3aed39a640 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dafdc790 con 0x5606daf504d0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3aed39a640 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322880 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0038a0 tx=0x7f3adc003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b38161551c4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3acd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3af6e45740 10 client.0 _ll_register_callbacks cb 0x5606db2746b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3acd7fa640 1 -- 172.21.15.31:0/3799628012 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc011440 con 0x5606daf504d0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ae00b6320 con 0x5606daf504d0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.962+0000 7f3acd7fa640 1 -- 172.21.15.31:0/3799628012 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f3adc0115e0 con 0x5606daf504d0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3acd7fa640 1 -- 172.21.15.31:0/3799628012 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3adc0117e0 con 0x5606daf504d0 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3af6e45740 10 client.27035 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5606db055620 con 0x5606daf504d0 2024-09-21T06:20:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3acd7fa640 1 -- 172.21.15.31:0/3799628012 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f3adc011980 con 0x5606daf504d0 2024-09-21T06:20:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3af6e45740 10 client.27035 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3af6e45740 10 client.27035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3af6e45740 20 client.27035 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.963+0000 7f3af6e45740 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf2fd80 0x5606db334170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.964+0000 7f3af6e45740 4 client.27035 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.964+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ae00b6320 con 0x5606daf2fd80 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.964+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf2fd80 0x5606db334170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.964+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf2fd80 0x5606db334170 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55ce4b875e74e0e server_cookie=fc2b697b1b8a8500 in_seq=0 out_seq=0 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.964+0000 7f3acd7fa640 10 client.27035 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.965+0000 7f3acd7fa640 1 -- 172.21.15.31:0/3799628012 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3ae00b6320 con 0x5606daf2fd80 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.965+0000 7f3acd7fa640 10 client.27035 handle_command_reply: tid=0 2024-09-21T06:20:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.965+0000 7f3af6e45740 1 client.27035 shutdown 2024-09-21T06:20:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 msgr2=0x5606db322880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322880 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0038a0 tx=0x7f3adc003d60 comp rx=0 tx=0).stop 2024-09-21T06:20:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 shutdown_connections 2024-09-21T06:20:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf2fd80 0x5606db334170 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322880 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 --2- 172.21.15.31:0/3799628012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db322340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.966+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 >> 172.21.15.31:0/3799628012 conn(0x5606daf65300 msgr2=0x5606db317bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.967+0000 7f3acd7fa640 0 client.27035 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.967+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 shutdown_connections 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.967+0000 7f3af6e45740 1 -- 172.21.15.31:0/3799628012 wait complete. 2024-09-21T06:20:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.967+0000 7f3af6e45740 20 client.27035 trim_cache size 0 max 16384 2024-09-21T06:20:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 Processor -- start 2024-09-21T06:20:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 -- start start 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db324220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db3182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b3a10 con 0x5606daf482f0 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b6250 con 0x5606daf504d0 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.997+0000 7f3af6e45740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b4060 con 0x5606daf711b0 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41732/0 (socket says 172.21.15.31:41732) 2024-09-21T06:20:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 -- 172.21.15.31:0/4116457798 learned_addr learned my addr 172.21.15.31:0/4116457798 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db324220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 -- 172.21.15.31:0/4116457798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 msgr2=0x5606db3182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db3182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 -- 172.21.15.31:0/4116457798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 msgr2=0x5606db324220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db324220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 -- 172.21.15.31:0/4116457798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606db315800 con 0x5606daf504d0 2024-09-21T06:20:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.998+0000 7f3aed39a640 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3adc005880 tx=0x7f3adc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d97e84869f1823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.999+0000 7f3acf7fe640 1 -- 172.21.15.31:0/4116457798 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc011440 con 0x5606daf504d0 2024-09-21T06:20:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.999+0000 7f3acf7fe640 1 -- 172.21.15.31:0/4116457798 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3adc015030 con 0x5606daf504d0 2024-09-21T06:20:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:26.999+0000 7f3acf7fe640 1 -- 172.21.15.31:0/4116457798 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc023cf0 con 0x5606daf504d0 2024-09-21T06:20:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 -- 172.21.15.31:0/4116457798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 msgr2=0x5606db322820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3adc005880 tx=0x7f3adc013000 comp rx=0 tx=0).stop 2024-09-21T06:20:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 -- 172.21.15.31:0/4116457798 shutdown_connections 2024-09-21T06:20:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db3182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf504d0 0x5606db322820 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 --2- 172.21.15.31:0/4116457798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf482f0 0x5606db324220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.000+0000 7f3af6e45740 1 -- 172.21.15.31:0/4116457798 >> 172.21.15.31:0/4116457798 conn(0x5606daf65300 msgr2=0x5606db317bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.001+0000 7f3af6e45740 1 -- 172.21.15.31:0/4116457798 shutdown_connections 2024-09-21T06:20:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.001+0000 7f3af6e45740 1 -- 172.21.15.31:0/4116457798 wait complete. 2024-09-21T06:20:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.001+0000 7f3af6e45740 1 -- start start 2024-09-21T06:20:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.001+0000 7f3af6e45740 1 Processor -- start 2024-09-21T06:20:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 0x5606db397490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf504d0 0x5606db397a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b4710 con 0x5606daf504d0 2024-09-21T06:20:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b5b80 con 0x5606daf482f0 2024-09-21T06:20:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae81b4de0 con 0x5606daf711b0 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3af6e45740 10 client.0 _ll_register_callbacks cb 0x7f3adc004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3aedb9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 0x5606db397490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3aedb9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 0x5606db397490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41748/0 (socket says 172.21.15.31:41748) 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3aedb9b640 1 -- 172.21.15.31:0/962906894 learned_addr learned my addr 172.21.15.31:0/962906894 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3aee39c640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.002+0000 7f3aed39a640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf504d0 0x5606db397a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aedb9b640 1 -- 172.21.15.31:0/962906894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 msgr2=0x5606db32a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aedb9b640 1 -- 172.21.15.31:0/962906894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf504d0 msgr2=0x5606db397a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf504d0 0x5606db397a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aedb9b640 1 -- 172.21.15.31:0/962906894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606db32abf0 con 0x5606daf482f0 2024-09-21T06:20:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aed39a640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf504d0 0x5606db397a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3aedb9b640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 0x5606db397490 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80315d0 tx=0x7f3ad8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61b3ed9350d981 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3acd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3acd7fa640 1 -- 172.21.15.31:0/962906894 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8035040 con 0x5606daf482f0 2024-09-21T06:20:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3acd7fa640 1 -- 172.21.15.31:0/962906894 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f3ad80048c0 con 0x5606daf482f0 2024-09-21T06:20:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.003+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3adc015030 con 0x5606daf482f0 2024-09-21T06:20:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.004+0000 7f3acd7fa640 1 -- 172.21.15.31:0/962906894 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3ad8004ac0 con 0x5606daf482f0 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.004+0000 7f3af6e45740 10 client.27041 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.004+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5606db327f40 con 0x5606daf482f0 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3acd7fa640 1 -- 172.21.15.31:0/962906894 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3adc015030 con 0x5606daf482f0 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3acd7fa640 1 -- 172.21.15.31:0/962906894 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f3ad8052070 con 0x5606daf482f0 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3af6e45740 10 client.27041 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3af6e45740 10 client.27041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3af6e45740 20 client.27041 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3af6e45740 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf051c0 0x5606db3b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3af6e45740 4 client.27041 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.005+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606db28d640 con 0x5606daf051c0 2024-09-21T06:20:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.006+0000 7f3aed39a640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf051c0 0x5606db3b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.006+0000 7f3aed39a640 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf051c0 0x5606db3b1d50 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eaae8097227b64 server_cookie=b570840c861b586a in_seq=0 out_seq=0 2024-09-21T06:20:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.006+0000 7f3acd7fa640 10 client.27041 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:27.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.041+0000 7f0a47fff640 1 -- 172.21.15.31:0/1151156818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400ab190 msgr2=0x7f0a400b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.041+0000 7f0a47fff640 1 --2- 172.21.15.31:0/1151156818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400ab190 0x7f0a400b2af0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a48069e80 tx=0x7f0a4808f8d0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.045+0000 7f0a47fff640 1 -- 172.21.15.31:0/1151156818 shutdown_connections 2024-09-21T06:20:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.045+0000 7f0a47fff640 1 --2- 172.21.15.31:0/1151156818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400ab190 0x7f0a400b2af0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.045+0000 7f0a47fff640 1 --2- 172.21.15.31:0/1151156818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400aa840 0x7f0a400aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.045+0000 7f0a47fff640 1 --2- 172.21.15.31:0/1151156818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a400abb90 0x7f0a400abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.045+0000 7f0a47fff640 1 -- 172.21.15.31:0/1151156818 >> 172.21.15.31:0/1151156818 conn(0x7f0a4001cc70 msgr2=0x7f0a4001d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.050+0000 7f0a47fff640 1 -- 172.21.15.31:0/1151156818 shutdown_connections 2024-09-21T06:20:27.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.052+0000 7f0a47fff640 1 -- 172.21.15.31:0/1151156818 wait complete. 2024-09-21T06:20:27.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.054+0000 7f0a47fff640 1 Processor -- start 2024-09-21T06:20:27.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.054+0000 7f0a47fff640 1 -- start start 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400aa840 0x7f0a4013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a400ab190 0x7f0a4013cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 0x7f0a40142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a47fff640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400b5eb0 con 0x7f0a400ab190 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a47fff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400b5d30 con 0x7f0a400abb90 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a47fff640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400b6030 con 0x7f0a400aa840 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 0x7f0a40142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 0x7f0a40142160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41770/0 (socket says 172.21.15.31:41770) 2024-09-21T06:20:27.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.055+0000 7f0a477fe640 1 -- 172.21.15.31:0/3878024772 learned_addr learned my addr 172.21.15.31:0/3878024772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.056+0000 7f0a477fe640 1 -- 172.21.15.31:0/3878024772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400aa840 msgr2=0x7f0a4013c920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:20:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.056+0000 7f0a477fe640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400aa840 0x7f0a4013c920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.056+0000 7f0a477fe640 1 -- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a400ab190 msgr2=0x7f0a4013cec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:20:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.056+0000 7f0a477fe640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a400ab190 0x7f0a4013cec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.056+0000 7f0a477fe640 1 -- 172.21.15.31:0/3878024772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a40142890 con 0x7f0a400abb90 2024-09-21T06:20:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.057+0000 7f0a477fe640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 0x7f0a40142160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000b60 tx=0x7f0a4806a800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.058+0000 7f0a3ffff640 1 -- 172.21.15.31:0/3878024772 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480a4070 con 0x7f0a400abb90 2024-09-21T06:20:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.058+0000 7f0a3ffff640 1 -- 172.21.15.31:0/3878024772 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a4806ab70 con 0x7f0a400abb90 2024-09-21T06:20:27.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.058+0000 7f0a3ffff640 1 -- 172.21.15.31:0/3878024772 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4809b040 con 0x7f0a400abb90 2024-09-21T06:20:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.058+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a40142b20 con 0x7f0a400abb90 2024-09-21T06:20:27.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.059+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a40142ec0 con 0x7f0a400abb90 2024-09-21T06:20:27.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.062+0000 7f0a53b7a740 1 Processor -- start 2024-09-21T06:20:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 -- start start 2024-09-21T06:20:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8624340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8624880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8637d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5632d8597e50 con 0x5632d827a070 2024-09-21T06:20:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5632d8587280 con 0x5632d8259390 2024-09-21T06:20:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.065+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5632d858b090 con 0x5632d82511b0 2024-09-21T06:20:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.066+0000 7f0a3ffff640 1 -- 172.21.15.31:0/3878024772 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0a480a7070 con 0x7f0a400abb90 2024-09-21T06:20:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.067+0000 7f0a3ffff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0a340b6880 0x7f0a340b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.068+0000 7f0a477fe640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0a340b6880 0x7f0a340b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.068+0000 7f0a3ffff640 1 -- 172.21.15.31:0/3878024772 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f0a4816cb70 con 0x7f0a400abb90 2024-09-21T06:20:27.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.070+0000 7f0a477fe640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0a340b6880 0x7f0a340b8d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a4812ef40 tx=0x7f0a48156040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.071+0000 7f0a46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8624880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.071+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8637d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.071+0000 7f0a46ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8624880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:43980/0 (socket says 172.21.15.31:43980) 2024-09-21T06:20:27.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.071+0000 7f0a46ffd640 1 -- 172.21.15.31:0/787343282 learned_addr learned my addr 172.21.15.31:0/787343282 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.077+0000 7f0a467fc640 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8624340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.078+0000 7f0a467fc640 1 -- 172.21.15.31:0/787343282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 msgr2=0x5632d8624880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.078+0000 7f0a467fc640 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8624880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.079+0000 7f0a467fc640 1 -- 172.21.15.31:0/787343282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 msgr2=0x5632d8624340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.079+0000 7f0a467fc640 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8624340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.079+0000 7f0a467fc640 1 -- 172.21.15.31:0/787343282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d82e5650 con 0x5632d8259390 2024-09-21T06:20:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.081+0000 7f3acd7fa640 1 -- 172.21.15.31:0/962906894 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606db28d640 con 0x5606daf051c0 2024-09-21T06:20:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.081+0000 7f3acd7fa640 10 client.27041 handle_command_reply: tid=0 2024-09-21T06:20:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.081+0000 7f3af6e45740 1 client.27041 shutdown 2024-09-21T06:20:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.082+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 msgr2=0x5606db397490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.082+0000 7f3af6e45740 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 0x5606db397490 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80315d0 tx=0x7f3ad8005fd0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 shutdown_connections 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3af6e45740 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5606daf051c0 0x5606db3b1d50 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3af6e45740 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606daf711b0 0x5606db32a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3af6e45740 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606daf504d0 0x5606db397a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3af6e45740 1 --2- 172.21.15.31:0/962906894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606daf482f0 0x5606db397490 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 >> 172.21.15.31:0/962906894 conn(0x5606daf65300 msgr2=0x5606db323f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.083+0000 7f3acd7fa640 0 client.27041 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.084+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 shutdown_connections 2024-09-21T06:20:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.084+0000 7f0a467fc640 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8637d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a40018700 tx=0x7f0a38009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5415c8f5f42355f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.084+0000 7f3af6e45740 1 -- 172.21.15.31:0/962906894 wait complete. 2024-09-21T06:20:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:20:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:20:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:20:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:20:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.085+0000 7f3af6e45740 20 client.27041 trim_cache size 0 max 16384 2024-09-21T06:20:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.087+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3ad00b66f0 msgr2=0x7f3ad00b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.087+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3ad00b66f0 0x7f3ad00b8c00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009230 tx=0x7f3ad8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.087+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 msgr2=0x7f3ae819fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.087+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 0x7f3ae819fad0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3ae000b970 tx=0x7f3ae000be30 comp rx=0 tx=0).stop 2024-09-21T06:20:27.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f0a3d7fa640 1 -- 172.21.15.31:0/787343282 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38004030 con 0x5632d8259390 2024-09-21T06:20:27.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f0a3d7fa640 1 -- 172.21.15.31:0/787343282 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a380054d0 con 0x5632d8259390 2024-09-21T06:20:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 shutdown_connections 2024-09-21T06:20:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8108a70 0x7f3ae819fad0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ae8108120 0x7f3ae819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f3ad00b66f0 0x7f3ad00b8c00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f3aeeb9d640 1 --2- 172.21.15.31:0/2943613589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ae81027f0 0x7f3ae819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.088+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 >> 172.21.15.31:0/2943613589 conn(0x7f3ae80fe230 msgr2=0x7f3ae8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.090+0000 7f0a53b7a740 1 -- 172.21.15.31:0/787343282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 msgr2=0x5632d8637d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.090+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8637d70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a40018700 tx=0x7f0a38009590 comp rx=0 tx=0).stop 2024-09-21T06:20:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.090+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 shutdown_connections 2024-09-21T06:20:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:27.090+0000 7f3aeeb9d640 1 -- 172.21.15.31:0/2943613589 wait complete. 2024-09-21T06:20:27.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.096+0000 7f0a53b7a740 1 -- 172.21.15.31:0/787343282 shutdown_connections 2024-09-21T06:20:27.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.096+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8637d70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.096+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8624880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.096+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/787343282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8624340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.096+0000 7f0a53b7a740 1 -- 172.21.15.31:0/787343282 >> 172.21.15.31:0/787343282 conn(0x5632d826e460 msgr2=0x5632d861eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.096+0000 7f0a53b7a740 1 -- 172.21.15.31:0/787343282 shutdown_connections 2024-09-21T06:20:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.101+0000 7f0a53b7a740 1 -- 172.21.15.31:0/787343282 wait complete. 2024-09-21T06:20:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.103+0000 7f0a53b7a740 1 -- start start 2024-09-21T06:20:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.103+0000 7f0a53b7a740 1 Processor -- start 2024-09-21T06:20:27.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8635500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d8259390 0x5632d8635a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 0x5632d8635f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5632d83e8700 con 0x5632d8259390 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5632d8522110 con 0x5632d827a070 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5632d8638790 con 0x5632d82511b0 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.104+0000 7f0a53b7a740 10 client.0 _ll_register_callbacks cb 0x5632d853b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.108+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8635500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.108+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 0x5632d8635f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.115+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 0x5632d8635f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41782/0 (socket says 172.21.15.31:41782) 2024-09-21T06:20:27.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.115+0000 7f0a467fc640 1 -- 172.21.15.31:0/4261611841 learned_addr learned my addr 172.21.15.31:0/4261611841 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.126+0000 7f0a46ffd640 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d8259390 0x5632d8635a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.126+0000 7f0a467fc640 1 -- 172.21.15.31:0/4261611841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 msgr2=0x5632d8635500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.126+0000 7f0a467fc640 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8635500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.126+0000 7f0a467fc640 1 -- 172.21.15.31:0/4261611841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d8259390 msgr2=0x5632d8635a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.126+0000 7f0a467fc640 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d8259390 0x5632d8635a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.126+0000 7f0a467fc640 1 -- 172.21.15.31:0/4261611841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d854fc40 con 0x5632d827a070 2024-09-21T06:20:27.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.133+0000 7f0a467fc640 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 0x5632d8635f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a38001eb0 tx=0x7f0a38009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f845635d492ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.133+0000 7f0a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.137+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/4261611841 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38023030 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.137+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a380054d0 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.137+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/4261611841 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f0a38005000 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.137+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/4261611841 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0a38035050 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.138+0000 7f0a53b7a740 10 client.27053 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.138+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5632d86444b0 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.138+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/4261611841 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a380054d0 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/4261611841 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0a38010b40 con 0x5632d827a070 2024-09-21T06:20:27.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a53b7a740 10 client.27053 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:27.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a53b7a740 10 client.27053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:27.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a53b7a740 20 client.27053 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:27.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d82389f0 0x5632d8647c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a53b7a740 4 client.27053 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:27.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.139+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632d8599da0 con 0x5632d82389f0 2024-09-21T06:20:27.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.144+0000 7f0a46ffd640 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d82389f0 0x5632d8647c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.153+0000 7f0a46ffd640 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d82389f0 0x5632d8647c70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866cdd8190d3a15d server_cookie=45ff46e75d14022b in_seq=0 out_seq=0 2024-09-21T06:20:27.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.153+0000 7f0a2e7fc640 10 client.27053 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:27.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/4261611841 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632d8599da0 con 0x5632d82389f0 2024-09-21T06:20:27.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a2e7fc640 10 client.27053 handle_command_reply: tid=0 2024-09-21T06:20:27.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 client.27053 shutdown 2024-09-21T06:20:27.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 msgr2=0x5632d8635f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 0x5632d8635f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a38001eb0 tx=0x7f0a38009cf0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 shutdown_connections 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d82389f0 0x5632d8647c70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d827a070 0x5632d8635f80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d8259390 0x5632d8635a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/4261611841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d8635500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 >> 172.21.15.31:0/4261611841 conn(0x5632d826e460 msgr2=0x5632d861efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.159+0000 7f0a2e7fc640 0 client.27053 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.162+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 shutdown_connections 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.162+0000 7f0a53b7a740 1 -- 172.21.15.31:0/4261611841 wait complete. 2024-09-21T06:20:27.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.162+0000 7f0a53b7a740 20 client.27053 trim_cache size 0 max 16384 2024-09-21T06:20:27.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.209+0000 7f0a53b7a740 1 Processor -- start 2024-09-21T06:20:27.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.210+0000 7f0a53b7a740 1 -- start start 2024-09-21T06:20:27.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.210+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 0x5632d8637130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.211+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d8259390 0x5632d86326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.211+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8632c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.211+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4014e9c0 con 0x5632d827a070 2024-09-21T06:20:27.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.211+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40151220 con 0x5632d82511b0 2024-09-21T06:20:27.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.211+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4014f010 con 0x5632d8259390 2024-09-21T06:20:27.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.214+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 0x5632d8637130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.214+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 0x5632d8637130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:41790/0 (socket says 172.21.15.31:41790) 2024-09-21T06:20:27.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.214+0000 7f0a467fc640 1 -- 172.21.15.31:0/1835870629 learned_addr learned my addr 172.21.15.31:0/1835870629 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.214+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8632c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.218+0000 7f0a46ffd640 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d8259390 0x5632d86326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.218+0000 7f0a467fc640 1 -- 172.21.15.31:0/1835870629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d8259390 msgr2=0x5632d86326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.218+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d8259390 0x5632d86326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.218+0000 7f0a467fc640 1 -- 172.21.15.31:0/1835870629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 msgr2=0x5632d8632c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.218+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8632c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.218+0000 7f0a467fc640 1 -- 172.21.15.31:0/1835870629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d861f500 con 0x5632d82511b0 2024-09-21T06:20:27.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.222+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 0x5632d8637130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800a010 tx=0x7f0a38009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b352d6b73a5a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.224+0000 7f0a3cff9640 1 -- 172.21.15.31:0/1835870629 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38040070 con 0x5632d82511b0 2024-09-21T06:20:27.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.224+0000 7f0a3cff9640 1 -- 172.21.15.31:0/1835870629 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a38005490 con 0x5632d82511b0 2024-09-21T06:20:27.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.224+0000 7f0a3cff9640 1 -- 172.21.15.31:0/1835870629 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38023040 con 0x5632d82511b0 2024-09-21T06:20:27.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.226+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1835870629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 msgr2=0x5632d8637130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.227+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 0x5632d8637130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800a010 tx=0x7f0a38009260 comp rx=0 tx=0).stop 2024-09-21T06:20:27.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.231+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1835870629 shutdown_connections 2024-09-21T06:20:27.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.231+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8632c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.231+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d8259390 0x5632d86326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.231+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1835870629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d82511b0 0x5632d8637130 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.231+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1835870629 >> 172.21.15.31:0/1835870629 conn(0x5632d826e460 msgr2=0x5632d8636a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.237+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1835870629 shutdown_connections 2024-09-21T06:20:27.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.237+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1835870629 wait complete. 2024-09-21T06:20:27.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.238+0000 7f0a53b7a740 1 -- start start 2024-09-21T06:20:27.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.238+0000 7f0a53b7a740 1 Processor -- start 2024-09-21T06:20:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.239+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d863e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.239+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8639520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.239+0000 7f0a53b7a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8639a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.239+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d863e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.239+0000 7f0a53b7a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4014f6e0 con 0x5632d827a070 2024-09-21T06:20:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.240+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d863e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44012/0 (socket says 172.21.15.31:44012) 2024-09-21T06:20:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.240+0000 7f0a467fc640 1 -- 172.21.15.31:0/1862141803 learned_addr learned my addr 172.21.15.31:0/1862141803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.240+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a40150b50 con 0x5632d8259390 2024-09-21T06:20:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.240+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4014fdb0 con 0x5632d82511b0 2024-09-21T06:20:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.240+0000 7f0a53b7a740 10 client.0 _ll_register_callbacks cb 0x7f0a38009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.240+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8639a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.241+0000 7f0a467fc640 1 -- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 msgr2=0x5632d8639520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.241+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8639520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.241+0000 7f0a467fc640 1 -- 172.21.15.31:0/1862141803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 msgr2=0x5632d8639a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.241+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8639a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.241+0000 7f0a467fc640 1 -- 172.21.15.31:0/1862141803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d8639fa0 con 0x5632d82511b0 2024-09-21T06:20:27.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.246+0000 7f0a467fc640 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d863e070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a3803e040 tx=0x7f0a38009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4059e6af1b098431 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:27.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.248+0000 7f0a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:27.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.251+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/1862141803 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38023070 con 0x5632d82511b0 2024-09-21T06:20:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.251+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/1862141803 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f0a38010c40 con 0x5632d82511b0 2024-09-21T06:20:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.254+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a38005490 con 0x5632d82511b0 2024-09-21T06:20:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.258+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/1862141803 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0a3804a050 con 0x5632d82511b0 2024-09-21T06:20:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.259+0000 7f0a53b7a740 10 client.27094 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.259+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5632d86b2360 con 0x5632d82511b0 2024-09-21T06:20:27.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.264+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/1862141803 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a38005490 con 0x5632d82511b0 2024-09-21T06:20:27.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.264+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/1862141803 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0a38038040 con 0x5632d82511b0 2024-09-21T06:20:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.266+0000 7f0a53b7a740 10 client.27094 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.266+0000 7f0a53b7a740 10 client.27094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.266+0000 7f0a53b7a740 20 client.27094 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.266+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d820dda0 0x5632d86b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.266+0000 7f0a53b7a740 4 client.27094 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:20:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.266+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5632d8595850 con 0x5632d820dda0 2024-09-21T06:20:27.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.267+0000 7f0a46ffd640 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d820dda0 0x5632d86b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:27.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.273+0000 7f0a46ffd640 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d820dda0 0x5632d86b4a30 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c2ad0af2d361a6 server_cookie=9e7a8e28062e3f50 in_seq=0 out_seq=0 2024-09-21T06:20:27.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.273+0000 7f0a2e7fc640 10 client.27094 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.274+0000 7f0a2e7fc640 1 -- 172.21.15.31:0/1862141803 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x5632d8595850 con 0x5632d820dda0 2024-09-21T06:20:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.274+0000 7f0a2e7fc640 10 client.27094 handle_command_reply: tid=0 2024-09-21T06:20:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.274+0000 7f0a53b7a740 1 client.27094 shutdown 2024-09-21T06:20:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 msgr2=0x5632d863e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d863e070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a3803e040 tx=0x7f0a38009d10 comp rx=0 tx=0).stop 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 shutdown_connections 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5632d820dda0 0x5632d86b4a30 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5632d827a070 0x5632d8639a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632d8259390 0x5632d8639520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 --2- 172.21.15.31:0/1862141803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5632d82511b0 0x5632d863e070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a2e7fc640 0 client.27094 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 >> 172.21.15.31:0/1862141803 conn(0x5632d826e460 msgr2=0x5632d8640a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 shutdown_connections 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 1 -- 172.21.15.31:0/1862141803 wait complete. 2024-09-21T06:20:27.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.275+0000 7f0a53b7a740 20 client.27094 trim_cache size 0 max 16384 2024-09-21T06:20:27.280 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:20:27.280 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:20:27.280 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:20:27.284 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:20:27.286 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:20:27.288 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:20:27.290 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:20:27.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.293 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:27.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:27.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:27.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:27.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:27.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:27.295 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:20:27.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0a340b6880 msgr2=0x7f0a340b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0a340b6880 0x7f0a340b8d90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0a4812ef40 tx=0x7f0a48156040 comp rx=0 tx=0).stop 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 msgr2=0x7f0a40142160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 0x7f0a40142160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000b60 tx=0x7f0a4806a800 comp rx=0 tx=0).stop 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 shutdown_connections 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a400abb90 0x7f0a40142160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0a340b6880 0x7f0a340b8d90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a400ab190 0x7f0a4013cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 --2- 172.21.15.31:0/3878024772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a400aa840 0x7f0a4013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 >> 172.21.15.31:0/3878024772 conn(0x7f0a4001cc70 msgr2=0x7f0a400a3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 shutdown_connections 2024-09-21T06:20:27.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:27.280+0000 7f0a47fff640 1 -- 172.21.15.31:0/3878024772 wait complete. 2024-09-21T06:20:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:27 smithi110 ceph-mon[34550]: pgmap v591: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 780 op/s 2024-09-21T06:20:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:27 smithi107 ceph-mon[34380]: pgmap v591: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 780 op/s 2024-09-21T06:20:27.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:27 smithi031 ceph-mon[31237]: pgmap v591: 97 pgs: 97 active+clean; 4.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 780 op/s 2024-09-21T06:20:28.369 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:20:28.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:20:28.793 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:29.616 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:29 smithi031 ceph-mon[31237]: pgmap v592: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 821 op/s 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.671+0000 7f7785637640 1 -- 172.21.15.31:0/285135657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778010de10 msgr2=0x7f7780110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.671+0000 7f7785637640 1 --2- 172.21.15.31:0/285135657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778010de10 0x7f7780110300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7778009900 tx=0x7f777802a410 comp rx=0 tx=0).stop 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 -- 172.21.15.31:0/285135657 shutdown_connections 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 --2- 172.21.15.31:0/285135657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778010de10 0x7f7780110300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 --2- 172.21.15.31:0/285135657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7780105300 0x7f778010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 --2- 172.21.15.31:0/285135657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77801049b0 0x7f7780104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 -- 172.21.15.31:0/285135657 >> 172.21.15.31:0/285135657 conn(0x7f7780079730 msgr2=0x7f7780079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 -- 172.21.15.31:0/285135657 shutdown_connections 2024-09-21T06:20:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.672+0000 7f7785637640 1 -- 172.21.15.31:0/285135657 wait complete. 2024-09-21T06:20:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.673+0000 7f7785637640 1 Processor -- start 2024-09-21T06:20:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.673+0000 7f7785637640 1 -- start start 2024-09-21T06:20:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.673+0000 7f7785637640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77801049b0 0x7f778010acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.673+0000 7f7785637640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 0x7f778010b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f7785637640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f778010de10 0x7f778010b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f7785637640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7780115d20 con 0x7f778010de10 2024-09-21T06:20:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f7785637640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7780115ba0 con 0x7f7780105300 2024-09-21T06:20:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f7785637640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7780115ea0 con 0x7f77801049b0 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 0x7f778010b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f778010de10 0x7f778010b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 0x7f778010b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52494/0 (socket says 172.21.15.31:52494) 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 -- 172.21.15.31:0/2477999191 learned_addr learned my addr 172.21.15.31:0/2477999191 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 -- 172.21.15.31:0/2477999191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77801049b0 msgr2=0x7f778010acc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77801049b0 0x7f778010acc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 -- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f778010de10 msgr2=0x7f778010b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f778010de10 0x7f778010b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.674+0000 7f777e7fc640 1 -- 172.21.15.31:0/2477999191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77801acb90 con 0x7f7780105300 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.675+0000 7f777e7fc640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 0x7f778010b200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7768006fd0 tx=0x7f776800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.675+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768011830 con 0x7f7780105300 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.675+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7768016070 con 0x7f7780105300 2024-09-21T06:20:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.675+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768010df0 con 0x7f7780105300 2024-09-21T06:20:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.675+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77801ace80 con 0x7f7780105300 2024-09-21T06:20:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.675+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77801ad360 con 0x7f7780105300 2024-09-21T06:20:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.677+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7780104dc0 con 0x7f7780105300 2024-09-21T06:20:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.679+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f77680104a0 con 0x7f7780105300 2024-09-21T06:20:29.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.680+0000 7f7777fff640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f776c0b6700 0x7f776c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:29.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.680+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f77680e6cb0 con 0x7f7780105300 2024-09-21T06:20:29.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.681+0000 7f777effd640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f776c0b6700 0x7f776c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:29.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.682+0000 7f777effd640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f776c0b6700 0x7f776c0b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7770005c40 tx=0x7f7770005bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:29.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:29.685+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77680ad210 con 0x7f7780105300 2024-09-21T06:20:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:29 smithi110 ceph-mon[34550]: pgmap v592: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 821 op/s 2024-09-21T06:20:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:29 smithi107 ceph-mon[34380]: pgmap v592: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 821 op/s 2024-09-21T06:20:30.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.110+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77801ad840 con 0x7f7780105300 2024-09-21T06:20:30.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.111+0000 7f7777fff640 1 -- 172.21.15.31:0/2477999191 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f77680b15f0 con 0x7f7780105300 2024-09-21T06:20:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:20:30.113 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:20:30.114 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.115+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f776c0b6700 msgr2=0x7f776c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.115+0000 7f7785637640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f776c0b6700 0x7f776c0b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7770005c40 tx=0x7f7770005bd0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.115+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 msgr2=0x7f778010b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.115+0000 7f7785637640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 0x7f778010b200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7768006fd0 tx=0x7f776800b8e0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.115+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 shutdown_connections 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.115+0000 7f7785637640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f776c0b6700 0x7f776c0b8c10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.116+0000 7f7785637640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f778010de10 0x7f778010b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.116+0000 7f7785637640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7780105300 0x7f778010b200 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.116+0000 7f7785637640 1 --2- 172.21.15.31:0/2477999191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77801049b0 0x7f778010acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.116+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 >> 172.21.15.31:0/2477999191 conn(0x7f7780079730 msgr2=0x7f7780113e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:30.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.116+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 shutdown_connections 2024-09-21T06:20:30.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:30.116+0000 7f7785637640 1 -- 172.21.15.31:0/2477999191 wait complete. 2024-09-21T06:20:30.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 530.0 s, 2706.4 tps, lat 36.907 ms stddev 81.992 2024-09-21T06:20:30.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:30 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2477999191' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:30.781 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:30 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2477999191' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:30.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:30 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2477999191' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:30.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:20:31.273 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:31.743 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:31 smithi031 ceph-mon[31237]: pgmap v593: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-09-21T06:20:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:31 smithi110 ceph-mon[34550]: pgmap v593: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-09-21T06:20:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:31 smithi107 ceph-mon[34380]: pgmap v593: 97 pgs: 97 active+clean; 4.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 -- 172.21.15.31:0/3273552066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c114750 msgr2=0x7f815c116b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 --2- 172.21.15.31:0/3273552066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c114750 0x7f815c116b70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8154009ca0 tx=0x7f815402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 -- 172.21.15.31:0/3273552066 shutdown_connections 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 --2- 172.21.15.31:0/3273552066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c114750 0x7f815c116b70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 --2- 172.21.15.31:0/3273552066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c078290 0x7f815c114210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 --2- 172.21.15.31:0/3273552066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c077940 0x7f815c077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 -- 172.21.15.31:0/3273552066 >> 172.21.15.31:0/3273552066 conn(0x7f815c1017e0 msgr2=0x7f815c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 -- 172.21.15.31:0/3273552066 shutdown_connections 2024-09-21T06:20:31.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.869+0000 7f816349f640 1 -- 172.21.15.31:0/3273552066 wait complete. 2024-09-21T06:20:31.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.870+0000 7f816349f640 1 Processor -- start 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.870+0000 7f816349f640 1 -- start start 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816349f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c077940 0x7f815c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816349f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c078290 0x7f815c1a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816349f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 0x7f815c1a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816349f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f815c11a0c0 con 0x7f815c077940 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816349f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f815c119f40 con 0x7f815c114750 2024-09-21T06:20:31.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816349f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f815c11a240 con 0x7f815c078290 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 0x7f815c1a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 0x7f815c1a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52512/0 (socket says 172.21.15.31:52512) 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 -- 172.21.15.31:0/87909471 learned_addr learned my addr 172.21.15.31:0/87909471 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f816123b640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c077940 0x7f815c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8160a3a640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c078290 0x7f815c1a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 -- 172.21.15.31:0/87909471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c078290 msgr2=0x7f815c1a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c078290 0x7f815c1a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 -- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c077940 msgr2=0x7f815c1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c077940 0x7f815c1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.871+0000 7f8161a3c640 1 -- 172.21.15.31:0/87909471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c1a69c0 con 0x7f815c114750 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.872+0000 7f8161a3c640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 0x7f815c1a6290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f81540098d0 tx=0x7f81540317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.872+0000 7f815a7fc640 1 -- 172.21.15.31:0/87909471 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154003b70 con 0x7f815c114750 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.872+0000 7f815a7fc640 1 -- 172.21.15.31:0/87909471 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8154003d10 con 0x7f815c114750 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.872+0000 7f815a7fc640 1 -- 172.21.15.31:0/87909471 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815402fa20 con 0x7f815c114750 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.872+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c1a6c50 con 0x7f815c114750 2024-09-21T06:20:31.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.872+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1a7080 con 0x7f815c114750 2024-09-21T06:20:31.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.876+0000 7f8169a13740 1 Processor -- start 2024-09-21T06:20:31.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.876+0000 7f815a7fc640 1 -- 172.21.15.31:0/87909471 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f815402fc40 con 0x7f815c114750 2024-09-21T06:20:31.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.876+0000 7f8169a13740 1 -- start start 2024-09-21T06:20:31.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8169a13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2eded92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8169a13740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb091b0 0x55a2eded9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8169a13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb11390 0x55a2eded9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8169a13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a2ede4fea0 con 0x55a2edb32070 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8169a13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2ede3f2d0 con 0x55a2edb11390 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8169a13740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a2ede430e0 con 0x55a2edb091b0 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8160a3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2eded92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8160a3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2eded92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35090/0 (socket says 172.21.15.31:35090) 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8160a3a640 1 -- 172.21.15.31:0/3679270185 learned_addr learned my addr 172.21.15.31:0/3679270185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f816123b640 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb091b0 0x55a2eded9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.877+0000 7f8160a3a640 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb11390 0x55a2eded9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f816123b640 1 -- 172.21.15.31:0/3679270185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb11390 msgr2=0x55a2eded9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f816123b640 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb11390 0x55a2eded9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f816123b640 1 -- 172.21.15.31:0/3679270185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 msgr2=0x55a2eded92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f816123b640 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2eded92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f816123b640 1 -- 172.21.15.31:0/3679270185 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2edb9d650 con 0x55a2edb091b0 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f816123b640 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb091b0 0x55a2eded9800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f814c009c20 tx=0x7f814c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d662fec8a11a8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f81437fe640 1 -- 172.21.15.31:0/3679270185 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c004050 con 0x55a2edb091b0 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f815a7fc640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81440b6700 0x7f81440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.878+0000 7f81437fe640 1 -- 172.21.15.31:0/3679270185 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814c0054d0 con 0x55a2edb091b0 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.879+0000 7f8160a3a640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81440b6700 0x7f81440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.879+0000 7f815a7fc640 1 -- 172.21.15.31:0/87909471 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f81541088e0 con 0x7f815c114750 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.879+0000 7f81437fe640 1 -- 172.21.15.31:0/3679270185 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c004c80 con 0x55a2edb091b0 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 -- 172.21.15.31:0/3679270185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb091b0 msgr2=0x55a2eded9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb091b0 0x55a2eded9800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f814c009c20 tx=0x7f814c011810 comp rx=0 tx=0).stop 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8160a3a640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81440b6700 0x7f81440b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f815c06aa20 tx=0x7f815000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 -- 172.21.15.31:0/3679270185 shutdown_connections 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb11390 0x55a2eded9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb091b0 0x55a2eded9800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 --2- 172.21.15.31:0/3679270185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2eded92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 -- 172.21.15.31:0/3679270185 >> 172.21.15.31:0/3679270185 conn(0x55a2edb26460 msgr2=0x55a2eded71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 -- 172.21.15.31:0/3679270185 shutdown_connections 2024-09-21T06:20:31.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.880+0000 7f8169a13740 1 -- 172.21.15.31:0/3679270185 wait complete. 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.881+0000 7f8169a13740 1 -- start start 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.881+0000 7f8169a13740 1 Processor -- start 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f8169a13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 0x55a2edee4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f8169a13740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f816123b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 0x55a2edee4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f816123b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 0x55a2edee4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52526/0 (socket says 172.21.15.31:52526) 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f8160a3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f8160a3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37598/0 (socket says 172.21.15.31:37598) 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.882+0000 7f816123b640 1 -- 172.21.15.31:0/351977326 learned_addr learned my addr 172.21.15.31:0/351977326 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2edee52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a2edca0700 con 0x55a2edb32070 2024-09-21T06:20:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a2eddda160 con 0x55a2edb091b0 2024-09-21T06:20:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f8161a3c640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2edee52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a2edee7b40 con 0x55a2edb11390 2024-09-21T06:20:31.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f816123b640 1 -- 172.21.15.31:0/351977326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 msgr2=0x55a2edee4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f816123b640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f816123b640 1 -- 172.21.15.31:0/351977326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 msgr2=0x55a2edee52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f816123b640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2edee52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.883+0000 7f816123b640 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ede07c90 con 0x55a2edb091b0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8169a13740 10 client.0 _ll_register_callbacks cb 0x55a2eddf39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8161a3c640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2edee52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c0054d0 con 0x55a2edb091b0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f816123b640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 0x55a2edee4840 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f814c01b040 tx=0x7f814c004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb60a4fa9e85664 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8140ff9640 10 client.27074 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8140ff9640 1 -- 172.21.15.31:0/351977326 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c033050 con 0x55a2edb091b0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8140ff9640 1 -- 172.21.15.31:0/351977326 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f814c003a00 con 0x55a2edb091b0 2024-09-21T06:20:31.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8140ff9640 1 -- 172.21.15.31:0/351977326 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f814c029070 con 0x55a2edb091b0 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8169a13740 10 client.27074 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.884+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55a2edef3590 con 0x55a2edb091b0 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8140ff9640 1 -- 172.21.15.31:0/351977326 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f814c003c60 con 0x55a2edb091b0 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8169a13740 10 client.27074 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8169a13740 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8169a13740 20 client.27074 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edaf09f0 0x55a2edef68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8169a13740 4 client.27074 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2ede51df0 con 0x55a2edaf09f0 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.885+0000 7f8160a3a640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edaf09f0 0x55a2edef68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.886+0000 7f8160a3a640 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edaf09f0 0x55a2edef68f0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6949baf72d6338 server_cookie=869b6721eeb48b72 in_seq=0 out_seq=0 2024-09-21T06:20:31.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.886+0000 7f8140ff9640 10 client.27074 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:31.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.886+0000 7f8140ff9640 1 -- 172.21.15.31:0/351977326 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2ede51df0 con 0x55a2edaf09f0 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.886+0000 7f8140ff9640 10 client.27074 handle_command_reply: tid=0 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 client.27074 shutdown 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 msgr2=0x55a2edee4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 0x55a2edee4840 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f814c01b040 tx=0x7f814c004790 comp rx=0 tx=0).stop 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 shutdown_connections 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edaf09f0 0x55a2edef68f0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb32070 0x55a2edee52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 --2- 172.21.15.31:0/351977326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb091b0 0x55a2edee4840 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 >> 172.21.15.31:0/351977326 conn(0x55a2edb26460 msgr2=0x55a2ededc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.887+0000 7f8140ff9640 0 client.27074 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.888+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 shutdown_connections 2024-09-21T06:20:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.888+0000 7f8169a13740 1 -- 172.21.15.31:0/351977326 wait complete. 2024-09-21T06:20:31.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.888+0000 7f8169a13740 20 client.27074 trim_cache size 0 max 16384 2024-09-21T06:20:31.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.923+0000 7f8169a13740 1 Processor -- start 2024-09-21T06:20:31.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.925+0000 7f8169a13740 1 -- start start 2024-09-21T06:20:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.926+0000 7f8169a13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2ededadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.926+0000 7f8169a13740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2eded6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.926+0000 7f8169a13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2eded6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8169a13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1b3cd0 con 0x55a2edb091b0 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8169a13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1b6530 con 0x55a2edb32070 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8161a3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2eded6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8161a3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2eded6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52536/0 (socket says 172.21.15.31:52536) 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8161a3c640 1 -- 172.21.15.31:0/513467080 learned_addr learned my addr 172.21.15.31:0/513467080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:31.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8169a13740 1 -- 172.21.15.31:0/513467080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1b4320 con 0x55a2edb11390 2024-09-21T06:20:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.927+0000 7f8160a3a640 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2eded6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.928+0000 7f816123b640 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2ededadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.931+0000 7f8161a3c640 1 -- 172.21.15.31:0/513467080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 msgr2=0x55a2eded6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.931+0000 7f8161a3c640 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2eded6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.931+0000 7f8161a3c640 1 -- 172.21.15.31:0/513467080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 msgr2=0x55a2ededadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.931+0000 7f8161a3c640 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2ededadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.931+0000 7f8161a3c640 1 -- 172.21.15.31:0/513467080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2eded1440 con 0x55a2edb32070 2024-09-21T06:20:31.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.932+0000 7f8161a3c640 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2eded6ca0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f81540ce440 tx=0x7f81540ca6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eca199931870f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.933+0000 7f8142ffd640 1 -- 172.21.15.31:0/513467080 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540ce250 con 0x55a2edb32070 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.933+0000 7f8142ffd640 1 -- 172.21.15.31:0/513467080 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8154036030 con 0x55a2edb32070 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.933+0000 7f8142ffd640 1 -- 172.21.15.31:0/513467080 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540d7f10 con 0x55a2edb32070 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 -- 172.21.15.31:0/513467080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 msgr2=0x55a2eded6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2eded6ca0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f81540ce440 tx=0x7f81540ca6a0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 -- 172.21.15.31:0/513467080 shutdown_connections 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2eded6ca0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2eded6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 --2- 172.21.15.31:0/513467080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2ededadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.935+0000 7f8169a13740 1 -- 172.21.15.31:0/513467080 >> 172.21.15.31:0/513467080 conn(0x55a2edb26460 msgr2=0x55a2edef0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.939+0000 7f8169a13740 1 -- 172.21.15.31:0/513467080 shutdown_connections 2024-09-21T06:20:31.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.939+0000 7f8169a13740 1 -- 172.21.15.31:0/513467080 wait complete. 2024-09-21T06:20:31.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.939+0000 7f8169a13740 1 -- start start 2024-09-21T06:20:31.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.939+0000 7f8169a13740 1 Processor -- start 2024-09-21T06:20:31.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8169a13740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2edee8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8169a13740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8169a13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2edeecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8169a13740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1b49f0 con 0x55a2edb091b0 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8169a13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1b5e60 con 0x55a2edb32070 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8160a3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8160a3a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37618/0 (socket says 172.21.15.31:37618) 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8160a3a640 1 -- 172.21.15.31:0/1807039618 learned_addr learned my addr 172.21.15.31:0/1807039618 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1b50c0 con 0x55a2edb11390 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.941+0000 7f8161a3c640 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2edeecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.942+0000 7f8169a13740 10 client.0 _ll_register_callbacks cb 0x7f815000ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.942+0000 7f8160a3a640 1 -- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 msgr2=0x55a2edeecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.942+0000 7f8160a3a640 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2edeecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.942+0000 7f8160a3a640 1 -- 172.21.15.31:0/1807039618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 msgr2=0x55a2edee8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.942+0000 7f8160a3a640 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2edee8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.942+0000 7f8160a3a640 1 -- 172.21.15.31:0/1807039618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2edeed390 con 0x55a2edb11390 2024-09-21T06:20:31.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.943+0000 7f8160a3a640 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee8e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8150005d00 tx=0x7f81500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2033a628bcf56589 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:31.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.946+0000 7f8140ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:31.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.946+0000 7f8140ff9640 1 -- 172.21.15.31:0/1807039618 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150004020 con 0x55a2edb11390 2024-09-21T06:20:31.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.946+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8154036030 con 0x55a2edb11390 2024-09-21T06:20:31.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.946+0000 7f8140ff9640 1 -- 172.21.15.31:0/1807039618 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f815001b030 con 0x55a2edb11390 2024-09-21T06:20:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.951+0000 7f8140ff9640 1 -- 172.21.15.31:0/1807039618 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f81500041c0 con 0x55a2edb11390 2024-09-21T06:20:31.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.955+0000 7f8169a13740 10 client.27112 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:31.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.955+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a2edf60cf0 con 0x55a2edb11390 2024-09-21T06:20:31.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.956+0000 7f8140ff9640 1 -- 172.21.15.31:0/1807039618 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8154036030 con 0x55a2edb11390 2024-09-21T06:20:31.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.957+0000 7f8140ff9640 1 -- 172.21.15.31:0/1807039618 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f815002e410 con 0x55a2edb11390 2024-09-21T06:20:31.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.957+0000 7f8169a13740 10 client.27112 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:31.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.957+0000 7f8169a13740 10 client.27112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:31.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.957+0000 7f8169a13740 20 client.27112 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:31.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.957+0000 7f8169a13740 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edac5da0 0x55a2edf63500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:31.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.958+0000 7f8169a13740 4 client.27112 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:31.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.958+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2ede4d8a0 con 0x55a2edac5da0 2024-09-21T06:20:31.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.959+0000 7f816123b640 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edac5da0 0x55a2edf63500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:31.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.962+0000 7f816123b640 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edac5da0 0x55a2edf63500 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3397530cd93e60fa server_cookie=748bd6153efaa282 in_seq=0 out_seq=0 2024-09-21T06:20:31.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.963+0000 7f8140ff9640 10 client.27112 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:31.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.963+0000 7f8140ff9640 1 -- 172.21.15.31:0/1807039618 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2ede4d8a0 con 0x55a2edac5da0 2024-09-21T06:20:31.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.963+0000 7f8140ff9640 10 client.27112 handle_command_reply: tid=0 2024-09-21T06:20:31.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.964+0000 7f8169a13740 1 client.27112 shutdown 2024-09-21T06:20:31.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 msgr2=0x55a2edee8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee8e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8150005d00 tx=0x7f81500046c0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 shutdown_connections 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a2edac5da0 0x55a2edf63500 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a2edb32070 0x55a2edeecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a2edb11390 0x55a2edee8e30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 --2- 172.21.15.31:0/1807039618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a2edb091b0 0x55a2edee8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.965+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 >> 172.21.15.31:0/1807039618 conn(0x55a2edb26460 msgr2=0x55a2edee5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.966+0000 7f8140ff9640 0 client.27112 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:31.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.966+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 shutdown_connections 2024-09-21T06:20:31.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.966+0000 7f8169a13740 1 -- 172.21.15.31:0/1807039618 wait complete. 2024-09-21T06:20:31.970 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:20:31.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.968+0000 7f8169a13740 20 client.27112 trim_cache size 0 max 16384 2024-09-21T06:20:31.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81440b6700 msgr2=0x7f81440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81440b6700 0x7f81440b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f815c06aa20 tx=0x7f815000b410 comp rx=0 tx=0).stop 2024-09-21T06:20:31.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 msgr2=0x7f815c1a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 0x7f815c1a6290 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f81540098d0 tx=0x7f81540317b0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 shutdown_connections 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c114750 0x7f815c1a6290 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f815c078290 0x7f815c1a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f81440b6700 0x7f81440b8c10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 --2- 172.21.15.31:0/87909471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f815c077940 0x7f815c1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.972+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 >> 172.21.15.31:0/87909471 conn(0x7f815c1017e0 msgr2=0x7f815c1167b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:31.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.973+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 shutdown_connections 2024-09-21T06:20:31.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:31.973+0000 7f816349f640 1 -- 172.21.15.31:0/87909471 wait complete. 2024-09-21T06:20:33.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:20:33.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:20:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:33 smithi110 ceph-mon[34550]: pgmap v594: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 2.00k op/s 2024-09-21T06:20:33.786 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:33 smithi031 ceph-mon[31237]: pgmap v594: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 2.00k op/s 2024-09-21T06:20:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:33 smithi107 ceph-mon[34380]: pgmap v594: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 2.00k op/s 2024-09-21T06:20:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:34.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/3953086955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c341083a0 msgr2=0x7f2c341087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/3953086955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c341083a0 0x7f2c341087b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009900 tx=0x7f2c2402a410 comp rx=0 tx=0).stop 2024-09-21T06:20:34.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/3953086955 shutdown_connections 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/3953086955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c34108cf0 0x7f2c3410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/3953086955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c341083a0 0x7f2c341087b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/3953086955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 0x7f2c34102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/3953086955 >> 172.21.15.31:0/3953086955 conn(0x7f2c34079730 msgr2=0x7f2c34079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/3953086955 shutdown_connections 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.749+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/3953086955 wait complete. 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 Processor -- start 2024-09-21T06:20:34.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 -- start start 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 0x7f2c3419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c341083a0 0x7f2c3419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c34108cf0 0x7f2c3419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34113ad0 con 0x7f2c34108cf0 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34113950 con 0x7f2c34102a60 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3b5d3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c34113c50 con 0x7f2c341083a0 2024-09-21T06:20:34.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c38b6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c341083a0 0x7f2c3419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c3936f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 0x7f2c3419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c38b6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c341083a0 0x7f2c3419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37630/0 (socket says 172.21.15.31:37630) 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c34108cf0 0x7f2c3419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.751+0000 7f2c38b6e640 1 -- 172.21.15.31:0/446832651 learned_addr learned my addr 172.21.15.31:0/446832651 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3936f640 1 -- 172.21.15.31:0/446832651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c341083a0 msgr2=0x7f2c3419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3936f640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c341083a0 0x7f2c3419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3936f640 1 -- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c34108cf0 msgr2=0x7f2c3419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3936f640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c34108cf0 0x7f2c3419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3936f640 1 -- 172.21.15.31:0/446832651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c341a0490 con 0x7f2c34102a60 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3936f640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 0x7f2c3419a520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9f0 tx=0x7f2c3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c2a7fc640 1 -- 172.21.15.31:0/446832651 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3000ccc0 con 0x7f2c34102a60 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c2a7fc640 1 -- 172.21.15.31:0/446832651 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c300108e0 con 0x7f2c34102a60 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c2a7fc640 1 -- 172.21.15.31:0/446832651 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c300188e0 con 0x7f2c34102a60 2024-09-21T06:20:34.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c341a0780 con 0x7f2c34102a60 2024-09-21T06:20:34.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.752+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c341a6ee0 con 0x7f2c34102a60 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.758+0000 7f2c41b47740 1 Processor -- start 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.758+0000 7f2c2a7fc640 1 -- 172.21.15.31:0/446832651 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2c30010430 con 0x7f2c34102a60 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.758+0000 7f2c41b47740 1 -- start start 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e01e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e02370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e0a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c41b47740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5630d6d7ae60 con 0x5630d6a5d350 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c41b47740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630d6d6a290 con 0x5630d6a34490 2024-09-21T06:20:34.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c41b47740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630d6d6e0a0 con 0x5630d6a3c670 2024-09-21T06:20:34.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e01e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e01e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35116/0 (socket says 172.21.15.31:35116) 2024-09-21T06:20:34.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c39b70640 1 -- 172.21.15.31:0/283334472 learned_addr learned my addr 172.21.15.31:0/283334472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:34.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.759+0000 7f2c3936f640 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e02370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c39b70640 1 -- 172.21.15.31:0/283334472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 msgr2=0x5630d6e0a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c39b70640 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e0a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c39b70640 1 -- 172.21.15.31:0/283334472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 msgr2=0x5630d6e02370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c39b70640 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e02370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c39b70640 1 -- 172.21.15.31:0/283334472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d6ac8930 con 0x5630d6a5d350 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c3936f640 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e02370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c2a7fc640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2c200b6700 0x7f2c200b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c39b70640 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e01e30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009f50 tx=0x7f2c2c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44718d50642415ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c3936f640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2c200b6700 0x7f2c200b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c1f7fe640 1 -- 172.21.15.31:0/283334472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c004d20 con 0x5630d6a5d350 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c1f7fe640 1 -- 172.21.15.31:0/283334472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c004050 con 0x5630d6a5d350 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c1f7fe640 1 -- 172.21.15.31:0/283334472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c005280 con 0x5630d6a5d350 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.760+0000 7f2c2a7fc640 1 -- 172.21.15.31:0/446832651 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f2c300e7ae0 con 0x7f2c34102a60 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 -- 172.21.15.31:0/283334472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 msgr2=0x5630d6e01e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e01e30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009f50 tx=0x7f2c2c0118d0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c3936f640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2c200b6700 0x7f2c200b8c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9c0 tx=0x7f2c300ec040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 -- 172.21.15.31:0/283334472 shutdown_connections 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e0a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e02370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 --2- 172.21.15.31:0/283334472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e01e30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.761+0000 7f2c41b47740 1 -- 172.21.15.31:0/283334472 >> 172.21.15.31:0/283334472 conn(0x5630d6a51740 msgr2=0x5630d6e043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.762+0000 7f2c41b47740 1 -- 172.21.15.31:0/283334472 shutdown_connections 2024-09-21T06:20:34.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.762+0000 7f2c41b47740 1 -- 172.21.15.31:0/283334472 wait complete. 2024-09-21T06:20:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.762+0000 7f2c41b47740 1 -- start start 2024-09-21T06:20:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.762+0000 7f2c41b47740 1 Processor -- start 2024-09-21T06:20:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e24c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e23a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5630d6bcb890 con 0x5630d6a5d350 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e24c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5630d6d04d40 con 0x5630d6a34490 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630d6e0aa60 con 0x5630d6a3c670 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e24c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52584/0 (socket says 172.21.15.31:52584) 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c39b70640 1 -- 172.21.15.31:0/438240690 learned_addr learned my addr 172.21.15.31:0/438240690 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c41b47740 10 client.0 _ll_register_callbacks cb 0x5630d6d68c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.763+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c39b70640 1 -- 172.21.15.31:0/438240690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 msgr2=0x5630d6e251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c39b70640 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c39b70640 1 -- 172.21.15.31:0/438240690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 msgr2=0x5630d6e23a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c39b70640 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e23a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c39b70640 1 -- 172.21.15.31:0/438240690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d6e24070 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c39b70640 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e24c30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009f20 tx=0x7f2c2c009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb72dcc796314958 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c1cff9640 1 -- 172.21.15.31:0/438240690 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c028070 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c1cff9640 1 -- 172.21.15.31:0/438240690 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f2c2c004c00 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c004050 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.764+0000 7f2c1cff9640 1 -- 172.21.15.31:0/438240690 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2c2c004e00 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 10 client.27092 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5630d6e2fd30 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c1cff9640 1 -- 172.21.15.31:0/438240690 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c0054f0 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c1cff9640 1 -- 172.21.15.31:0/438240690 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2c2c024810 con 0x5630d6a34490 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 10 client.27092 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:34.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 20 client.27092 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d6a1bcd0 0x5630d6e334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 4 client.27092 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2c2c004050 con 0x5630d6a1bcd0 2024-09-21T06:20:34.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.765+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d6a1bcd0 0x5630d6e334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.766+0000 7f2c1cff9640 10 client.27092 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.766+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d6a1bcd0 0x5630d6e334a0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5dc9f1677324e55 server_cookie=e198b8315c6a0ab4 in_seq=0 out_seq=0 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c1cff9640 1 -- 172.21.15.31:0/438240690 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f2c2c004050 con 0x5630d6a1bcd0 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c1cff9640 10 client.27092 handle_command_reply: tid=0 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 client.27092 shutdown 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 msgr2=0x5630d6e24c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e24c30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009f20 tx=0x7f2c2c009be0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 shutdown_connections 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d6a1bcd0 0x5630d6e334a0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e23a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a3c670 0x5630d6e251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 --2- 172.21.15.31:0/438240690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a34490 0x5630d6e24c30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 >> 172.21.15.31:0/438240690 conn(0x5630d6a51740 msgr2=0x5630d6e01210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c1cff9640 0 client.27092 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.767+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 shutdown_connections 2024-09-21T06:20:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.768+0000 7f2c41b47740 1 -- 172.21.15.31:0/438240690 wait complete. 2024-09-21T06:20:34.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.768+0000 7f2c41b47740 20 client.27092 trim_cache size 0 max 16384 2024-09-21T06:20:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.796+0000 7f2c41b47740 1 Processor -- start 2024-09-21T06:20:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.796+0000 7f2c41b47740 1 -- start start 2024-09-21T06:20:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.796+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e23990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.796+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a3c670 0x5630d6e03f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.796+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a5d350 0x5630d6e04480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c41b47740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341ab670 con 0x5630d6a3c670 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c41b47740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341aded0 con 0x5630d6a5d350 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c41b47740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341abcc0 con 0x5630d6a34490 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e23990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e23990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37670/0 (socket says 172.21.15.31:37670) 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 -- 172.21.15.31:0/3172233685 learned_addr learned my addr 172.21.15.31:0/3172233685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a5d350 0x5630d6e04480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a3c670 0x5630d6e03f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 -- 172.21.15.31:0/3172233685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a5d350 msgr2=0x5630d6e04480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a5d350 0x5630d6e04480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 -- 172.21.15.31:0/3172233685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a3c670 msgr2=0x5630d6e03f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a3c670 0x5630d6e03f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.797+0000 7f2c39b70640 1 -- 172.21.15.31:0/3172233685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d6e0a3a0 con 0x5630d6a34490 2024-09-21T06:20:34.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.798+0000 7f2c39b70640 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e23990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009560 tx=0x7f2c2c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ea8888d7ac0b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.798+0000 7f2c1effd640 1 -- 172.21.15.31:0/3172233685 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c028040 con 0x5630d6a34490 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.798+0000 7f2c1effd640 1 -- 172.21.15.31:0/3172233685 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c024e50 con 0x5630d6a34490 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.798+0000 7f2c1effd640 1 -- 172.21.15.31:0/3172233685 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00f9d0 con 0x5630d6a34490 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 -- 172.21.15.31:0/3172233685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 msgr2=0x5630d6e23990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e23990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c009560 tx=0x7f2c2c0118d0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 -- 172.21.15.31:0/3172233685 shutdown_connections 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a5d350 0x5630d6e04480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a3c670 0x5630d6e03f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 --2- 172.21.15.31:0/3172233685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e23990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 -- 172.21.15.31:0/3172233685 >> 172.21.15.31:0/3172233685 conn(0x5630d6a51740 msgr2=0x5630d6e0b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 -- 172.21.15.31:0/3172233685 shutdown_connections 2024-09-21T06:20:34.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.799+0000 7f2c41b47740 1 -- 172.21.15.31:0/3172233685 wait complete. 2024-09-21T06:20:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 -- start start 2024-09-21T06:20:34.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 Processor -- start 2024-09-21T06:20:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e2c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 0x5630d6e38900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e38e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341ac390 con 0x5630d6a5d350 2024-09-21T06:20:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341ad800 con 0x5630d6a3c670 2024-09-21T06:20:34.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c41b47740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2c341aca60 con 0x5630d6a34490 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.800+0000 7f2c38b6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 0x5630d6e38900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c41b47740 10 client.0 _ll_register_callbacks cb 0x7f2c300eb3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 0x5630d6e38900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52608/0 (socket says 172.21.15.31:52608) 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 -- 172.21.15.31:0/2932488957 learned_addr learned my addr 172.21.15.31:0/2932488957 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c39b70640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e38e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c39b70640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e2c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 -- 172.21.15.31:0/2932488957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 msgr2=0x5630d6e2c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e2c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 -- 172.21.15.31:0/2932488957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 msgr2=0x5630d6e38e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e38e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 -- 172.21.15.31:0/2932488957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d6e395b0 con 0x5630d6a3c670 2024-09-21T06:20:34.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c38b6e640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 0x5630d6e38900 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2c240096c0 tx=0x7f2c24004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957204918b4e2b65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c1cff9640 1 -- 172.21.15.31:0/2932488957 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24049070 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.801+0000 7f2c1cff9640 1 -- 172.21.15.31:0/2932488957 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f2c24005220 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c2c024e50 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c1cff9640 1 -- 172.21.15.31:0/2932488957 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2c24035040 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c41b47740 10 client.27098 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5630d6e9de60 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c1cff9640 1 -- 172.21.15.31:0/2932488957 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c2c024e50 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c1cff9640 1 -- 172.21.15.31:0/2932488957 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2c24031870 con 0x5630d6a3c670 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c41b47740 10 client.27098 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.802+0000 7f2c41b47740 10 client.27098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:34.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.803+0000 7f2c41b47740 20 client.27098 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.803+0000 7f2c41b47740 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d69f1080 0x5630d6ea0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.803+0000 7f2c41b47740 4 client.27098 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:20:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.803+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5630d6d7cdb0 con 0x5630d69f1080 2024-09-21T06:20:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.804+0000 7f2c39b70640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d69f1080 0x5630d6ea0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.807+0000 7f2c39b70640 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d69f1080 0x5630d6ea0650 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce9deee50ae9a19f server_cookie=efdbc2e74a469597 in_seq=0 out_seq=0 2024-09-21T06:20:34.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.807+0000 7f2c1cff9640 10 client.27098 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:34.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c1cff9640 1 -- 172.21.15.31:0/2932488957 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5630d6d7cdb0 con 0x5630d69f1080 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c1cff9640 10 client.27098 handle_command_reply: tid=0 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 client.27098 shutdown 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 msgr2=0x5630d6e38900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 0x5630d6e38900 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2c240096c0 tx=0x7f2c24004430 comp rx=0 tx=0).stop 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 shutdown_connections 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5630d69f1080 0x5630d6ea0650 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5630d6a5d350 0x5630d6e38e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5630d6a3c670 0x5630d6e38900 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 --2- 172.21.15.31:0/2932488957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630d6a34490 0x5630d6e2c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 >> 172.21.15.31:0/2932488957 conn(0x5630d6a51740 msgr2=0x5630d6e0ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c1cff9640 0 client.27098 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:34.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 shutdown_connections 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.809+0000 7f2c41b47740 1 -- 172.21.15.31:0/2932488957 wait complete. 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c41b47740 20 client.27098 trim_cache size 0 max 16384 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "7168e319-4a53-46ce-aaf6-68516fe27218", 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:20:34.813 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:20:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2c200b6700 msgr2=0x7f2c200b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2c200b6700 0x7f2c200b8c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9c0 tx=0x7f2c300ec040 comp rx=0 tx=0).stop 2024-09-21T06:20:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 msgr2=0x7f2c3419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 0x7f2c3419a520 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000e9f0 tx=0x7f2c3000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 shutdown_connections 2024-09-21T06:20:34.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2c200b6700 0x7f2c200b8c10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.810+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c34108cf0 0x7f2c3419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.811+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2c341083a0 0x7f2c3419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.811+0000 7f2c3b5d3640 1 --2- 172.21.15.31:0/446832651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c34102a60 0x7f2c3419a520 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.811+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 >> 172.21.15.31:0/446832651 conn(0x7f2c34079730 msgr2=0x7f2c341052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.811+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 shutdown_connections 2024-09-21T06:20:34.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:34.811+0000 7f2c3b5d3640 1 -- 172.21.15.31:0/446832651 wait complete. 2024-09-21T06:20:35.206 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 535.0 s, 2229.6 tps, lat 44.774 ms stddev 51.030 2024-09-21T06:20:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:35 smithi110 ceph-mon[34550]: pgmap v595: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-09-21T06:20:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:35 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:20:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:35 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:35 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:20:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:35 smithi107 ceph-mon[34380]: pgmap v595: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-09-21T06:20:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:35 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:20:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:35 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:35 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:20:35.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:35 smithi031 ceph-mon[31237]: pgmap v595: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.89k op/s 2024-09-21T06:20:35.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:35 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:20:35.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:35 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:35.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:35 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:20:35.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:20:36.452 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:36.731 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:36 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:36.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:36 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:36.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:36 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 -- 172.21.15.31:0/2150770436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8102230 msgr2=0x7f02a8102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 --2- 172.21.15.31:0/2150770436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8102230 0x7f02a8102640 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f02a000adb0 tx=0x7f02a002b7a0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 -- 172.21.15.31:0/2150770436 shutdown_connections 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 --2- 172.21.15.31:0/2150770436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a81084c0 0x7f02a810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 --2- 172.21.15.31:0/2150770436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a8107b70 0x7f02a8107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 --2- 172.21.15.31:0/2150770436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8102230 0x7f02a8102640 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 -- 172.21.15.31:0/2150770436 >> 172.21.15.31:0/2150770436 conn(0x7f02a8079e40 msgr2=0x7f02a807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 -- 172.21.15.31:0/2150770436 shutdown_connections 2024-09-21T06:20:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.037+0000 7f02acac9640 1 -- 172.21.15.31:0/2150770436 wait complete. 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.038+0000 7f02acac9640 1 Processor -- start 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.038+0000 7f02acac9640 1 -- start start 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02acac9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a8102230 0x7f02a819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02acac9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 0x7f02a819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02acac9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a81084c0 0x7f02a819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02acac9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8111950 con 0x7f02a8107b70 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02acac9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81117d0 con 0x7f02a81084c0 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02acac9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8111ad0 con 0x7f02a8102230 2024-09-21T06:20:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 0x7f02a819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a8102230 0x7f02a819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 0x7f02a819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35156/0 (socket says 172.21.15.31:35156) 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 -- 172.21.15.31:0/4138742715 learned_addr learned my addr 172.21.15.31:0/4138742715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a81084c0 0x7f02a819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 -- 172.21.15.31:0/4138742715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a8102230 msgr2=0x7f02a819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a8102230 0x7f02a819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 -- 172.21.15.31:0/4138742715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a81084c0 msgr2=0x7f02a819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.039+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a81084c0 0x7f02a819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02a5d9b640 1 -- 172.21.15.31:0/4138742715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a81a0460 con 0x7f02a8107b70 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02a659c640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a8102230 0x7f02a819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a81084c0 0x7f02a819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 0x7f02a819aa90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f029800d740 tx=0x7f029800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02977fe640 1 -- 172.21.15.31:0/4138742715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298017070 con 0x7f02a8107b70 2024-09-21T06:20:37.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a81a0750 con 0x7f02a8107b70 2024-09-21T06:20:37.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a8197fd0 con 0x7f02a8107b70 2024-09-21T06:20:37.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02977fe640 1 -- 172.21.15.31:0/4138742715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f029800de50 con 0x7f02a8107b70 2024-09-21T06:20:37.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.040+0000 7f02977fe640 1 -- 172.21.15.31:0/4138742715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298021060 con 0x7f02a8107b70 2024-09-21T06:20:37.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 Processor -- start 2024-09-21T06:20:37.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 -- start start 2024-09-21T06:20:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 0x559bb95f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9222390 0x559bb95f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bb9560e50 con 0x559bb921a1b0 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559bb9550280 con 0x559bb9222390 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.044+0000 7f02b303d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559bb9554090 con 0x559bb9243070 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 0x559bb95f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 0x559bb95f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37702/0 (socket says 172.21.15.31:37702) 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 -- 172.21.15.31:0/2145375109 learned_addr learned my addr 172.21.15.31:0/2145375109 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a659c640 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 -- 172.21.15.31:0/2145375109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9222390 msgr2=0x559bb95f38e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9222390 0x559bb95f38e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 -- 172.21.15.31:0/2145375109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 msgr2=0x559bb95f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a6d9d640 1 -- 172.21.15.31:0/2145375109 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb92ae650 con 0x559bb9243070 2024-09-21T06:20:37.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.045+0000 7f02a659c640 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:37.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.046+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 0x559bb95f2e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f029c011540 tx=0x7f029c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91dbc821a0f35f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.049+0000 7f02977fe640 1 -- 172.21.15.31:0/4138742715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f029801a070 con 0x7f02a8107b70 2024-09-21T06:20:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.049+0000 7f0294ff9640 1 -- 172.21.15.31:0/2145375109 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c00f5c0 con 0x559bb9243070 2024-09-21T06:20:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.049+0000 7f0294ff9640 1 -- 172.21.15.31:0/2145375109 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f029c00fc10 con 0x559bb9243070 2024-09-21T06:20:37.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.049+0000 7f0294ff9640 1 -- 172.21.15.31:0/2145375109 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c019ca0 con 0x559bb9243070 2024-09-21T06:20:37.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 -- 172.21.15.31:0/2145375109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 msgr2=0x559bb95f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 0x559bb95f2e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f029c011540 tx=0x7f029c011a00 comp rx=0 tx=0).stop 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02977fe640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f02900b6780 0x7f02900b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 -- 172.21.15.31:0/2145375109 shutdown_connections 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9222390 0x559bb95f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 --2- 172.21.15.31:0/2145375109 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9243070 0x559bb95f2e60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.050+0000 7f02b303d740 1 -- 172.21.15.31:0/2145375109 >> 172.21.15.31:0/2145375109 conn(0x559bb9237460 msgr2=0x559bb95f0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.051+0000 7f02a659c640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f02900b6780 0x7f02900b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.051+0000 7f02977fe640 1 -- 172.21.15.31:0/4138742715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f02980e7030 con 0x7f02a8107b70 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.051+0000 7f02b303d740 1 -- 172.21.15.31:0/2145375109 shutdown_connections 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.051+0000 7f02b303d740 1 -- 172.21.15.31:0/2145375109 wait complete. 2024-09-21T06:20:37.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.051+0000 7f02b303d740 1 -- start start 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02a659c640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f02900b6780 0x7f02900b8c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006010 tx=0x7f02a003e040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 Processor -- start 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9222390 0x559bb95fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 0x559bb960a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bb93b1700 con 0x559bb921a1b0 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559bb94eb110 con 0x559bb9243070 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559bb9601640 con 0x559bb9222390 2024-09-21T06:20:37.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.052+0000 7f02b303d740 10 client.0 _ll_register_callbacks cb 0x559bb9504990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:37.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.054+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 0x559bb960a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.054+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9222390 0x559bb95fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.054+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 0x559bb960a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52640/0 (socket says 172.21.15.31:52640) 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.054+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9222390 0x559bb95fead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37714/0 (socket says 172.21.15.31:37714) 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.054+0000 7f02a6d9d640 1 -- 172.21.15.31:0/4051603761 learned_addr learned my addr 172.21.15.31:0/4051603761 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a659c640 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a6d9d640 1 -- 172.21.15.31:0/4051603761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9222390 msgr2=0x559bb95fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9222390 0x559bb95fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a6d9d640 1 -- 172.21.15.31:0/4051603761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 msgr2=0x559bb95fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a6d9d640 1 -- 172.21.15.31:0/4051603761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb9518c40 con 0x559bb9243070 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f02a6d9d640 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 0x559bb960a960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f029c0057c0 tx=0x7f029c009d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f1c2462c4f06c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.055+0000 7f028dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f028dffb640 1 -- 172.21.15.31:0/4051603761 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c026020 con 0x559bb9243070 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f029c00fc10 con 0x559bb9243070 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f028dffb640 1 -- 172.21.15.31:0/4051603761 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f029c019df0 con 0x559bb9243070 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f028dffb640 1 -- 172.21.15.31:0/4051603761 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f029c031050 con 0x559bb9243070 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f02b303d740 10 client.27107 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559bb960d390 con 0x559bb9243070 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f028dffb640 1 -- 172.21.15.31:0/4051603761 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f029c00f5c0 con 0x559bb9243070 2024-09-21T06:20:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.056+0000 7f028dffb640 1 -- 172.21.15.31:0/4051603761 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f029c003920 con 0x559bb9243070 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02b303d740 10 client.27107 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02b303d740 10 client.27107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02b303d740 20 client.27107 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02b303d740 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb92019f0 0x559bb9610b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02b303d740 4 client.27107 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f029c00fc10 con 0x559bb92019f0 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.057+0000 7f02a659c640 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb92019f0 0x559bb9610b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.058+0000 7f028dffb640 10 client.27107 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.058+0000 7f02a659c640 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb92019f0 0x559bb9610b70 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c6b7fa2798e1dc server_cookie=691750e64f24c886 in_seq=0 out_seq=0 2024-09-21T06:20:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f028dffb640 1 -- 172.21.15.31:0/4051603761 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f029c00fc10 con 0x559bb92019f0 2024-09-21T06:20:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f028dffb640 10 client.27107 handle_command_reply: tid=0 2024-09-21T06:20:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 client.27107 shutdown 2024-09-21T06:20:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 msgr2=0x559bb960a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 0x559bb960a960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f029c0057c0 tx=0x7f029c009d70 comp rx=0 tx=0).stop 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 shutdown_connections 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb92019f0 0x559bb9610b70 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb9243070 0x559bb960a960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f028dffb640 0 client.27107 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb9222390 0x559bb95fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 --2- 172.21.15.31:0/4051603761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb921a1b0 0x559bb95fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 >> 172.21.15.31:0/4051603761 conn(0x559bb9237460 msgr2=0x559bb95f5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.059+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 shutdown_connections 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.060+0000 7f02b303d740 1 -- 172.21.15.31:0/4051603761 wait complete. 2024-09-21T06:20:37.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.061+0000 7f02b303d740 20 client.27107 trim_cache size 0 max 16384 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.089+0000 7f02b303d740 1 Processor -- start 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 -- start start 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb95f4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb960a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb95fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81ab6f0 con 0x559bb9222390 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81adf50 con 0x559bb92019f0 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02b303d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81abd40 con 0x559bb921a1b0 2024-09-21T06:20:37.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb95fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb960a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb95fb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35188/0 (socket says 172.21.15.31:35188) 2024-09-21T06:20:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a6d9d640 1 -- 172.21.15.31:0/2512018007 learned_addr learned my addr 172.21.15.31:0/2512018007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:37.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a659c640 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb95f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a5d9b640 1 -- 172.21.15.31:0/2512018007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 msgr2=0x559bb95f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb95f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a5d9b640 1 -- 172.21.15.31:0/2512018007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 msgr2=0x559bb95fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb95fb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a5d9b640 1 -- 172.21.15.31:0/2512018007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bb95ff630 con 0x559bb921a1b0 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.090+0000 7f02a659c640 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb95f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.091+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb960a860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f02980e4730 tx=0x7f02980ac650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa477712ea9ab5c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.091+0000 7f028ffff640 1 -- 172.21.15.31:0/2512018007 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02980b6850 con 0x559bb921a1b0 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.091+0000 7f028ffff640 1 -- 172.21.15.31:0/2512018007 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02980a9150 con 0x559bb921a1b0 2024-09-21T06:20:37.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.091+0000 7f028ffff640 1 -- 172.21.15.31:0/2512018007 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02980a8b20 con 0x559bb921a1b0 2024-09-21T06:20:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.092+0000 7f02b303d740 1 -- 172.21.15.31:0/2512018007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 msgr2=0x559bb960a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.092+0000 7f02b303d740 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb960a860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f02980e4730 tx=0x7f02980ac650 comp rx=0 tx=0).stop 2024-09-21T06:20:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.093+0000 7f02b303d740 1 -- 172.21.15.31:0/2512018007 shutdown_connections 2024-09-21T06:20:37.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.093+0000 7f02b303d740 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb95fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.093+0000 7f02b303d740 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb960a860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.093+0000 7f02b303d740 1 --2- 172.21.15.31:0/2512018007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb95f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.093+0000 7f02b303d740 1 -- 172.21.15.31:0/2512018007 >> 172.21.15.31:0/2512018007 conn(0x559bb9237460 msgr2=0x559bb95fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.094+0000 7f02b303d740 1 -- 172.21.15.31:0/2512018007 shutdown_connections 2024-09-21T06:20:37.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.094+0000 7f02b303d740 1 -- 172.21.15.31:0/2512018007 wait complete. 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.094+0000 7f02b303d740 1 -- start start 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.094+0000 7f02b303d740 1 Processor -- start 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb9607160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb9602610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb9602c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81ac410 con 0x559bb9222390 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81ad880 con 0x559bb92019f0 2024-09-21T06:20:37.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f02a81acae0 con 0x559bb921a1b0 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02b303d740 10 client.0 _ll_register_callbacks cb 0x7f02a000ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb9602610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb9602c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb9602610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:37732/0 (socket says 172.21.15.31:37732) 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 -- 172.21.15.31:0/1991800971 learned_addr learned my addr 172.21.15.31:0/1991800971 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a659c640 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb9607160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 -- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 msgr2=0x559bb9607160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb9607160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 -- 172.21.15.31:0/1991800971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 msgr2=0x559bb9602c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb9602c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.095+0000 7f02a5d9b640 1 -- 172.21.15.31:0/1991800971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bb96033f0 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.096+0000 7f02a5d9b640 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb9602610 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0298000c00 tx=0x7f0298009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4049c83cd6cc1f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.096+0000 7f028dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.096+0000 7f028dffb640 1 -- 172.21.15.31:0/1991800971 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029805c020 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.096+0000 7f028dffb640 1 -- 172.21.15.31:0/1991800971 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f029805c1c0 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.096+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02980a9150 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.096+0000 7f028dffb640 1 -- 172.21.15.31:0/1991800971 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f029805c3c0 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 10 client.27145 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559bb967b570 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f028dffb640 1 -- 172.21.15.31:0/1991800971 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02980b6850 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f028dffb640 1 -- 172.21.15.31:0/1991800971 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f02980abbb0 con 0x559bb921a1b0 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 10 client.27145 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 10 client.27145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:37.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 20 client.27145 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:37.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb9243070 0x559bb967dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:37.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 4 client.27145 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:20:37.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f02980a9150 con 0x559bb9243070 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.097+0000 7f02a659c640 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb9243070 0x559bb967dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.098+0000 7f02a659c640 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb9243070 0x559bb967dc30 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3579a12ab2a25195 server_cookie=d4bb94fbfb796f8b in_seq=0 out_seq=0 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.099+0000 7f028dffb640 10 client.27145 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.099+0000 7f028dffb640 1 -- 172.21.15.31:0/1991800971 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x7f02980a9150 con 0x559bb9243070 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.099+0000 7f028dffb640 10 client.27145 handle_command_reply: tid=0 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.099+0000 7f02b303d740 1 client.27145 shutdown 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 msgr2=0x559bb9602610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb9602610 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0298000c00 tx=0x7f0298009f50 comp rx=0 tx=0).stop 2024-09-21T06:20:37.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 shutdown_connections 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bb9243070 0x559bb967dc30 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bb9222390 0x559bb9602c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bb921a1b0 0x559bb9602610 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 --2- 172.21.15.31:0/1991800971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bb92019f0 0x559bb9607160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 >> 172.21.15.31:0/1991800971 conn(0x559bb9237460 msgr2=0x559bb95f5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f028dffb640 0 client.27145 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 shutdown_connections 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 1 -- 172.21.15.31:0/1991800971 wait complete. 2024-09-21T06:20:37.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.100+0000 7f02b303d740 20 client.27145 trim_cache size 0 max 16384 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:20:37.106 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:20:37.109 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:20:37.112 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.116 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:37.117 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f02900b6780 msgr2=0x7f02900b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f02900b6780 0x7f02900b8c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006010 tx=0x7f02a003e040 comp rx=0 tx=0).stop 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 msgr2=0x7f02a819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 0x7f02a819aa90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f029800d740 tx=0x7f029800dc00 comp rx=0 tx=0).stop 2024-09-21T06:20:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 shutdown_connections 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a81084c0 0x7f02a819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f02900b6780 0x7f02900b8c90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02a8107b70 0x7f02a819aa90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 --2- 172.21.15.31:0/4138742715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02a8102230 0x7f02a819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 >> 172.21.15.31:0/4138742715 conn(0x7f02a8079e40 msgr2=0x7f02a8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 shutdown_connections 2024-09-21T06:20:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:37.101+0000 7f02acac9640 1 -- 172.21.15.31:0/4138742715 wait complete. 2024-09-21T06:20:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:37 smithi110 ceph-mon[34550]: pgmap v596: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 1.89k op/s 2024-09-21T06:20:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:37 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:20:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:37 smithi107 ceph-mon[34380]: pgmap v596: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 1.89k op/s 2024-09-21T06:20:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:37 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:20:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:37 smithi031 ceph-mon[31237]: pgmap v596: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 1.89k op/s 2024-09-21T06:20:37.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:37 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:20:38.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:20:38.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:20:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.010+0000 7fcfbfec6640 1 -- 172.21.15.31:0/3612912091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb8108cf0 msgr2=0x7fcfb810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/3612912091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb8108cf0 0x7fcfb810f520 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009ca0 tx=0x7fcfb002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 -- 172.21.15.31:0/3612912091 shutdown_connections 2024-09-21T06:20:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/3612912091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb8108cf0 0x7fcfb810f520 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/3612912091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb81083a0 0x7fcfb81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/3612912091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8102a60 0x7fcfb8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 -- 172.21.15.31:0/3612912091 >> 172.21.15.31:0/3612912091 conn(0x7fcfb8079730 msgr2=0x7fcfb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 -- 172.21.15.31:0/3612912091 shutdown_connections 2024-09-21T06:20:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.011+0000 7fcfbfec6640 1 -- 172.21.15.31:0/3612912091 wait complete. 2024-09-21T06:20:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.012+0000 7fcfbfec6640 1 Processor -- start 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.013+0000 7fcfbfec6640 1 -- start start 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbfec6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb8102a60 0x7fcfb819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbfec6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb81083a0 0x7fcfb819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbfec6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 0x7fcfb819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbfec6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8112960 con 0x7fcfb81083a0 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbfec6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb81127e0 con 0x7fcfb8108cf0 2024-09-21T06:20:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbfec6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8112ae0 con 0x7fcfb8102a60 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbe463640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 0x7fcfb819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbe463640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 0x7fcfb819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52670/0 (socket says 172.21.15.31:52670) 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbd461640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb81083a0 0x7fcfb819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbe463640 1 -- 172.21.15.31:0/2155883961 learned_addr learned my addr 172.21.15.31:0/2155883961 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.014+0000 7fcfbdc62640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb8102a60 0x7fcfb819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbe463640 1 -- 172.21.15.31:0/2155883961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb8102a60 msgr2=0x7fcfb819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbe463640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb8102a60 0x7fcfb819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbe463640 1 -- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb81083a0 msgr2=0x7fcfb819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbe463640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb81083a0 0x7fcfb819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbe463640 1 -- 172.21.15.31:0/2155883961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb81a03e0 con 0x7fcfb8108cf0 2024-09-21T06:20:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbdc62640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb8102a60 0x7fcfb819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:20:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.015+0000 7fcfbe463640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 0x7fcfb819fce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006010 tx=0x7fcfb0031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.016+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003b70 con 0x7fcfb8108cf0 2024-09-21T06:20:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.016+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb81a0670 con 0x7fcfb8108cf0 2024-09-21T06:20:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.016+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb80ff550 con 0x7fcfb8108cf0 2024-09-21T06:20:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.016+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb00318f0 con 0x7fcfb8108cf0 2024-09-21T06:20:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.016+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0004860 con 0x7fcfb8108cf0 2024-09-21T06:20:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.018+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c0052c0 con 0x7fcfb8108cf0 2024-09-21T06:20:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.019+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcfb0004040 con 0x7fcfb8108cf0 2024-09-21T06:20:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.020+0000 7fcfaeffd640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcfa40b6780 0x7fcfa40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.020+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fcfb0108940 con 0x7fcfb8108cf0 2024-09-21T06:20:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.020+0000 7fcfbdc62640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcfa40b6780 0x7fcfa40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.021+0000 7fcfbdc62640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcfa40b6780 0x7fcfa40b8c90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4012b90 tx=0x7fcfb4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.025+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcfb00cee40 con 0x7fcfb8108cf0 2024-09-21T06:20:39.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.409+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf9c005880 con 0x7fcfb8108cf0 2024-09-21T06:20:39.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.410+0000 7fcfaeffd640 1 -- 172.21.15.31:0/2155883961 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7fcfb00d3220 con 0x7fcfb8108cf0 2024-09-21T06:20:39.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:20:39.413 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:20:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:20:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcfa40b6780 msgr2=0x7fcfa40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcfa40b6780 0x7fcfa40b8c90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4012b90 tx=0x7fcfb4005f90 comp rx=0 tx=0).stop 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 msgr2=0x7fcfb819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 0x7fcfb819fce0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006010 tx=0x7fcfb0031700 comp rx=0 tx=0).stop 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 shutdown_connections 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfb8108cf0 0x7fcfb819fce0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fcfa40b6780 0x7fcfa40b8c90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfb81083a0 0x7fcfb819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 --2- 172.21.15.31:0/2155883961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfb8102a60 0x7fcfb819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 >> 172.21.15.31:0/2155883961 conn(0x7fcfb8079730 msgr2=0x7fcfb806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.413+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 shutdown_connections 2024-09-21T06:20:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:39.414+0000 7fcfbfec6640 1 -- 172.21.15.31:0/2155883961 wait complete. 2024-09-21T06:20:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:39 smithi110 ceph-mon[34550]: pgmap v597: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 21 MiB/s wr, 1.81k op/s 2024-09-21T06:20:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:39 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2155883961' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:39 smithi107 ceph-mon[34380]: pgmap v597: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 21 MiB/s wr, 1.81k op/s 2024-09-21T06:20:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:39 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2155883961' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:39.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:39 smithi031 ceph-mon[31237]: pgmap v597: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 21 MiB/s wr, 1.81k op/s 2024-09-21T06:20:39.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:39 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2155883961' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:20:40.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:20:40.206 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 540.0 s, 1844.4 tps, lat 53.795 ms stddev 57.023 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 -- 172.21.15.31:0/2875006024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c1165a0 msgr2=0x7f2e6c118a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 --2- 172.21.15.31:0/2875006024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c1165a0 0x7f2e6c118a80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009900 tx=0x7f2e5c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 -- 172.21.15.31:0/2875006024 shutdown_connections 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 --2- 172.21.15.31:0/2875006024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c1165a0 0x7f2e6c118a80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 --2- 172.21.15.31:0/2875006024 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 --2- 172.21.15.31:0/2875006024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e6c10a5a0 0x7f2e6c10a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 -- 172.21.15.31:0/2875006024 >> 172.21.15.31:0/2875006024 conn(0x7f2e6c070dd0 msgr2=0x7f2e6c0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 -- 172.21.15.31:0/2875006024 shutdown_connections 2024-09-21T06:20:40.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.302+0000 7f2e7104d640 1 -- 172.21.15.31:0/2875006024 wait complete. 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 Processor -- start 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 -- start start 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e6c10a5a0 0x7f2e6c1a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c1a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e6c1165a0 0x7f2e6c1a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c11c200 con 0x7f2e6c10a5a0 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c11c080 con 0x7f2e6c10aef0 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.303+0000 7f2e7104d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6c11c380 con 0x7f2e6c1165a0 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c1a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c1a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33002/0 (socket says 172.21.15.31:33002) 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 -- 172.21.15.31:0/913859265 learned_addr learned my addr 172.21.15.31:0/913859265 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e6c10a5a0 0x7f2e6c1a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e6c1165a0 0x7f2e6c1a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 -- 172.21.15.31:0/913859265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e6c1165a0 msgr2=0x7f2e6c1a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e6c1165a0 0x7f2e6c1a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 -- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e6c10a5a0 msgr2=0x7f2e6c1a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e6c10a5a0 0x7f2e6c1a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 -- 172.21.15.31:0/913859265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6c1a8b50 con 0x7f2e6c10aef0 2024-09-21T06:20:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c1a31c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000d740 tx=0x7f2e6000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.304+0000 7f2e53fff640 1 -- 172.21.15.31:0/913859265 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60017070 con 0x7f2e6c10aef0 2024-09-21T06:20:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.305+0000 7f2e53fff640 1 -- 172.21.15.31:0/913859265 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6000de50 con 0x7f2e6c10aef0 2024-09-21T06:20:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.305+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6c1a8e40 con 0x7f2e6c10aef0 2024-09-21T06:20:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.305+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e6c1af5d0 con 0x7f2e6c10aef0 2024-09-21T06:20:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.305+0000 7f2e53fff640 1 -- 172.21.15.31:0/913859265 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000f620 con 0x7f2e6c10aef0 2024-09-21T06:20:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.308+0000 7f2e77fbb740 1 Processor -- start 2024-09-21T06:20:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.311+0000 7f2e77fbb740 1 -- start start 2024-09-21T06:20:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.311+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185b71b0 0x56361895a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.311+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 0x56361895a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.311+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185964d0 0x56361896cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5636188d16f0 con 0x5636185964d0 2024-09-21T06:20:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5636188c0b20 con 0x5636185b71b0 2024-09-21T06:20:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5636188c4930 con 0x56361858e2f0 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 0x56361895a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 0x56361895a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47966/0 (socket says 172.21.15.31:47966) 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/2623826981 learned_addr learned my addr 172.21.15.31:0/2623826981 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185b71b0 0x56361895a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/2623826981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185b71b0 msgr2=0x56361895a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185b71b0 0x56361895a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/2623826981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185964d0 msgr2=0x56361896cd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185964d0 0x56361896cd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/2623826981 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563618886530 con 0x56361858e2f0 2024-09-21T06:20:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.312+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185b71b0 0x56361895a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.313+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 0x56361895a9a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e640114f0 tx=0x7f2e640119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3f61a281489610f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.313+0000 7f2e53fff640 1 -- 172.21.15.31:0/913859265 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2e6001a070 con 0x7f2e6c10aef0 2024-09-21T06:20:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.313+0000 7f2e517fa640 1 -- 172.21.15.31:0/2623826981 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6401e070 con 0x56361858e2f0 2024-09-21T06:20:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.313+0000 7f2e517fa640 1 -- 172.21.15.31:0/2623826981 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6400fad0 con 0x56361858e2f0 2024-09-21T06:20:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.313+0000 7f2e517fa640 1 -- 172.21.15.31:0/2623826981 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64019b10 con 0x56361858e2f0 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e53fff640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e540b6700 0x7f2e540b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e53fff640 1 -- 172.21.15.31:0/913859265 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f2e600e6ea0 con 0x7f2e6c10aef0 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e540b6700 0x7f2e540b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 -- 172.21.15.31:0/2623826981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 msgr2=0x56361895a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 0x56361895a9a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e640114f0 tx=0x7f2e640119b0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 -- 172.21.15.31:0/2623826981 shutdown_connections 2024-09-21T06:20:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185964d0 0x56361896cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56361858e2f0 0x56361895a9a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/2623826981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185b71b0 0x56361895a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.314+0000 7f2e77fbb740 1 -- 172.21.15.31:0/2623826981 >> 172.21.15.31:0/2623826981 conn(0x5636185ab300 msgr2=0x56361895c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.315+0000 7f2e77fbb740 1 -- 172.21.15.31:0/2623826981 shutdown_connections 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.315+0000 7f2e77fbb740 1 -- 172.21.15.31:0/2623826981 wait complete. 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.315+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e540b6700 0x7f2e540b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005940 tx=0x7f2e5c0058b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.315+0000 7f2e77fbb740 1 -- start start 2024-09-21T06:20:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.315+0000 7f2e77fbb740 1 Processor -- start 2024-09-21T06:20:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 0x5636189711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636185964d0 0x563618971790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185b71b0 0x5636189767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5636187f8ab0 con 0x5636185b71b0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563618866730 con 0x56361858e2f0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56361896d8a0 con 0x5636185964d0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e77fbb740 10 client.0 _ll_register_callbacks cb 0x5636188ba6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 0x5636189711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 0x5636189711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33024/0 (socket says 172.21.15.31:33024) 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/4119472352 learned_addr learned my addr 172.21.15.31:0/4119472352 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636185964d0 0x563618971790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.316+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185b71b0 0x5636189767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/4119472352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636185964d0 msgr2=0x563618971790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636185964d0 0x563618971790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/4119472352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185b71b0 msgr2=0x5636189767a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185b71b0 0x5636189767a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/4119472352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563618622790 con 0x56361858e2f0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185b71b0 0x5636189767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 0x5636189711b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e640114c0 tx=0x7f2e6400f620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e851959ee73c991 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e4eff5640 1 -- 172.21.15.31:0/4119472352 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64029060 con 0x56361858e2f0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e4eff5640 1 -- 172.21.15.31:0/4119472352 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f2e6400f8d0 con 0x56361858e2f0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6400fad0 con 0x56361858e2f0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e4eff5640 1 -- 172.21.15.31:0/4119472352 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e6401e040 con 0x56361858e2f0 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e77fbb740 10 client.27131 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.317+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56361869b620 con 0x56361858e2f0 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e4eff5640 1 -- 172.21.15.31:0/4119472352 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6400fad0 con 0x56361858e2f0 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e4eff5640 1 -- 172.21.15.31:0/4119472352 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2e64005040 con 0x56361858e2f0 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e77fbb740 10 client.27131 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e77fbb740 10 client.27131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e77fbb740 20 client.27131 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563618575d80 0x563618982930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e77fbb740 4 client.27131 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636188d3640 con 0x563618575d80 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563618575d80 0x563618982930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.318+0000 7f2e4eff5640 10 client.27131 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.319+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563618575d80 0x563618982930 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ace44494c93c415 server_cookie=d94ca9700ae0b9c2 in_seq=0 out_seq=0 2024-09-21T06:20:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e4eff5640 1 -- 172.21.15.31:0/4119472352 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636188d3640 con 0x563618575d80 2024-09-21T06:20:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e4eff5640 10 client.27131 handle_command_reply: tid=0 2024-09-21T06:20:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 client.27131 shutdown 2024-09-21T06:20:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 msgr2=0x5636189711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 0x5636189711b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2e640114c0 tx=0x7f2e6400f620 comp rx=0 tx=0).stop 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 shutdown_connections 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563618575d80 0x563618982930 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e4eff5640 0 client.27131 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5636185b71b0 0x5636189767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5636185964d0 0x563618971790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/4119472352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56361858e2f0 0x5636189711b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 >> 172.21.15.31:0/4119472352 conn(0x5636185ab300 msgr2=0x563618959860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 shutdown_connections 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 1 -- 172.21.15.31:0/4119472352 wait complete. 2024-09-21T06:20:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.320+0000 7f2e77fbb740 20 client.27131 trim_cache size 0 max 16384 2024-09-21T06:20:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.339+0000 7f2e77fbb740 1 Processor -- start 2024-09-21T06:20:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.340+0000 7f2e77fbb740 1 -- start start 2024-09-21T06:20:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.340+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361896eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361896eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361896eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47970/0 (socket says 172.21.15.31:47970) 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/3673080778 learned_addr learned my addr 172.21.15.31:0/3673080778 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5800c7c0 con 0x56361858e2f0 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5800f020 con 0x5636185964d0 2024-09-21T06:20:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.341+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5800ce10 con 0x563618575d80 2024-09-21T06:20:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/3673080778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 msgr2=0x5636189555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/3673080778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 msgr2=0x56361897d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/3673080778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56361895b190 con 0x563618575d80 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.342+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361896eb50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2e640114f0 tx=0x7f2e64005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac6e5532cd803628 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.343+0000 7f2e50ff9640 1 -- 172.21.15.31:0/3673080778 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64029070 con 0x563618575d80 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.343+0000 7f2e50ff9640 1 -- 172.21.15.31:0/3673080778 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e64005240 con 0x563618575d80 2024-09-21T06:20:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.343+0000 7f2e50ff9640 1 -- 172.21.15.31:0/3673080778 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e640196e0 con 0x563618575d80 2024-09-21T06:20:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 msgr2=0x56361896eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361896eb50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2e640114f0 tx=0x7f2e64005cf0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 shutdown_connections 2024-09-21T06:20:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/3673080778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361896eb50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 >> 172.21.15.31:0/3673080778 conn(0x5636185ab300 msgr2=0x56361896c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 shutdown_connections 2024-09-21T06:20:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.344+0000 7f2e77fbb740 1 -- 172.21.15.31:0/3673080778 wait complete. 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.345+0000 7f2e77fbb740 1 -- start start 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.345+0000 7f2e77fbb740 1 Processor -- start 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361897a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189e51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5800d4e0 con 0x56361858e2f0 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5800e950 con 0x5636185964d0 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5800dbb0 con 0x563618575d80 2024-09-21T06:20:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e77fbb740 10 client.0 _ll_register_callbacks cb 0x7f2e6401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361897a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361897a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:47974/0 (socket says 172.21.15.31:47974) 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/901591077 learned_addr learned my addr 172.21.15.31:0/901591077 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6b59e640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189e51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 msgr2=0x5636189e51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189e51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/901591077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 msgr2=0x56361897abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 -- 172.21.15.31:0/901591077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636189e5840 con 0x563618575d80 2024-09-21T06:20:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:20:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.346+0000 7f2e6ad9d640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361897a640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005b40 tx=0x7f2e64009d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45cb5a1b3be52afe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e4eff5640 1 -- 172.21.15.31:0/901591077 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64029040 con 0x563618575d80 2024-09-21T06:20:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e4eff5640 1 -- 172.21.15.31:0/901591077 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f2e64003b70 con 0x563618575d80 2024-09-21T06:20:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e64005240 con 0x563618575d80 2024-09-21T06:20:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e4eff5640 1 -- 172.21.15.31:0/901591077 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e64003d70 con 0x563618575d80 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e77fbb740 10 client.27166 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5636189ecc60 con 0x563618575d80 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.347+0000 7f2e4eff5640 1 -- 172.21.15.31:0/901591077 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e64005240 con 0x563618575d80 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e4eff5640 1 -- 172.21.15.31:0/901591077 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2e64035070 con 0x563618575d80 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e77fbb740 10 client.27166 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e77fbb740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e77fbb740 20 client.27166 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5636185b71b0 0x5636189ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e77fbb740 4 client.27166 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636188cf0f0 con 0x5636185b71b0 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.348+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5636185b71b0 0x5636189ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.349+0000 7f2e4eff5640 10 client.27166 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:40.349+0000 7f2e6a59c640 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5636185b71b0 0x5636189ef270 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c86f46dae2f19a server_cookie=1aaa904efd2680d7 in_seq=0 out_seq=0 2024-09-21T06:20:40.650 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.184+0000 7fe8a87a9640 1 -- 172.21.15.31:0/2676847310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a0103100 msgr2=0x7fe8a0107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.184+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/2676847310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a0103100 0x7fe8a0107630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe890009930 tx=0x7fe89002a420 comp rx=0 tx=0).stop 2024-09-21T06:20:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 -- 172.21.15.31:0/2676847310 shutdown_connections 2024-09-21T06:20:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/2676847310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0107bf0 0x7fe8a01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/2676847310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a0103100 0x7fe8a0107630 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/2676847310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a01027b0 0x7fe8a0102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 -- 172.21.15.31:0/2676847310 >> 172.21.15.31:0/2676847310 conn(0x7fe8a0079730 msgr2=0x7fe8a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 -- 172.21.15.31:0/2676847310 shutdown_connections 2024-09-21T06:20:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.185+0000 7fe8a87a9640 1 -- 172.21.15.31:0/2676847310 wait complete. 2024-09-21T06:20:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 Processor -- start 2024-09-21T06:20:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 -- start start 2024-09-21T06:20:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a01027b0 0x7fe8a01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0103100 0x7fe8a01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a0107bf0 0x7fe8a01a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a011a050 con 0x7fe8a0103100 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0119ed0 con 0x7fe8a01027b0 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a87a9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a011a1d0 con 0x7fe8a0107bf0 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0103100 0x7fe8a01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.186+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0103100 0x7fe8a01a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43312/0 (socket says 172.21.15.31:43312) 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a5d44640 1 -- 172.21.15.31:0/1873984488 learned_addr learned my addr 172.21.15.31:0/1873984488 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a0107bf0 0x7fe8a01a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6545640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a01027b0 0x7fe8a01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 -- 172.21.15.31:0/1873984488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a01027b0 msgr2=0x7fe8a01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a01027b0 0x7fe8a01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 -- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0103100 msgr2=0x7fe8a01a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0103100 0x7fe8a01a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 -- 172.21.15.31:0/1873984488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a01a6a50 con 0x7fe8a0107bf0 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.187+0000 7fe8a6d46640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a0107bf0 0x7fe8a01a6320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe89800da10 tx=0x7fe89800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.188+0000 7fe8977fe640 1 -- 172.21.15.31:0/1873984488 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89800bd70 con 0x7fe8a0107bf0 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.188+0000 7fe8977fe640 1 -- 172.21.15.31:0/1873984488 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe898003e20 con 0x7fe8a0107bf0 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.188+0000 7fe8977fe640 1 -- 172.21.15.31:0/1873984488 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8980106c0 con 0x7fe8a0107bf0 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.188+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a01a6d40 con 0x7fe8a0107bf0 2024-09-21T06:20:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.188+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01ad4a0 con 0x7fe8a0107bf0 2024-09-21T06:20:41.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.191+0000 7fe8977fe640 1 -- 172.21.15.31:0/1873984488 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe898004270 con 0x7fe8a0107bf0 2024-09-21T06:20:41.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.192+0000 7fe8aed1d740 1 Processor -- start 2024-09-21T06:20:41.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.192+0000 7fe8aed1d740 1 -- start start 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.192+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557ff864bea0 con 0x557ff830d390 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ff863b2d0 con 0x557ff832e070 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557ff863f0e0 con 0x557ff83051b0 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86d56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33060/0 (socket says 172.21.15.31:33060) 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2122278382 learned_addr learned my addr 172.21.15.31:0/2122278382 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8977fe640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe88c0b67d0 0x7fe88c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2122278382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 msgr2=0x557ff86d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2122278382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 msgr2=0x557ff86d6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2122278382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ff8399650 con 0x557ff832e070 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.193+0000 7fe8a6d46640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe88c0b67d0 0x7fe88c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.194+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86d56a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe890009230 tx=0x7fe890005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8351a5433d2d609c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.194+0000 7fe8977fe640 1 -- 172.21.15.31:0/1873984488 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fe8980e79a0 con 0x7fe8a0107bf0 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.194+0000 7fe894ff9640 1 -- 172.21.15.31:0/2122278382 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890035070 con 0x557ff832e070 2024-09-21T06:20:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.194+0000 7fe894ff9640 1 -- 172.21.15.31:0/2122278382 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe890005020 con 0x557ff832e070 2024-09-21T06:20:41.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.194+0000 7fe894ff9640 1 -- 172.21.15.31:0/2122278382 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8900044e0 con 0x557ff832e070 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8a6d46640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe88c0b67d0 0x7fe88c0b8ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe8980e51b0 tx=0x7fe898071e10 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2122278382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 msgr2=0x557ff86d56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86d56a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe890009230 tx=0x7fe890005e30 comp rx=0 tx=0).stop 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2122278382 shutdown_connections 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2122278382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86d56a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2122278382 >> 172.21.15.31:0/2122278382 conn(0x557ff8322460 msgr2=0x557ff86d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:41.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2122278382 shutdown_connections 2024-09-21T06:20:41.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.197+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2122278382 wait complete. 2024-09-21T06:20:41.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 -- start start 2024-09-21T06:20:41.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 Processor -- start 2024-09-21T06:20:41.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557ff849c700 con 0x557ff830d390 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ff85d6160 con 0x557ff832e070 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557ff86e3e20 con 0x557ff83051b0 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8aed1d740 10 client.0 _ll_register_callbacks cb 0x557ff85ef9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86ed140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33068/0 (socket says 172.21.15.31:33068) 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2330088210 learned_addr learned my addr 172.21.15.31:0/2330088210 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.198+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2330088210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 msgr2=0x557ff86e0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86e0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2330088210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 msgr2=0x557ff86e1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2330088210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ff8603c90 con 0x557ff832e070 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86ed140 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe890009340 tx=0x7fe890041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ef0c7d3880f9ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe889ffb640 1 -- 172.21.15.31:0/2330088210 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890035070 con 0x557ff832e070 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe889ffb640 1 -- 172.21.15.31:0/2330088210 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fe8900053d0 con 0x557ff832e070 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.199+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe890005020 con 0x557ff832e070 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe889ffb640 1 -- 172.21.15.31:0/2330088210 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe8900055d0 con 0x557ff832e070 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 10 client.27149 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557ff86efb40 con 0x557ff832e070 2024-09-21T06:20:41.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe889ffb640 1 -- 172.21.15.31:0/2330088210 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe890005020 con 0x557ff832e070 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe889ffb640 1 -- 172.21.15.31:0/2330088210 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe8900035f0 con 0x557ff832e070 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 10 client.27149 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 20 client.27149 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff82ec9f0 0x557ff86f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 4 client.27149 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ff864ddf0 con 0x557ff82ec9f0 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.200+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff82ec9f0 0x557ff86f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.201+0000 7fe889ffb640 10 client.27149 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:41.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.201+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff82ec9f0 0x557ff86f2ea0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eaa2e2cf2d3c7b5 server_cookie=85a1ab95db8b6e54 in_seq=0 out_seq=0 2024-09-21T06:20:41.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe889ffb640 1 -- 172.21.15.31:0/2330088210 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ff864ddf0 con 0x557ff82ec9f0 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe889ffb640 10 client.27149 handle_command_reply: tid=0 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 client.27149 shutdown 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 msgr2=0x557ff86ed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86ed140 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe890009340 tx=0x7fe890041040 comp rx=0 tx=0).stop 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 shutdown_connections 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff82ec9f0 0x557ff86f2ea0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff832e070 0x557ff86ed140 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe889ffb640 0 client.27149 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2330088210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.203+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 >> 172.21.15.31:0/2330088210 conn(0x557ff8322460 msgr2=0x557ff86d8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:41.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.204+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 shutdown_connections 2024-09-21T06:20:41.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.204+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2330088210 wait complete. 2024-09-21T06:20:41.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.204+0000 7fe8aed1d740 20 client.27149 trim_cache size 0 max 16384 2024-09-21T06:20:41.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.223+0000 7fe8aed1d740 1 Processor -- start 2024-09-21T06:20:41.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.223+0000 7fe8aed1d740 1 -- start start 2024-09-21T06:20:41.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.223+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff82ec9f0 0x557ff86e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.223+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.224+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.224+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b3cf0 con 0x557ff830d390 2024-09-21T06:20:41.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.224+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b6550 con 0x557ff82ec9f0 2024-09-21T06:20:41.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.224+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b4340 con 0x557ff83051b0 2024-09-21T06:20:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.230+0000 7fe8a6545640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.232+0000 7fe8a6545640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48008/0 (socket says 172.21.15.31:48008) 2024-09-21T06:20:41.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.232+0000 7fe8a6545640 1 -- 172.21.15.31:0/2943472060 learned_addr learned my addr 172.21.15.31:0/2943472060 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:41.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.234+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff82ec9f0 0x557ff86e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.234+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.237+0000 7fe8a6545640 1 -- 172.21.15.31:0/2943472060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff82ec9f0 msgr2=0x557ff86e2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.237+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff82ec9f0 0x557ff86e2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.237+0000 7fe8a6545640 1 -- 172.21.15.31:0/2943472060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 msgr2=0x557ff86d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.237+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.237+0000 7fe8a6545640 1 -- 172.21.15.31:0/2943472060 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ff86d7990 con 0x557ff83051b0 2024-09-21T06:20:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.239+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d6de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x557ff86d4320 tx=0x7fe89c00bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8348440befea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.239+0000 7fe88bfff640 1 -- 172.21.15.31:0/2943472060 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003ca0 con 0x557ff83051b0 2024-09-21T06:20:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.239+0000 7fe88bfff640 1 -- 172.21.15.31:0/2943472060 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c003e40 con 0x557ff83051b0 2024-09-21T06:20:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.239+0000 7fe88bfff640 1 -- 172.21.15.31:0/2943472060 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c023690 con 0x557ff83051b0 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2943472060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 msgr2=0x557ff86d6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d6de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x557ff86d4320 tx=0x7fe89c00bf20 comp rx=0 tx=0).stop 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2943472060 shutdown_connections 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff830d390 0x557ff86d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff83051b0 0x557ff86d6de0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2943472060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff82ec9f0 0x557ff86e2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.241+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2943472060 >> 172.21.15.31:0/2943472060 conn(0x557ff8322460 msgr2=0x557ff86d66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:41.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.242+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2943472060 shutdown_connections 2024-09-21T06:20:41.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.247+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2943472060 wait complete. 2024-09-21T06:20:41.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.248+0000 7fe8aed1d740 1 -- start start 2024-09-21T06:20:41.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.248+0000 7fe8aed1d740 1 Processor -- start 2024-09-21T06:20:41.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.249+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 0x557ff86f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.249+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff832e070 0x557ff86f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.249+0000 7fe8aed1d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff82c1da0 0x557ff86f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.249+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b4a10 con 0x557ff82c1da0 2024-09-21T06:20:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.250+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b5e80 con 0x557ff83051b0 2024-09-21T06:20:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.250+0000 7fe8aed1d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b50e0 con 0x557ff832e070 2024-09-21T06:20:41.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.250+0000 7fe8aed1d740 10 client.0 _ll_register_callbacks cb 0x7fe890032e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.253+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 0x557ff86f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.253+0000 7fe8a5d44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 0x557ff86f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33080/0 (socket says 172.21.15.31:33080) 2024-09-21T06:20:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.253+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2150838471 learned_addr learned my addr 172.21.15.31:0/2150838471 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:41.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.253+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff82c1da0 0x557ff86f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.254+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2150838471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff832e070 msgr2=0x557ff86f3bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:20:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.254+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff832e070 0x557ff86f3bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.254+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2150838471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff82c1da0 msgr2=0x557ff86f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.254+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff82c1da0 0x557ff86f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.254+0000 7fe8a5d44640 1 -- 172.21.15.31:0/2150838471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ff86f48e0 con 0x557ff83051b0 2024-09-21T06:20:41.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.258+0000 7fe8a5d44640 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 0x557ff86f66d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe890032fd0 tx=0x7fe890005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cbababa5b90c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:41.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.258+0000 7fe889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.263+0000 7fe889ffb640 1 -- 172.21.15.31:0/2150838471 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890035070 con 0x557ff83051b0 2024-09-21T06:20:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.263+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe89c003e40 con 0x557ff83051b0 2024-09-21T06:20:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.263+0000 7fe889ffb640 1 -- 172.21.15.31:0/2150838471 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fe890004190 con 0x557ff83051b0 2024-09-21T06:20:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.263+0000 7fe889ffb640 1 -- 172.21.15.31:0/2150838471 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe890003800 con 0x557ff83051b0 2024-09-21T06:20:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.264+0000 7fe8aed1d740 10 client.27155 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:41.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.264+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557ff875d580 con 0x557ff83051b0 2024-09-21T06:20:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.264+0000 7fe889ffb640 1 -- 172.21.15.31:0/2150838471 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c003e40 con 0x557ff83051b0 2024-09-21T06:20:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe889ffb640 1 -- 172.21.15.31:0/2150838471 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe8900322d0 con 0x557ff83051b0 2024-09-21T06:20:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8aed1d740 10 client.27155 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:41.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8aed1d740 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8aed1d740 20 client.27155 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff875db20 0x557ff875ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8aed1d740 4 client.27155 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:20:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ff86498a0 con 0x557ff875db20 2024-09-21T06:20:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.265+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff875db20 0x557ff875ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.266+0000 7fe889ffb640 10 client.27155 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.267+0000 7fe8a6545640 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff875db20 0x557ff875ff40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbcf2faa0f4b3719 server_cookie=14ad42fb235c246a in_seq=0 out_seq=0 2024-09-21T06:20:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.268+0000 7fe889ffb640 1 -- 172.21.15.31:0/2150838471 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ff86498a0 con 0x557ff875db20 2024-09-21T06:20:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.268+0000 7fe889ffb640 10 client.27155 handle_command_reply: tid=0 2024-09-21T06:20:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 client.27155 shutdown 2024-09-21T06:20:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 msgr2=0x557ff86f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 0x557ff86f66d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe890032fd0 tx=0x7fe890005850 comp rx=0 tx=0).stop 2024-09-21T06:20:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 shutdown_connections 2024-09-21T06:20:41.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ff875db20 0x557ff875ff40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ff82c1da0 0x557ff86f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ff832e070 0x557ff86f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 --2- 172.21.15.31:0/2150838471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ff83051b0 0x557ff86f66d0 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe890032fd0 tx=0x7fe890005850 comp rx=0 tx=0).stop 2024-09-21T06:20:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 >> 172.21.15.31:0/2150838471 conn(0x557ff8322460 msgr2=0x557ff86e2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.269+0000 7fe889ffb640 0 client.27155 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:41.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.270+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 shutdown_connections 2024-09-21T06:20:41.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.270+0000 7fe8aed1d740 1 -- 172.21.15.31:0/2150838471 wait complete. 2024-09-21T06:20:41.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.271+0000 7fe8aed1d740 20 client.27155 trim_cache size 0 max 16384 2024-09-21T06:20:41.273 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.272+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe88c0b67d0 msgr2=0x7fe88c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.272+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe88c0b67d0 0x7fe88c0b8ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe8980e51b0 tx=0x7fe898071e10 comp rx=0 tx=0).stop 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.272+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a0107bf0 msgr2=0x7fe8a01a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.272+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a0107bf0 0x7fe8a01a6320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe89800da10 tx=0x7fe89800ded0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 shutdown_connections 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8a0107bf0 0x7fe8a01a6320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe88c0b67d0 0x7fe88c0b8ce0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8a0103100 0x7fe8a01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 --2- 172.21.15.31:0/1873984488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a01027b0 0x7fe8a01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 >> 172.21.15.31:0/1873984488 conn(0x7fe8a0079730 msgr2=0x7fe8a0105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 shutdown_connections 2024-09-21T06:20:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:41.273+0000 7fe8a87a9640 1 -- 172.21.15.31:0/1873984488 wait complete. 2024-09-21T06:20:41.776 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:41 smithi031 ceph-mon[31237]: pgmap v598: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-09-21T06:20:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:41 smithi110 ceph-mon[34550]: pgmap v598: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-09-21T06:20:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:41 smithi107 ceph-mon[34380]: pgmap v598: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 20 MiB/s wr, 1.76k op/s 2024-09-21T06:20:43.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:20:43.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:20:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:43 smithi031 ceph-mon[31237]: pgmap v599: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 20 MiB/s wr, 1.83k op/s 2024-09-21T06:20:43.647 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:43 smithi110 ceph-mon[34550]: pgmap v599: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 20 MiB/s wr, 1.83k op/s 2024-09-21T06:20:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:43 smithi107 ceph-mon[34380]: pgmap v599: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 20 MiB/s wr, 1.83k op/s 2024-09-21T06:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 -- 172.21.15.31:0/416587398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8111cb0 msgr2=0x7fe9a81120c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/416587398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8111cb0 0x7fe9a81120c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009900 tx=0x7fe99c02a410 comp rx=0 tx=0).stop 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 -- 172.21.15.31:0/416587398 shutdown_connections 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/416587398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a8112600 0x7fe9a8118e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/416587398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8111cb0 0x7fe9a81120c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/416587398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9a810c370 0x7fe9a810c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 -- 172.21.15.31:0/416587398 >> 172.21.15.31:0/416587398 conn(0x7fe9a8071a80 msgr2=0x7fe9a810a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 -- 172.21.15.31:0/416587398 shutdown_connections 2024-09-21T06:20:44.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.220+0000 7fe9ae32d640 1 -- 172.21.15.31:0/416587398 wait complete. 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.221+0000 7fe9ae32d640 1 Processor -- start 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.221+0000 7fe9ae32d640 1 -- start start 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ae32d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 0x7fe9a81a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ae32d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9a8111cb0 0x7fe9a81a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ae32d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8112600 0x7fe9a81a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ae32d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a811c2c0 con 0x7fe9a8112600 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ae32d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a811c140 con 0x7fe9a810c370 2024-09-21T06:20:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ae32d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a811c440 con 0x7fe9a8111cb0 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 0x7fe9a81a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 0x7fe9a81a2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33098/0 (socket says 172.21.15.31:33098) 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 -- 172.21.15.31:0/569751792 learned_addr learned my addr 172.21.15.31:0/569751792 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8112600 0x7fe9a81a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9a8111cb0 0x7fe9a81a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 -- 172.21.15.31:0/569751792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9a8111cb0 msgr2=0x7fe9a81a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9a8111cb0 0x7fe9a81a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 -- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8112600 msgr2=0x7fe9a81a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8112600 0x7fe9a81a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.222+0000 7fe9a7fff640 1 -- 172.21.15.31:0/569751792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a81a8ce0 con 0x7fe9a810c370 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.223+0000 7fe9a7fff640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 0x7fe9a81a2d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe99800d950 tx=0x7fe99800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.223+0000 7fe9a57fa640 1 -- 172.21.15.31:0/569751792 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998017070 con 0x7fe9a810c370 2024-09-21T06:20:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.223+0000 7fe9a57fa640 1 -- 172.21.15.31:0/569751792 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe998009c30 con 0x7fe9a810c370 2024-09-21T06:20:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.223+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81a8fd0 con 0x7fe9a810c370 2024-09-21T06:20:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.223+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81af730 con 0x7fe9a810c370 2024-09-21T06:20:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.223+0000 7fe9a57fa640 1 -- 172.21.15.31:0/569751792 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99800f620 con 0x7fe9a810c370 2024-09-21T06:20:44.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.228+0000 7fe9b48a1740 1 Processor -- start 2024-09-21T06:20:44.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.228+0000 7fe9b48a1740 1 -- start start 2024-09-21T06:20:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d706350 0x55785daad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785daad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785daadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55785da23e60 con 0x55785d6e5670 2024-09-21T06:20:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55785da13290 con 0x55785d6dd490 2024-09-21T06:20:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55785da170a0 con 0x55785d706350 2024-09-21T06:20:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785daadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785daadeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43358/0 (socket says 172.21.15.31:43358) 2024-09-21T06:20:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9a77fe640 1 -- 172.21.15.31:0/3780755973 learned_addr learned my addr 172.21.15.31:0/3780755973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d706350 0x55785daad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9a57fa640 1 -- 172.21.15.31:0/569751792 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe998010460 con 0x7fe9a810c370 2024-09-21T06:20:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3780755973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 msgr2=0x55785daad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785daad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3780755973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 msgr2=0x55785daadeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785daadeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.229+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3780755973 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785d771930 con 0x55785d706350 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.230+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d706350 0x55785daad430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009f00 tx=0x7fe9a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231851ef16f5a362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.230+0000 7fe98e7fc640 1 -- 172.21.15.31:0/3780755973 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0004d20 con 0x55785d706350 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.230+0000 7fe98e7fc640 1 -- 172.21.15.31:0/3780755973 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a0004050 con 0x55785d706350 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.230+0000 7fe98e7fc640 1 -- 172.21.15.31:0/3780755973 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0005220 con 0x55785d706350 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9a57fa640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe9900b6700 0x7fe9900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9a7fff640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe9900b6700 0x7fe9900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9a57fa640 1 -- 172.21.15.31:0/569751792 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fe9980e7210 con 0x7fe9a810c370 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3780755973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d706350 msgr2=0x55785daad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d706350 0x55785daad430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009f00 tx=0x7fe9a0011810 comp rx=0 tx=0).stop 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9a7fff640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe9900b6700 0x7fe9900b8c10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55785daac0b0 tx=0x7fe998078040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3780755973 shutdown_connections 2024-09-21T06:20:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785daadeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785daad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3780755973 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d706350 0x55785daad430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3780755973 >> 172.21.15.31:0/3780755973 conn(0x55785d6fa740 msgr2=0x55785daab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.231+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3780755973 shutdown_connections 2024-09-21T06:20:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.232+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3780755973 wait complete. 2024-09-21T06:20:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.232+0000 7fe9b48a1740 1 -- start start 2024-09-21T06:20:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.232+0000 7fe9b48a1740 1 Processor -- start 2024-09-21T06:20:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785dabd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6e5670 0x55785dabd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d706350 0x55785dac9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55785d874890 con 0x55785d6dd490 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55785d9add40 con 0x55785d706350 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55785dabff70 con 0x55785d6e5670 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9b48a1740 10 client.0 _ll_register_callbacks cb 0x55785da11c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785dabd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785dabd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43368/0 (socket says 172.21.15.31:43368) 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3149212044 learned_addr learned my addr 172.21.15.31:0/3149212044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6e5670 0x55785dabd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d706350 0x55785dac9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3149212044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6e5670 msgr2=0x55785dabd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6e5670 0x55785dabd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d706350 msgr2=0x55785dac9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.233+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d706350 0x55785dac9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.234+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/3149212044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785dac9840 con 0x55785d6dd490 2024-09-21T06:20:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.234+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785dabd3a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009730 tx=0x7fe9a0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7085df6a08078e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.234+0000 7fe9877fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:20:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.235+0000 7fe9877fe640 1 -- 172.21.15.31:0/3149212044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0004410 con 0x55785d6dd490 2024-09-21T06:20:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.235+0000 7fe9877fe640 1 -- 172.21.15.31:0/3149212044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fe9a00045b0 con 0x55785d6dd490 2024-09-21T06:20:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.235+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a0004050 con 0x55785d6dd490 2024-09-21T06:20:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9877fe640 1 -- 172.21.15.31:0/3149212044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe9a0005040 con 0x55785d6dd490 2024-09-21T06:20:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 10 client.17379 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55785dacc0e0 con 0x55785d6dd490 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9877fe640 1 -- 172.21.15.31:0/3149212044 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a0004050 con 0x55785d6dd490 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9877fe640 1 -- 172.21.15.31:0/3149212044 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe9a00287d0 con 0x55785d6dd490 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 10 client.17379 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 20 client.17379 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d6c4cd0 0x55785dacf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 4 client.17379 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.236+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55785da25db0 con 0x55785d6c4cd0 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.237+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d6c4cd0 0x55785dacf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.237+0000 7fe9877fe640 10 client.17379 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:44.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.237+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d6c4cd0 0x55785dacf4f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da40b74034a18c6 server_cookie=f146df0682a09501 in_seq=0 out_seq=0 2024-09-21T06:20:44.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9877fe640 1 -- 172.21.15.31:0/3149212044 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55785da25db0 con 0x55785d6c4cd0 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9877fe640 10 client.17379 handle_command_reply: tid=0 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 client.17379 shutdown 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 msgr2=0x55785dabd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785dabd3a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009730 tx=0x7fe9a0003f90 comp rx=0 tx=0).stop 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 shutdown_connections 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d6c4cd0 0x55785dacf4f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9877fe640 0 client.17379 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d706350 0x55785dac9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6e5670 0x55785dabd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.238+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/3149212044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785dabd3a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.239+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 >> 172.21.15.31:0/3149212044 conn(0x55785d6fa740 msgr2=0x55785daaee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.239+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 shutdown_connections 2024-09-21T06:20:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.239+0000 7fe9b48a1740 1 -- 172.21.15.31:0/3149212044 wait complete. 2024-09-21T06:20:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.239+0000 7fe9b48a1740 20 client.17379 trim_cache size 0 max 16384 2024-09-21T06:20:44.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.264+0000 7fe9b48a1740 1 Processor -- start 2024-09-21T06:20:44.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.264+0000 7fe9b48a1740 1 -- start start 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.264+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dabca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.264+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785daab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.264+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6e5670 0x55785daba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b3f00 con 0x55785d6dd490 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b6760 con 0x55785d6e5670 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b4550 con 0x55785d6c4cd0 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dabca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dabca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48064/0 (socket says 172.21.15.31:48064) 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/2410027653 learned_addr learned my addr 172.21.15.31:0/2410027653 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6e5670 0x55785daba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785daab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/2410027653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6e5670 msgr2=0x55785daba390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6e5670 0x55785daba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/2410027653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 msgr2=0x55785daab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785daab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/2410027653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55785dab8a30 con 0x55785d6c4cd0 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.265+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dabca30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009f00 tx=0x7fe9a00096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1f31139524f18d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.266+0000 7fe98dffb640 1 -- 172.21.15.31:0/2410027653 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a003a070 con 0x55785d6c4cd0 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.266+0000 7fe98dffb640 1 -- 172.21.15.31:0/2410027653 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a003b040 con 0x55785d6c4cd0 2024-09-21T06:20:44.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.266+0000 7fe98dffb640 1 -- 172.21.15.31:0/2410027653 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0032660 con 0x55785d6c4cd0 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 -- 172.21.15.31:0/2410027653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 msgr2=0x55785dabca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dabca30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009f00 tx=0x7fe9a00096c0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 -- 172.21.15.31:0/2410027653 shutdown_connections 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6e5670 0x55785daba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6dd490 0x55785daab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/2410027653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dabca30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 -- 172.21.15.31:0/2410027653 >> 172.21.15.31:0/2410027653 conn(0x55785d6fa740 msgr2=0x55785daaac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 -- 172.21.15.31:0/2410027653 shutdown_connections 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 -- 172.21.15.31:0/2410027653 wait complete. 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 -- start start 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.267+0000 7fe9b48a1740 1 Processor -- start 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dab7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785dac5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785dac5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b4c20 con 0x55785d6e5670 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b6090 con 0x55785d6dd490 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81b52f0 con 0x55785d6c4cd0 2024-09-21T06:20:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9b48a1740 10 client.0 _ll_register_callbacks cb 0x7fe99c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9ac8ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785dac5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9ac8ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785dac5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43388/0 (socket says 172.21.15.31:43388) 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9ac8ca640 1 -- 172.21.15.31:0/4245980555 learned_addr learned my addr 172.21.15.31:0/4245980555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785dac5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dab7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9a77fe640 1 -- 172.21.15.31:0/4245980555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 msgr2=0x55785dab7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dab7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9a77fe640 1 -- 172.21.15.31:0/4245980555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 msgr2=0x55785dac5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785dac5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.268+0000 7fe9a77fe640 1 -- 172.21.15.31:0/4245980555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55785dac5fb0 con 0x55785d6dd490 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9a77fe640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785dac5260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe99c005ab0 tx=0x7fe99c004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f145f8d28cad200 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9877fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:20:44.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9877fe640 1 -- 172.21.15.31:0/4245980555 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c035040 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9877fe640 1 -- 172.21.15.31:0/4245980555 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7fe99c005510 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a003b040 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9877fe640 1 -- 172.21.15.31:0/4245980555 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe99c004750 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9b48a1740 10 client.27170 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.269+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55785db3e600 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9877fe640 1 -- 172.21.15.31:0/4245980555 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99c004da0 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9877fe640 1 -- 172.21.15.31:0/4245980555 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fe99c0314a0 con 0x55785d6dd490 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9b48a1740 10 client.27170 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9b48a1740 10 client.27170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9b48a1740 20 client.27170 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d706350 0x55785db40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:44.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9b48a1740 4 client.27170 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:20:44.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.270+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe9a003b040 con 0x55785d706350 2024-09-21T06:20:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.271+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d706350 0x55785db40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.271+0000 7fe9ac8ca640 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d706350 0x55785db40da0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7257b3076df85433 server_cookie=ad28198d92b0dfb in_seq=0 out_seq=0 2024-09-21T06:20:44.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.271+0000 7fe9877fe640 10 client.27170 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.273+0000 7fe9877fe640 1 -- 172.21.15.31:0/4245980555 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe9a003b040 con 0x55785d706350 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.273+0000 7fe9877fe640 10 client.27170 handle_command_reply: tid=0 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.273+0000 7fe9b48a1740 1 client.27170 shutdown 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.274+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 msgr2=0x55785dac5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.274+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785dac5260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe99c005ab0 tx=0x7fe99c004560 comp rx=0 tx=0).stop 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 shutdown_connections 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55785d706350 0x55785db40da0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9877fe640 0 client.27170 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55785d6e5670 0x55785dac5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55785d6dd490 0x55785dac5260 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 --2- 172.21.15.31:0/4245980555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55785d6c4cd0 0x55785dab7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 >> 172.21.15.31:0/4245980555 conn(0x55785d6fa740 msgr2=0x55785daa9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 shutdown_connections 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 1 -- 172.21.15.31:0/4245980555 wait complete. 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.275+0000 7fe9b48a1740 20 client.27170 trim_cache size 0 max 16384 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "e31d9f72-c104-4147-9ac7-1a5c9c0d8846", 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:20:44.279 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe9900b6700 msgr2=0x7fe9900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe9900b6700 0x7fe9900b8c10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x55785daac0b0 tx=0x7fe998078040 comp rx=0 tx=0).stop 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 msgr2=0x7fe9a81a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 0x7fe9a81a2d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe99800d950 tx=0x7fe99800de10 comp rx=0 tx=0).stop 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 shutdown_connections 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe9900b6700 0x7fe9900b8c10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a8112600 0x7fe9a81a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9a8111cb0 0x7fe9a81a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 --2- 172.21.15.31:0/569751792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a810c370 0x7fe9a81a2d70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 >> 172.21.15.31:0/569751792 conn(0x7fe9a8071a80 msgr2=0x7fe9a80722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 shutdown_connections 2024-09-21T06:20:44.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:44.276+0000 7fe9ae32d640 1 -- 172.21.15.31:0/569751792 wait complete. 2024-09-21T06:20:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 545.0 s, 1384.4 tps, lat 72.663 ms stddev 77.153 2024-09-21T06:20:45.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:20:45.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:45 smithi031 ceph-mon[31237]: pgmap v600: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 5.5 MiB/s wr, 648 op/s 2024-09-21T06:20:45.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:45 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:20:45.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:45 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:45.754 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:45 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:20:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:45 smithi110 ceph-mon[34550]: pgmap v600: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 5.5 MiB/s wr, 648 op/s 2024-09-21T06:20:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:45 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:20:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:45 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:45 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:20:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:45 smithi107 ceph-mon[34380]: pgmap v600: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 5.5 MiB/s wr, 648 op/s 2024-09-21T06:20:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:45 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:20:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:45 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:45 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:20:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:20:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 -- 172.21.15.31:0/854960035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 msgr2=0x7f90b810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 --2- 172.21.15.31:0/854960035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b810adf0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009ca0 tx=0x7f90b002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 -- 172.21.15.31:0/854960035 shutdown_connections 2024-09-21T06:20:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 --2- 172.21.15.31:0/854960035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b811cc30 0x7f90b811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 --2- 172.21.15.31:0/854960035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b810adf0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 --2- 172.21.15.31:0/854960035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b8111f40 0x7f90b810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 -- 172.21.15.31:0/854960035 >> 172.21.15.31:0/854960035 conn(0x7f90b8070ec0 msgr2=0x7f90b80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.713+0000 7f90bcef7640 1 -- 172.21.15.31:0/854960035 shutdown_connections 2024-09-21T06:20:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.714+0000 7f90bcef7640 1 -- 172.21.15.31:0/854960035 wait complete. 2024-09-21T06:20:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 Processor -- start 2024-09-21T06:20:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 -- start start 2024-09-21T06:20:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b81a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b8111f40 0x7f90b81a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b811cc30 0x7f90b81aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8122940 con 0x7f90b810a9e0 2024-09-21T06:20:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81227c0 con 0x7f90b811cc30 2024-09-21T06:20:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90bcef7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8122ac0 con 0x7f90b8111f40 2024-09-21T06:20:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.715+0000 7f90b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b81a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b81a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43400/0 (socket says 172.21.15.31:43400) 2024-09-21T06:20:46.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 -- 172.21.15.31:0/2526342623 learned_addr learned my addr 172.21.15.31:0/2526342623 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b811cc30 0x7f90b81aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b8111f40 0x7f90b81a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 -- 172.21.15.31:0/2526342623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b8111f40 msgr2=0x7f90b81a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b8111f40 0x7f90b81a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 -- 172.21.15.31:0/2526342623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b811cc30 msgr2=0x7f90b81aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b811cc30 0x7f90b81aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.716+0000 7f90b659c640 1 -- 172.21.15.31:0/2526342623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b81af250 con 0x7f90b810a9e0 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.717+0000 7f90b659c640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b81a9310 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c890 tx=0x7f90a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.717+0000 7f90a77fe640 1 -- 172.21.15.31:0/2526342623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8004220 con 0x7f90b810a9e0 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.717+0000 7f90a77fe640 1 -- 172.21.15.31:0/2526342623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a80043c0 con 0x7f90b810a9e0 2024-09-21T06:20:46.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.717+0000 7f90a77fe640 1 -- 172.21.15.31:0/2526342623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8004a00 con 0x7f90b810a9e0 2024-09-21T06:20:46.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.717+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81af540 con 0x7f90b810a9e0 2024-09-21T06:20:46.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.717+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b8112380 con 0x7f90b810a9e0 2024-09-21T06:20:46.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.721+0000 7f90c346b740 1 Processor -- start 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.724+0000 7f90c346b740 1 -- start start 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90a77fe640 1 -- 172.21.15.31:0/2526342623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f90a80050b0 con 0x7f90b810a9e0 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90c346b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b7e070 0x55c923f25270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90c346b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b551b0 0x55c923f257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90c346b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b5d390 0x55c923f25cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90c346b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c923e9be50 con 0x55c923b7e070 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90c346b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c923e8b280 con 0x55c923b5d390 2024-09-21T06:20:46.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.725+0000 7f90c346b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c923e8f090 con 0x55c923b551b0 2024-09-21T06:20:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.727+0000 7f90b5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b5d390 0x55c923f25cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.727+0000 7f90b5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b5d390 0x55c923f25cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33160/0 (socket says 172.21.15.31:33160) 2024-09-21T06:20:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.727+0000 7f90b5d9b640 1 -- 172.21.15.31:0/2981180677 learned_addr learned my addr 172.21.15.31:0/2981180677 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:46.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.729+0000 7f90a77fe640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90a00b6880 0x7f90a00b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.730+0000 7f90b659c640 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b551b0 0x55c923f257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.730+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b7e070 0x55c923f25270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.731+0000 7f90b6d9d640 1 -- 172.21.15.31:0/2981180677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b551b0 msgr2=0x55c923f257b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.731+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b551b0 0x55c923f257b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.731+0000 7f90b6d9d640 1 -- 172.21.15.31:0/2981180677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b5d390 msgr2=0x55c923f25cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.731+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b5d390 0x55c923f25cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.731+0000 7f90b6d9d640 1 -- 172.21.15.31:0/2981180677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c923be9650 con 0x55c923b7e070 2024-09-21T06:20:46.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.731+0000 7f90a77fe640 1 -- 172.21.15.31:0/2526342623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f90a80e76b0 con 0x7f90b810a9e0 2024-09-21T06:20:46.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.732+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90a00b6880 0x7f90a00b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.732+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b7e070 0x55c923f25270 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b6c0 tx=0x7f90ac00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ba6dc78e1326e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:46.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.732+0000 7f90a4ff9640 1 -- 172.21.15.31:0/2981180677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00de60 con 0x55c923b7e070 2024-09-21T06:20:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.732+0000 7f90a4ff9640 1 -- 172.21.15.31:0/2981180677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac019070 con 0x55c923b7e070 2024-09-21T06:20:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.733+0000 7f90a4ff9640 1 -- 172.21.15.31:0/2981180677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac006030 con 0x55c923b7e070 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90a00b6880 0x7f90a00b8d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90ac01e000 tx=0x7f90ac021000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 -- 172.21.15.31:0/2981180677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b7e070 msgr2=0x55c923f25270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b7e070 0x55c923f25270 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b6c0 tx=0x7f90ac00bb80 comp rx=0 tx=0).stop 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 -- 172.21.15.31:0/2981180677 shutdown_connections 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b5d390 0x55c923f25cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b551b0 0x55c923f257b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 --2- 172.21.15.31:0/2981180677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b7e070 0x55c923f25270 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 -- 172.21.15.31:0/2981180677 >> 172.21.15.31:0/2981180677 conn(0x55c923b72460 msgr2=0x55c923f23160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.734+0000 7f90c346b740 1 -- 172.21.15.31:0/2981180677 shutdown_connections 2024-09-21T06:20:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.735+0000 7f90c346b740 1 -- 172.21.15.31:0/2981180677 wait complete. 2024-09-21T06:20:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.735+0000 7f90c346b740 1 -- start start 2024-09-21T06:20:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.735+0000 7f90c346b740 1 Processor -- start 2024-09-21T06:20:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b551b0 0x55c923f308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b5d390 0x55c923f30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 0x55c923f31320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c923cec700 con 0x55c923b5d390 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c923e26110 con 0x55c923b551b0 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c923f33aa0 con 0x55c923b7e070 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90c346b740 10 client.0 _ll_register_callbacks cb 0x55c923e3f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 0x55c923f31320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 0x55c923f31320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48100/0 (socket says 172.21.15.31:48100) 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b6d9d640 1 -- 172.21.15.31:0/235874634 learned_addr learned my addr 172.21.15.31:0/235874634 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b659c640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b551b0 0x55c923f308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b6d9d640 1 -- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b551b0 msgr2=0x55c923f308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b551b0 0x55c923f308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.736+0000 7f90b6d9d640 1 -- 172.21.15.31:0/235874634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b5d390 msgr2=0x55c923f30de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.737+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b5d390 0x55c923f30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.737+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b5d390 0x55c923f30de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.737+0000 7f90b6d9d640 1 -- 172.21.15.31:0/235874634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c923e53c40 con 0x55c923b7e070 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.737+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b5d390 0x55c923f30de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.737+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 0x55c923f31320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90ac015a80 tx=0x7f90ac00da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f2e95e75a2f3a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.737+0000 7f909dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.738+0000 7f909dffb640 1 -- 172.21.15.31:0/235874634 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac02d020 con 0x55c923b7e070 2024-09-21T06:20:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.738+0000 7f909dffb640 1 -- 172.21.15.31:0/235874634 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f90ac005bc0 con 0x55c923b7e070 2024-09-21T06:20:46.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.739+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90ac019070 con 0x55c923b7e070 2024-09-21T06:20:46.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.739+0000 7f909dffb640 1 -- 172.21.15.31:0/235874634 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f90ac025040 con 0x55c923b7e070 2024-09-21T06:20:46.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.739+0000 7f90c346b740 10 client.27208 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:46.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.739+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c923f3f6d0 con 0x55c923b7e070 2024-09-21T06:20:46.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.740+0000 7f909dffb640 1 -- 172.21.15.31:0/235874634 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac019070 con 0x55c923b7e070 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.740+0000 7f909dffb640 1 -- 172.21.15.31:0/235874634 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f90ac028070 con 0x55c923b7e070 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90c346b740 10 client.27208 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90c346b740 10 client.27208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90c346b740 20 client.27208 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90c346b740 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b3c9f0 0x55c923f42a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90c346b740 4 client.27208 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c923e9dda0 con 0x55c923b3c9f0 2024-09-21T06:20:46.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90b659c640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b3c9f0 0x55c923f42a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f90b659c640 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b3c9f0 0x55c923f42a90 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1bda91dee79e6b server_cookie=586ae5e04289b5b0 in_seq=0 out_seq=0 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.741+0000 7f909dffb640 10 client.27208 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f909dffb640 1 -- 172.21.15.31:0/235874634 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c923e9dda0 con 0x55c923b3c9f0 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f909dffb640 10 client.27208 handle_command_reply: tid=0 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f90c346b740 1 client.27208 shutdown 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 msgr2=0x55c923f31320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f90c346b740 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 0x55c923f31320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f90ac015a80 tx=0x7f90ac00da70 comp rx=0 tx=0).stop 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 shutdown_connections 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f90c346b740 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b3c9f0 0x55c923f42a90 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f90c346b740 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b7e070 0x55c923f31320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.742+0000 7f909dffb640 0 client.27208 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:46.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.743+0000 7f90c346b740 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b5d390 0x55c923f30de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.743+0000 7f90c346b740 1 --2- 172.21.15.31:0/235874634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b551b0 0x55c923f308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.743+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 >> 172.21.15.31:0/235874634 conn(0x55c923b72460 msgr2=0x55c923f28450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:46.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.743+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 shutdown_connections 2024-09-21T06:20:46.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.743+0000 7f90c346b740 1 -- 172.21.15.31:0/235874634 wait complete. 2024-09-21T06:20:46.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.743+0000 7f90c346b740 20 client.27208 trim_cache size 0 max 16384 2024-09-21T06:20:46.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.768+0000 7f90c346b740 1 Processor -- start 2024-09-21T06:20:46.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 -- start start 2024-09-21T06:20:46.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f3c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f22710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f22c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f3c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f3c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43420/0 (socket says 172.21.15.31:43420) 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90b659c640 1 -- 172.21.15.31:0/2765043274 learned_addr learned my addr 172.21.15.31:0/2765043274 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81c83e0 con 0x55c923b551b0 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81c7640 con 0x55c923b7e070 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8119320 con 0x55c923b5d390 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.769+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f22710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f22c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b659c640 1 -- 172.21.15.31:0/2765043274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 msgr2=0x55c923f22710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b659c640 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f22710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b659c640 1 -- 172.21.15.31:0/2765043274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 msgr2=0x55c923f22c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b659c640 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f22c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b659c640 1 -- 172.21.15.31:0/2765043274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c923f33580 con 0x55c923b551b0 2024-09-21T06:20:46.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f90b659c640 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f3c730 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f90a80ad230 tx=0x7f90a80ed530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914755487f452105 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:46.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f909ffff640 1 -- 172.21.15.31:0/2765043274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80ed020 con 0x55c923b551b0 2024-09-21T06:20:46.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f909ffff640 1 -- 172.21.15.31:0/2765043274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a80ad040 con 0x55c923b551b0 2024-09-21T06:20:46.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.770+0000 7f909ffff640 1 -- 172.21.15.31:0/2765043274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80b6ac0 con 0x55c923b551b0 2024-09-21T06:20:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 msgr2=0x55c923f3c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f3c730 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f90a80ad230 tx=0x7f90a80ed530 comp rx=0 tx=0).stop 2024-09-21T06:20:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 shutdown_connections 2024-09-21T06:20:46.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f22c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f22710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 --2- 172.21.15.31:0/2765043274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f3c730 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 >> 172.21.15.31:0/2765043274 conn(0x55c923b72460 msgr2=0x55c923f32660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 shutdown_connections 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.771+0000 7f90c346b740 1 -- 172.21.15.31:0/2765043274 wait complete. 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.772+0000 7f90c346b740 1 -- start start 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.772+0000 7f90c346b740 1 Processor -- start 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.772+0000 7f90c346b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f45fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.772+0000 7f90c346b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.772+0000 7f90c346b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.772+0000 7f90b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f45fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90c346b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81199b0 con 0x55c923b551b0 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90c346b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81c6f70 con 0x55c923b7e070 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f45fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43424/0 (socket says 172.21.15.31:43424) 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90b811a080 con 0x55c923b5d390 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b659c640 1 -- 172.21.15.31:0/1336424177 learned_addr learned my addr 172.21.15.31:0/1336424177 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90c346b740 10 client.0 _ll_register_callbacks cb 0x7f90b0009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 -- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 msgr2=0x55c923f439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 -- 172.21.15.31:0/1336424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 msgr2=0x55c923f45fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f45fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 -- 172.21.15.31:0/1336424177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c923f1e580 con 0x55c923b5d390 2024-09-21T06:20:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b6d9d640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f90b5d9b640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f43470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009a60 tx=0x7f90b0031930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c991d62264beecb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.773+0000 7f909dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f909dffb640 1 -- 172.21.15.31:0/1336424177 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b004b060 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f909dffb640 1 -- 172.21.15.31:0/1336424177 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f90b0003d80 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a80ad040 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f909dffb640 1 -- 172.21.15.31:0/1336424177 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f90b0004040 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 10 client.27214 fetch_fsmap learned FSMap version 19 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c923f279c0 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f909dffb640 1 -- 172.21.15.31:0/1336424177 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a80ad040 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f909dffb640 1 -- 172.21.15.31:0/1336424177 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f90b002fb10 con 0x55c923b5d390 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 10 client.27214 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 10 client.27214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 20 client.27214 populate_metadata read hostname 'smithi031' 2024-09-21T06:20:46.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b11da0 0x55c923faf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:20:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 4 client.27214 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:20:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.774+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c923e99850 con 0x55c923b11da0 2024-09-21T06:20:46.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f90b659c640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b11da0 0x55c923faf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f909dffb640 10 client.27214 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f90b659c640 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b11da0 0x55c923faf2a0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df4bdae05d09754 server_cookie=4cc997a05ade4b42 in_seq=0 out_seq=0 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f909dffb640 1 -- 172.21.15.31:0/1336424177 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7108 (crc 0 0 0) 0x55c923e99850 con 0x55c923b11da0 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f909dffb640 10 client.27214 handle_command_reply: tid=0 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f90c346b740 1 client.27214 shutdown 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 msgr2=0x55c923f43470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.775+0000 7f90c346b740 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f43470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009a60 tx=0x7f90b0031930 comp rx=0 tx=0).stop 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 shutdown_connections 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c923b11da0 0x55c923faf2a0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c923b7e070 0x55c923f439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c923b5d390 0x55c923f43470 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 --2- 172.21.15.31:0/1336424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c923b551b0 0x55c923f45fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 >> 172.21.15.31:0/1336424177 conn(0x55c923b72460 msgr2=0x55c923f26190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f909dffb640 0 client.27214 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 shutdown_connections 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 1 -- 172.21.15.31:0/1336424177 wait complete. 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.776+0000 7f90c346b740 20 client.27214 trim_cache size 0 max 16384 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (2 inodes in the stack)", 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:20:46.783 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e31d9f72-c104-4147-9ac7-1a5c9c0d8846", 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.784 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.785 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:20:46.786 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.788 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.790 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.791 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.792 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:20:46.793 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.795 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:20:46.796 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:20:46.798 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90a00b6880 msgr2=0x7f90a00b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90a00b6880 0x7f90a00b8d90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90ac01e000 tx=0x7f90ac021000 comp rx=0 tx=0).stop 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 msgr2=0x7f90b81a9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b81a9310 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c890 tx=0x7f90a800cd50 comp rx=0 tx=0).stop 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 shutdown_connections 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b811cc30 0x7f90b81aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b8111f40 0x7f90b81a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90a00b6880 0x7f90a00b8d90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 --2- 172.21.15.31:0/2526342623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90b810a9e0 0x7f90b81a9310 secure :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c890 tx=0x7f90a800cd50 comp rx=0 tx=0).stop 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.778+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 >> 172.21.15.31:0/2526342623 conn(0x7f90b8070ec0 msgr2=0x7f90b811e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:46.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.779+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 shutdown_connections 2024-09-21T06:20:46.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:20:46.779+0000 7f90bcef7640 1 -- 172.21.15.31:0/2526342623 wait complete. 2024-09-21T06:20:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e4eff5640 1 -- 172.21.15.31:0/901591077 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636188cf0f0 con 0x5636185b71b0 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e4eff5640 10 client.27166 handle_command_reply: tid=0 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 client.27166 shutdown 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 msgr2=0x56361897a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361897a640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005b40 tx=0x7f2e64009d70 comp rx=0 tx=0).stop 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 shutdown_connections 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5636185b71b0 0x5636189ef270 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5636185964d0 0x5636189e51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56361858e2f0 0x56361897abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e4eff5640 0 client.27166 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 --2- 172.21.15.31:0/901591077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563618575d80 0x56361897a640 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.072+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 >> 172.21.15.31:0/901591077 conn(0x5636185ab300 msgr2=0x56361896f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.073+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 shutdown_connections 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.073+0000 7f2e77fbb740 1 -- 172.21.15.31:0/901591077 wait complete. 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.073+0000 7f2e77fbb740 20 client.27166 trim_cache size 0 max 16384 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:20:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e540b6700 msgr2=0x7f2e540b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e540b6700 0x7f2e540b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c005940 tx=0x7f2e5c0058b0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 msgr2=0x7f2e6c1a31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c1a31c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000d740 tx=0x7f2e6000dc00 comp rx=0 tx=0).stop 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 shutdown_connections 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e6c1165a0 0x7f2e6c1a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e6c10aef0 0x7f2e6c1a31c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2e540b6700 0x7f2e540b8c10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 --2- 172.21.15.31:0/913859265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e6c10a5a0 0x7f2e6c1a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 >> 172.21.15.31:0/913859265 conn(0x7f2e6c070dd0 msgr2=0x7f2e6c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 shutdown_connections 2024-09-21T06:20:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:20:47.074+0000 7f2e7104d640 1 -- 172.21.15.31:0/913859265 wait complete. 2024-09-21T06:20:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:47 smithi031 ceph-mon[31237]: pgmap v601: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.5 MiB/s wr, 814 op/s 2024-09-21T06:20:47.699 INFO:tasks.cephfs.filesystem:scrub status for tag:e31d9f72-c104-4147-9ac7-1a5c9c0d8846 - {'path': '/', 'tag': 'e31d9f72-c104-4147-9ac7-1a5c9c0d8846', 'options': 'recursive,force'} 2024-09-21T06:20:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:47 smithi110 ceph-mon[34550]: pgmap v601: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.5 MiB/s wr, 814 op/s 2024-09-21T06:20:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:47 smithi107 ceph-mon[34380]: pgmap v601: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.5 MiB/s wr, 814 op/s 2024-09-21T06:20:48.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:48 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:48.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:48 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:48.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:48 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:20:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:49 smithi110 ceph-mon[34550]: pgmap v602: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.9 MiB/s wr, 738 op/s 2024-09-21T06:20:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:49 smithi107 ceph-mon[34380]: pgmap v602: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.9 MiB/s wr, 738 op/s 2024-09-21T06:20:49.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:49 smithi031 ceph-mon[31237]: pgmap v602: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.9 MiB/s wr, 738 op/s 2024-09-21T06:20:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 550.0 s, 1478.0 tps, lat 67.446 ms stddev 69.576 2024-09-21T06:20:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:51 smithi110 ceph-mon[34550]: pgmap v603: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.1 MiB/s wr, 658 op/s 2024-09-21T06:20:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:51 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:20:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:51 smithi107 ceph-mon[34380]: pgmap v603: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.1 MiB/s wr, 658 op/s 2024-09-21T06:20:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:51 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:20:51.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:51 smithi031 ceph-mon[31237]: pgmap v603: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 5.1 MiB/s wr, 658 op/s 2024-09-21T06:20:51.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:51 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:20:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:52 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:20:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:52 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:20:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:52 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:20:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:52 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:20:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:52 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:20:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:52 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:20:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:52 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:20:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:52 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:20:52.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:52 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:20:52.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:52 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:20:52.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:52 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:20:52.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:52 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:20:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:53 smithi110 ceph-mon[34550]: pgmap v604: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 7.4 MiB/s wr, 905 op/s 2024-09-21T06:20:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:53 smithi107 ceph-mon[34380]: pgmap v604: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 7.4 MiB/s wr, 905 op/s 2024-09-21T06:20:53.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:53 smithi031 ceph-mon[31237]: pgmap v604: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 7.4 MiB/s wr, 905 op/s 2024-09-21T06:20:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 555.0 s, 1876.8 tps, lat 53.469 ms stddev 58.740 2024-09-21T06:20:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:55 smithi110 ceph-mon[34550]: pgmap v605: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 5.1 MiB/s wr, 647 op/s 2024-09-21T06:20:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:55 smithi107 ceph-mon[34380]: pgmap v605: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 5.1 MiB/s wr, 647 op/s 2024-09-21T06:20:55.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:55 smithi031 ceph-mon[31237]: pgmap v605: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 5.1 MiB/s wr, 647 op/s 2024-09-21T06:20:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:57 smithi107 ceph-mon[34380]: pgmap v606: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 6.7 MiB/s wr, 817 op/s 2024-09-21T06:20:57.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:57 smithi031 ceph-mon[31237]: pgmap v606: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 6.7 MiB/s wr, 817 op/s 2024-09-21T06:20:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:57 smithi110 ceph-mon[34550]: pgmap v606: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 6.7 MiB/s wr, 817 op/s 2024-09-21T06:20:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:20:59 smithi107 ceph-mon[34380]: pgmap v607: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.6 MiB/s wr, 733 op/s 2024-09-21T06:20:59.835 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:20:59 smithi031 ceph-mon[31237]: pgmap v607: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.6 MiB/s wr, 733 op/s 2024-09-21T06:21:00.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:20:59 smithi110 ceph-mon[34550]: pgmap v607: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.6 MiB/s wr, 733 op/s 2024-09-21T06:21:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 560.0 s, 1722.4 tps, lat 57.631 ms stddev 65.968 2024-09-21T06:21:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.246+0000 7f1e43f2b640 1 -- 172.21.15.31:0/1719911706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c108c50 msgr2=0x7f1e3c10f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.246+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/1719911706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c108c50 0x7f1e3c10f480 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1e3801a910 tx=0x7f1e3803b430 comp rx=0 tx=0).stop 2024-09-21T06:21:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 -- 172.21.15.31:0/1719911706 shutdown_connections 2024-09-21T06:21:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/1719911706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c108c50 0x7f1e3c10f480 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/1719911706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e3c108300 0x7f1e3c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/1719911706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 -- 172.21.15.31:0/1719911706 >> 172.21.15.31:0/1719911706 conn(0x7f1e3c07a6f0 msgr2=0x7f1e3c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 -- 172.21.15.31:0/1719911706 shutdown_connections 2024-09-21T06:21:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.247+0000 7f1e43f2b640 1 -- 172.21.15.31:0/1719911706 wait complete. 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 Processor -- start 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 -- start start 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c074fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c108300 0x7f1e3c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e3c108c50 0x7f1e3c06f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c06d6e0 con 0x7f1e3c1029d0 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c06d560 con 0x7f1e3c108300 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e43f2b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c06d860 con 0x7f1e3c108c50 2024-09-21T06:21:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.248+0000 7f1e414c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c108300 0x7f1e3c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.249+0000 7f1e41cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c074fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.249+0000 7f1e41cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c074fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48870/0 (socket says 172.21.15.31:48870) 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.249+0000 7f1e41cc7640 1 -- 172.21.15.31:0/387974613 learned_addr learned my addr 172.21.15.31:0/387974613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.249+0000 7f1e424c8640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e3c108c50 0x7f1e3c06f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.249+0000 7f1e41cc7640 1 -- 172.21.15.31:0/387974613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e3c108c50 msgr2=0x7f1e3c06f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.249+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e3c108c50 0x7f1e3c06f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.250+0000 7f1e41cc7640 1 -- 172.21.15.31:0/387974613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c108300 msgr2=0x7f1e3c06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.250+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c108300 0x7f1e3c06f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.250+0000 7f1e41cc7640 1 -- 172.21.15.31:0/387974613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c1accc0 con 0x7f1e3c1029d0 2024-09-21T06:21:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.250+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c074fa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400ba50 tx=0x7f1e3400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.251+0000 7f1e32ffd640 1 -- 172.21.15.31:0/387974613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34019070 con 0x7f1e3c1029d0 2024-09-21T06:21:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.251+0000 7f1e32ffd640 1 -- 172.21.15.31:0/387974613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3400cbe0 con 0x7f1e3c1029d0 2024-09-21T06:21:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.251+0000 7f1e32ffd640 1 -- 172.21.15.31:0/387974613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e340075f0 con 0x7f1e3c1029d0 2024-09-21T06:21:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.251+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3c1ace90 con 0x7f1e3c1029d0 2024-09-21T06:21:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.251+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e3c1ad2a0 con 0x7f1e3c1029d0 2024-09-21T06:21:00.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.255+0000 7f1e4ae99740 1 Processor -- start 2024-09-21T06:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 -- start start 2024-09-21T06:21:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c3322339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332233ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332234430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c3321a86f0 con 0x55c331e652f0 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c332197b20 con 0x55c331e6d4d0 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.256+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c33219b930 con 0x55c331e8e1b0 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.257+0000 7f1e424c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c3322339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.257+0000 7f1e424c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c3322339b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:39432/0 (socket says 172.21.15.31:39432) 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.257+0000 7f1e424c8640 1 -- 172.21.15.31:0/2327379801 learned_addr learned my addr 172.21.15.31:0/2327379801 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.258+0000 7f1e414c6640 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332234430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.258+0000 7f1e424c8640 1 -- 172.21.15.31:0/2327379801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 msgr2=0x55c332234430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.259+0000 7f1e424c8640 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332234430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.259+0000 7f1e424c8640 1 -- 172.21.15.31:0/2327379801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 msgr2=0x55c332233ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.259+0000 7f1e424c8640 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332233ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.259+0000 7f1e424c8640 1 -- 172.21.15.31:0/2327379801 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33215d530 con 0x55c331e8e1b0 2024-09-21T06:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.260+0000 7f1e424c8640 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c3322339b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e38044000 tx=0x7f1e38004850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4657128ee3ab9770 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.260+0000 7f1e27fff640 1 -- 172.21.15.31:0/2327379801 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38046070 con 0x55c331e8e1b0 2024-09-21T06:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.260+0000 7f1e27fff640 1 -- 172.21.15.31:0/2327379801 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e380049f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.260+0000 7f1e27fff640 1 -- 172.21.15.31:0/2327379801 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380035f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.261+0000 7f1e32ffd640 1 -- 172.21.15.31:0/387974613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1e34014980 con 0x7f1e3c1029d0 2024-09-21T06:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 -- 172.21.15.31:0/2327379801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 msgr2=0x55c3322339b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c3322339b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e38044000 tx=0x7f1e38004850 comp rx=0 tx=0).stop 2024-09-21T06:21:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 -- 172.21.15.31:0/2327379801 shutdown_connections 2024-09-21T06:21:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332234430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332233ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/2327379801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c3322339b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e4ae99740 1 -- 172.21.15.31:0/2327379801 >> 172.21.15.31:0/2327379801 conn(0x55c331e82300 msgr2=0x55c3322318a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e32ffd640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1e280b6800 0x7f1e280b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.262+0000 7f1e32ffd640 1 -- 172.21.15.31:0/387974613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f1e340e8ce0 con 0x7f1e3c1029d0 2024-09-21T06:21:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.263+0000 7f1e4ae99740 1 -- 172.21.15.31:0/2327379801 shutdown_connections 2024-09-21T06:21:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.263+0000 7f1e414c6640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1e280b6800 0x7f1e280b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.263+0000 7f1e4ae99740 1 -- 172.21.15.31:0/2327379801 wait complete. 2024-09-21T06:21:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.264+0000 7f1e4ae99740 1 -- start start 2024-09-21T06:21:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.264+0000 7f1e4ae99740 1 Processor -- start 2024-09-21T06:21:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.264+0000 7f1e414c6640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1e280b6800 0x7f1e280b8d10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c108040 tx=0x7f1e2c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c33223f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33223f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c33224b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c3320cfab0 con 0x55c331e652f0 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c33213d730 con 0x55c331e8e1b0 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c3322420d0 con 0x55c331e6d4d0 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e4ae99740 10 client.0 _ll_register_callbacks cb 0x55c3321916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e424c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c33224b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e424c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c33224b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52064/0 (socket says 172.21.15.31:52064) 2024-09-21T06:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.265+0000 7f1e424c8640 1 -- 172.21.15.31:0/151907588 learned_addr learned my addr 172.21.15.31:0/151907588 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c33223f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e424c8640 1 -- 172.21.15.31:0/151907588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 msgr2=0x55c33223f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e424c8640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33223f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e424c8640 1 -- 172.21.15.31:0/151907588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 msgr2=0x55c33223f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e424c8640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c33223f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e424c8640 1 -- 172.21.15.31:0/151907588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c331ef9790 con 0x55c331e8e1b0 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c33223f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e424c8640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c33224b470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e3803ba20 tx=0x7f1e38004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269f5e41b74d6e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e257fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e257fa640 1 -- 172.21.15.31:0/151907588 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38046070 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e380049f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.266+0000 7f1e257fa640 1 -- 172.21.15.31:0/151907588 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f1e38005370 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e257fa640 1 -- 172.21.15.31:0/151907588 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1e38067050 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 10 client.27194 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c331f72620 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e257fa640 1 -- 172.21.15.31:0/151907588 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e380049f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e257fa640 1 -- 172.21.15.31:0/151907588 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1e3804b040 con 0x55c331e8e1b0 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 10 client.27194 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 10 client.27194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 20 client.27194 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e4cd80 0x55c332251310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 4 client.27194 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.267+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3321aa640 con 0x55c331e4cd80 2024-09-21T06:21:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.270+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e4cd80 0x55c332251310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.271+0000 7f1e257fa640 10 client.27194 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.271+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e4cd80 0x55c332251310 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b17bae0d9a90d5 server_cookie=80846d77ea4e1a69 in_seq=0 out_seq=0 2024-09-21T06:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.272+0000 7f1e257fa640 1 -- 172.21.15.31:0/151907588 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3321aa640 con 0x55c331e4cd80 2024-09-21T06:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.272+0000 7f1e257fa640 10 client.27194 handle_command_reply: tid=0 2024-09-21T06:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.272+0000 7f1e4ae99740 1 client.27194 shutdown 2024-09-21T06:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.272+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 msgr2=0x55c33224b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.272+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c33224b470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e3803ba20 tx=0x7f1e38004ba0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 shutdown_connections 2024-09-21T06:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e4cd80 0x55c332251310 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c33224b470 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33223f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/151907588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c33223f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 >> 172.21.15.31:0/151907588 conn(0x55c331e82300 msgr2=0x55c332234970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e257fa640 0 client.27194 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 shutdown_connections 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 1 -- 172.21.15.31:0/151907588 wait complete. 2024-09-21T06:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.274+0000 7f1e4ae99740 20 client.27194 trim_cache size 0 max 16384 2024-09-21T06:21:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.299+0000 7f1e4ae99740 1 Processor -- start 2024-09-21T06:21:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 -- start start 2024-09-21T06:21:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332240cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33224b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c332230eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1b3d20 con 0x55c331e652f0 2024-09-21T06:21:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1b6580 con 0x55c331e8e1b0 2024-09-21T06:21:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1b4370 con 0x55c331e6d4d0 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e41cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332240cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e414c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33224b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e41cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332240cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48890/0 (socket says 172.21.15.31:48890) 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e41cc7640 1 -- 172.21.15.31:0/1004890826 learned_addr learned my addr 172.21.15.31:0/1004890826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e414c6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33224b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:39448/0 (socket says 172.21.15.31:39448) 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.300+0000 7f1e424c8640 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c332230eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e414c6640 1 -- 172.21.15.31:0/1004890826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 msgr2=0x55c332230eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e414c6640 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c332230eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e414c6640 1 -- 172.21.15.31:0/1004890826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 msgr2=0x55c332240cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e414c6640 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332240cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e414c6640 1 -- 172.21.15.31:0/1004890826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3322326d0 con 0x55c331e6d4d0 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e414c6640 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33224b370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009790 tx=0x7f1e2c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c89b0d8f54ceb1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e277fe640 1 -- 172.21.15.31:0/1004890826 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c020cd0 con 0x55c331e6d4d0 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e277fe640 1 -- 172.21.15.31:0/1004890826 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e2c0035f0 con 0x55c331e6d4d0 2024-09-21T06:21:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.301+0000 7f1e277fe640 1 -- 172.21.15.31:0/1004890826 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0044e0 con 0x55c331e6d4d0 2024-09-21T06:21:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.302+0000 7f1e4ae99740 1 -- 172.21.15.31:0/1004890826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 msgr2=0x55c33224b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33224b370 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009790 tx=0x7f1e2c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 -- 172.21.15.31:0/1004890826 shutdown_connections 2024-09-21T06:21:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e8e1b0 0x55c332230eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e6d4d0 0x55c33224b370 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/1004890826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332240cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 -- 172.21.15.31:0/1004890826 >> 172.21.15.31:0/1004890826 conn(0x55c331e82300 msgr2=0x55c3322405a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 -- 172.21.15.31:0/1004890826 shutdown_connections 2024-09-21T06:21:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.303+0000 7f1e4ae99740 1 -- 172.21.15.31:0/1004890826 wait complete. 2024-09-21T06:21:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.304+0000 7f1e4ae99740 1 -- start start 2024-09-21T06:21:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.304+0000 7f1e4ae99740 1 Processor -- start 2024-09-21T06:21:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.305+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332247c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.305+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332242390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.305+0000 7f1e4ae99740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c332242950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e4ae99740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1b4a40 con 0x55c331e652f0 2024-09-21T06:21:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c332242950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e414c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332242390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e414c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332242390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52090/0 (socket says 172.21.15.31:52090) 2024-09-21T06:21:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c332242950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:39456/0 (socket says 172.21.15.31:39456) 2024-09-21T06:21:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e414c6640 1 -- 172.21.15.31:0/83546845 learned_addr learned my addr 172.21.15.31:0/83546845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332247c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1b5eb0 con 0x55c331e6d4d0 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c1b5110 con 0x55c331e8e1b0 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e4ae99740 10 client.0 _ll_register_callbacks cb 0x7f1e2c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 -- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 msgr2=0x55c332242390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332242390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 -- 172.21.15.31:0/83546845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 msgr2=0x55c332247c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332247c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.306+0000 7f1e424c8640 1 -- 172.21.15.31:0/83546845 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3322430e0 con 0x55c331e8e1b0 2024-09-21T06:21:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e424c8640 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c332242950 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1e38003b40 tx=0x7f1e380050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66ca330bae935581 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e257fa640 10 client.27229 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e2c0035f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e257fa640 1 -- 172.21.15.31:0/83546845 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38046040 con 0x55c331e8e1b0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e257fa640 1 -- 172.21.15.31:0/83546845 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8499+0+0 (secure 0 0 0) 0x7f1e380035f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e257fa640 1 -- 172.21.15.31:0/83546845 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1e380037f0 con 0x55c331e8e1b0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e4ae99740 10 client.27229 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.307+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55c3322bab60 con 0x55c331e8e1b0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e257fa640 1 -- 172.21.15.31:0/83546845 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1e380052b0 con 0x55c331e8e1b0 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e4ae99740 10 client.27229 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e4ae99740 10 client.27229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e4ae99740 20 client.27229 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e221c0 0x55c3322bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e4ae99740 4 client.27229 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.308+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1e2c0035f0 con 0x55c331e221c0 2024-09-21T06:21:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.309+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e221c0 0x55c3322bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.310+0000 7f1e41cc7640 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e221c0 0x55c3322bd290 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236af8f3fa517a29 server_cookie=2dd82e3ee0284d89 in_seq=0 out_seq=0 2024-09-21T06:21:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.310+0000 7f1e257fa640 10 client.27229 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.339+0000 7f1e257fa640 1 -- 172.21.15.31:0/83546845 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1e2c0035f0 con 0x55c331e221c0 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.339+0000 7f1e257fa640 10 client.27229 handle_command_reply: tid=0 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.339+0000 7f1e4ae99740 1 client.27229 shutdown 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.339+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 msgr2=0x55c332242950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.339+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c332242950 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1e38003b40 tx=0x7f1e380050b0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 shutdown_connections 2024-09-21T06:21:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c331e221c0 0x55c3322bd290 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c331e8e1b0 0x55c332242950 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c331e6d4d0 0x55c332242390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 --2- 172.21.15.31:0/83546845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c331e652f0 0x55c332247c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 >> 172.21.15.31:0/83546845 conn(0x55c331e82300 msgr2=0x55c332234900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e257fa640 0 client.27229 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 shutdown_connections 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.340+0000 7f1e4ae99740 1 -- 172.21.15.31:0/83546845 wait complete. 2024-09-21T06:21:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.341+0000 7f1e4ae99740 20 client.27229 trim_cache size 0 max 16384 2024-09-21T06:21:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:21:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:21:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:21:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.342+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1e280b6800 msgr2=0x7f1e280b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.342+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1e280b6800 0x7f1e280b8d10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c108040 tx=0x7f1e2c005f90 comp rx=0 tx=0).stop 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.342+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 msgr2=0x7f1e3c074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.342+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c074fa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400ba50 tx=0x7f1e3400bf10 comp rx=0 tx=0).stop 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 shutdown_connections 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e3c108c50 0x7f1e3c06f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e3c108300 0x7f1e3c06f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1e280b6800 0x7f1e280b8d10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 --2- 172.21.15.31:0/387974613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e3c1029d0 0x7f1e3c074fa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 >> 172.21.15.31:0/387974613 conn(0x7f1e3c07a6f0 msgr2=0x7f1e3c1063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 shutdown_connections 2024-09-21T06:21:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:00.343+0000 7f1e43f2b640 1 -- 172.21.15.31:0/387974613 wait complete. 2024-09-21T06:21:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:01 smithi107 ceph-mon[34380]: pgmap v608: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.8 MiB/s wr, 636 op/s 2024-09-21T06:21:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:01 smithi031 ceph-mon[31237]: pgmap v608: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.8 MiB/s wr, 636 op/s 2024-09-21T06:21:02.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:01 smithi110 ceph-mon[34550]: pgmap v608: 97 pgs: 97 active+clean; 4.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.8 MiB/s wr, 636 op/s 2024-09-21T06:21:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:03 smithi031 ceph-mon[31237]: pgmap v609: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 898 op/s 2024-09-21T06:21:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:03 smithi031 ceph-mon[31237]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:21:04.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:03 smithi110 ceph-mon[34550]: pgmap v609: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 898 op/s 2024-09-21T06:21:04.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:03 smithi110 ceph-mon[34550]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:21:04.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:03 smithi107 ceph-mon[34380]: pgmap v609: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 13 MiB/s wr, 898 op/s 2024-09-21T06:21:04.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:03 smithi107 ceph-mon[34380]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:21:05.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 565.0 s, 229.2 tps, lat 255.816 ms stddev 293.655 2024-09-21T06:21:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:05 smithi031 ceph-mon[31237]: pgmap v611: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 728 op/s 2024-09-21T06:21:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:05 smithi110 ceph-mon[34550]: pgmap v611: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 728 op/s 2024-09-21T06:21:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:05 smithi107 ceph-mon[34380]: pgmap v611: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 12 MiB/s wr, 728 op/s 2024-09-21T06:21:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:07 smithi031 ceph-mon[31237]: pgmap v612: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 596 op/s 2024-09-21T06:21:08.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:07 smithi110 ceph-mon[34550]: pgmap v612: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 596 op/s 2024-09-21T06:21:08.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:07 smithi107 ceph-mon[34380]: pgmap v612: 97 pgs: 13 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 62 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 596 op/s 2024-09-21T06:21:09.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:09 smithi031 ceph-mon[31237]: pgmap v613: 97 pgs: 18 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 45 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 510 op/s 2024-09-21T06:21:10.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:09 smithi110 ceph-mon[34550]: pgmap v613: 97 pgs: 18 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 45 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 510 op/s 2024-09-21T06:21:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:09 smithi107 ceph-mon[34380]: pgmap v613: 97 pgs: 18 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 45 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 510 op/s 2024-09-21T06:21:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 570.0 s, 62.4 tps, lat 1529.238 ms stddev 1977.602 2024-09-21T06:21:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:11 smithi031 ceph-mon[31237]: pgmap v614: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 574 op/s 2024-09-21T06:21:12.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:11 smithi110 ceph-mon[34550]: pgmap v614: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 574 op/s 2024-09-21T06:21:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:11 smithi107 ceph-mon[34380]: pgmap v614: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 11 MiB/s wr, 574 op/s 2024-09-21T06:21:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.507+0000 7fdefaed7640 1 -- 172.21.15.31:0/1941125046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 msgr2=0x7fdeec0ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.507+0000 7fdefaed7640 1 --2- 172.21.15.31:0/1941125046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec0ab630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009900 tx=0x7fdef002e830 comp rx=0 tx=0).stop 2024-09-21T06:21:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.512+0000 7fdefaed7640 1 -- 172.21.15.31:0/1941125046 shutdown_connections 2024-09-21T06:21:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.512+0000 7fdefaed7640 1 --2- 172.21.15.31:0/1941125046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeec0afe50 0x7fdeec0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.512+0000 7fdefaed7640 1 --2- 172.21.15.31:0/1941125046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec0ab630 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.512+0000 7fdefaed7640 1 --2- 172.21.15.31:0/1941125046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeec0a4ba0 0x7fdeec0aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.512+0000 7fdefaed7640 1 -- 172.21.15.31:0/1941125046 >> 172.21.15.31:0/1941125046 conn(0x7fdeec01cc70 msgr2=0x7fdeec01d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.515+0000 7fdefaed7640 1 -- 172.21.15.31:0/1941125046 shutdown_connections 2024-09-21T06:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.518+0000 7fdefaed7640 1 -- 172.21.15.31:0/1941125046 wait complete. 2024-09-21T06:21:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.519+0000 7fdefaed7640 1 Processor -- start 2024-09-21T06:21:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.519+0000 7fdefaed7640 1 -- start start 2024-09-21T06:21:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.519+0000 7fdefaed7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeec0a4ba0 0x7fdeec144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.519+0000 7fdefaed7640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec1446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdefaed7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeec0afe50 0x7fdeec149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdefaed7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0b4ed0 con 0x7fdeec0afe50 2024-09-21T06:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdefaed7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0b4d50 con 0x7fdeec0a4ba0 2024-09-21T06:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdefaed7640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec0b5050 con 0x7fdeec0ab220 2024-09-21T06:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec1446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec1446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34144/0 (socket says 172.21.15.31:34144) 2024-09-21T06:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 -- 172.21.15.31:0/2679255305 learned_addr learned my addr 172.21.15.31:0/2679255305 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 -- 172.21.15.31:0/2679255305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeec0a4ba0 msgr2=0x7fdeec144140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeec0a4ba0 0x7fdeec144140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 -- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeec0afe50 msgr2=0x7fdeec149970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:21:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdefa6d6640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeec0afe50 0x7fdeec149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeec0afe50 0x7fdeec149970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.520+0000 7fdef96d4640 1 -- 172.21.15.31:0/2679255305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec14a070 con 0x7fdeec0ab220 2024-09-21T06:21:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.521+0000 7fdef96d4640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec1446e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdef002ee20 tx=0x7fdef0003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.521+0000 7fdeeaffd640 1 -- 172.21.15.31:0/2679255305 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef003a030 con 0x7fdeec0ab220 2024-09-21T06:21:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.521+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec14a300 con 0x7fdeec0ab220 2024-09-21T06:21:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.521+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec150af0 con 0x7fdeec0ab220 2024-09-21T06:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.521+0000 7fdeeaffd640 1 -- 172.21.15.31:0/2679255305 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef0043070 con 0x7fdeec0ab220 2024-09-21T06:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.521+0000 7fdeeaffd640 1 -- 172.21.15.31:0/2679255305 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef003e440 con 0x7fdeec0ab220 2024-09-21T06:21:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.529+0000 7fdeeaffd640 1 -- 172.21.15.31:0/2679255305 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdef003e5e0 con 0x7fdeec0ab220 2024-09-21T06:21:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.535+0000 7fdf030a9740 1 Processor -- start 2024-09-21T06:21:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.535+0000 7fdf030a9740 1 -- start start 2024-09-21T06:21:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.535+0000 7fdeeaffd640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdee00b67b0 0x7fdee00b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c0bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 0x557a72c0c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c0c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdf030a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557a72b806f0 con 0x557a728661b0 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdf030a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557a72b6fb20 con 0x557a728454d0 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdf030a9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a72b73930 con 0x557a7283d2f0 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdeeaffd640 1 -- 172.21.15.31:0/2679255305 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7fdef0107800 con 0x7fdeec0ab220 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdef96d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 0x557a72c0c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdef96d4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 0x557a72c0c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34158/0 (socket says 172.21.15.31:34158) 2024-09-21T06:21:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.536+0000 7fdef96d4640 1 -- 172.21.15.31:0/815890209 learned_addr learned my addr 172.21.15.31:0/815890209 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdef96d4640 1 -- 172.21.15.31:0/815890209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 msgr2=0x557a72c0c620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:21:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdef96d4640 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c0c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdef96d4640 1 -- 172.21.15.31:0/815890209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 msgr2=0x557a72c0bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdef96d4640 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c0bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdef96d4640 1 -- 172.21.15.31:0/815890209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a72b35530 con 0x557a7283d2f0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdef96d4640 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 0x557a72c0c0e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdef00cd300 tx=0x7fdef002e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94cc0da8edfbd206 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.537+0000 7fdefa6d6640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdee00b67b0 0x7fdee00b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.539+0000 7fdedffff640 1 -- 172.21.15.31:0/815890209 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0009530 con 0x557a7283d2f0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.539+0000 7fdedffff640 1 -- 172.21.15.31:0/815890209 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef00cd110 con 0x557a7283d2f0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.539+0000 7fdefa6d6640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdee00b67b0 0x7fdee00b8cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdef4051230 tx=0x7fdef40692f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.539+0000 7fdedffff640 1 -- 172.21.15.31:0/815890209 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00d7020 con 0x557a7283d2f0 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 -- 172.21.15.31:0/815890209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 msgr2=0x557a72c0c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 0x557a72c0c0e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdef00cd300 tx=0x7fdef002e830 comp rx=0 tx=0).stop 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 -- 172.21.15.31:0/815890209 shutdown_connections 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c0c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a7283d2f0 0x557a72c0c0e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 --2- 172.21.15.31:0/815890209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c0bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 -- 172.21.15.31:0/815890209 >> 172.21.15.31:0/815890209 conn(0x557a7285a300 msgr2=0x557a72c09a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 -- 172.21.15.31:0/815890209 shutdown_connections 2024-09-21T06:21:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.540+0000 7fdf030a9740 1 -- 172.21.15.31:0/815890209 wait complete. 2024-09-21T06:21:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.541+0000 7fdf030a9740 1 -- start start 2024-09-21T06:21:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.541+0000 7fdf030a9740 1 Processor -- start 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 0x557a72c17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c17770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728661b0 0x557a72c34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557a72aa7ab0 con 0x557a7283d2f0 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557a72b15730 con 0x557a728454d0 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a72c2b0f0 con 0x557a728661b0 2024-09-21T06:21:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdf030a9740 10 client.0 _ll_register_callbacks cb 0x557a72b696b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 0x557a72c17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 0x557a72c17230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58292/0 (socket says 172.21.15.31:58292) 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3583599365 learned_addr learned my addr 172.21.15.31:0/3583599365 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdefa6d6640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728661b0 0x557a72c34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef96d4640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c17770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3583599365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728661b0 msgr2=0x557a72c34350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728661b0 0x557a72c34350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 msgr2=0x557a72c17770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.542+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c17770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.543+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3583599365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a728d1790 con 0x557a7283d2f0 2024-09-21T06:21:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.545+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 0x557a72c17230 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e3a0 tx=0x7fdee400e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21593f5f4f3a93f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.545+0000 7fdedd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:21:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.545+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3583599365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4010900 con 0x557a7283d2f0 2024-09-21T06:21:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.545+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3583599365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7fdee4010aa0 con 0x557a7283d2f0 2024-09-21T06:21:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.545+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef00cd110 con 0x557a7283d2f0 2024-09-21T06:21:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3583599365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdee4010ca0 con 0x557a7283d2f0 2024-09-21T06:21:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 10 client.17418 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557a7294a620 con 0x557a7283d2f0 2024-09-21T06:21:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3583599365 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef00cd110 con 0x557a7283d2f0 2024-09-21T06:21:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3583599365 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdee4018420 con 0x557a7283d2f0 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 10 client.17418 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 20 client.17418 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72824d80 0x557a72c3a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 4 client.17418 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a72b82640 con 0x557a72824d80 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.546+0000 7fdef96d4640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72824d80 0x557a72c3a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.547+0000 7fdedd7fa640 10 client.17418 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.547+0000 7fdef96d4640 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72824d80 0x557a72c3a1c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3af2e9fd582c3e0 server_cookie=4db61fe828fa475f in_seq=0 out_seq=0 2024-09-21T06:21:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3583599365 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a72b82640 con 0x557a72824d80 2024-09-21T06:21:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdedd7fa640 10 client.17418 handle_command_reply: tid=0 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 client.17418 shutdown 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 msgr2=0x557a72c17230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 0x557a72c17230 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e3a0 tx=0x7fdee400e860 comp rx=0 tx=0).stop 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 shutdown_connections 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72824d80 0x557a72c3a1c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728661b0 0x557a72c34350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728454d0 0x557a72c17770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3583599365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a7283d2f0 0x557a72c17230 secure :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdee400e3a0 tx=0x7fdee400e860 comp rx=0 tx=0).stop 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 >> 172.21.15.31:0/3583599365 conn(0x557a7285a300 msgr2=0x557a72c08150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdedd7fa640 0 client.17418 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 shutdown_connections 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.549+0000 7fdf030a9740 1 -- 172.21.15.31:0/3583599365 wait complete. 2024-09-21T06:21:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.550+0000 7fdf030a9740 20 client.17418 trim_cache size 0 max 16384 2024-09-21T06:21:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 Processor -- start 2024-09-21T06:21:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 -- start start 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c09ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728661b0 0x557a72c34250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 0x557a72c14400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fded800c7b0 con 0x557a727fa1c0 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fded800f010 con 0x557a728661b0 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdf030a9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fded800ce00 con 0x557a728454d0 2024-09-21T06:21:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdefa6d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 0x557a72c14400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdefa6d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 0x557a72c14400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58296/0 (socket says 172.21.15.31:58296) 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdefa6d6640 1 -- 172.21.15.31:0/2641978199 learned_addr learned my addr 172.21.15.31:0/2641978199 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c09ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef96d4640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728661b0 0x557a72c34250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef9ed5640 1 -- 172.21.15.31:0/2641978199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728661b0 msgr2=0x557a72c34250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728661b0 0x557a72c34250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef9ed5640 1 -- 172.21.15.31:0/2641978199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 msgr2=0x557a72c14400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 0x557a72c14400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.570+0000 7fdef9ed5640 1 -- 172.21.15.31:0/2641978199 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a72c0e9b0 con 0x557a728454d0 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.571+0000 7fdefa6d6640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 0x557a72c14400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.571+0000 7fdef96d4640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728661b0 0x557a72c34250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.571+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c09ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdee4008000 tx=0x7fdee40096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d13e920580d26d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.571+0000 7fdedf7fe640 1 -- 172.21.15.31:0/2641978199 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4030070 con 0x557a728454d0 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.571+0000 7fdedf7fe640 1 -- 172.21.15.31:0/2641978199 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee4018a80 con 0x557a728454d0 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.571+0000 7fdedf7fe640 1 -- 172.21.15.31:0/2641978199 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4005680 con 0x557a728454d0 2024-09-21T06:21:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 -- 172.21.15.31:0/2641978199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 msgr2=0x557a72c09ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c09ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdee4008000 tx=0x7fdee40096d0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 -- 172.21.15.31:0/2641978199 shutdown_connections 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a727fa1c0 0x557a72c14400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a728661b0 0x557a72c34250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 --2- 172.21.15.31:0/2641978199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c09ac0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 -- 172.21.15.31:0/2641978199 >> 172.21.15.31:0/2641978199 conn(0x557a7285a300 msgr2=0x557a72c093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 -- 172.21.15.31:0/2641978199 shutdown_connections 2024-09-21T06:21:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 -- 172.21.15.31:0/2641978199 wait complete. 2024-09-21T06:21:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 -- start start 2024-09-21T06:21:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.572+0000 7fdf030a9740 1 Processor -- start 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.573+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 0x557a72c2c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.573+0000 7fdf030a9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c2c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.573+0000 7fdef9ed5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 0x557a72c2c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.573+0000 7fdef9ed5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 0x557a72c2c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33822/0 (socket says 172.21.15.31:33822) 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3835634607 learned_addr learned my addr 172.21.15.31:0/3835634607 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c2ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdef96d4640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c2c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fded800d4d0 con 0x557a728661b0 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdefa6d6640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c2ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fded800e940 con 0x557a727fa1c0 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fded800dba0 con 0x557a728454d0 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdf030a9740 10 client.0 _ll_register_callbacks cb 0x7fdee4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3835634607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 msgr2=0x557a72c2c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c2c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3835634607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 msgr2=0x557a72c2ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.574+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c2ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdef9ed5640 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a72ca3cf0 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdef9ed5640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 0x557a72c2c3e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdee4000c00 tx=0x7fdee4018650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c14eedb4c4776a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdedd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3835634607 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4030070 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3835634607 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7fdee4010900 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdee4018a80 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3835634607 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdee4005490 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdf030a9740 10 client.27212 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.575+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557a72ca4f60 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3835634607 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdee4018a80 con 0x557a727fa1c0 2024-09-21T06:21:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3835634607 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdee402a070 con 0x557a727fa1c0 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdf030a9740 10 client.27212 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdf030a9740 10 client.27212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdf030a9740 20 client.27212 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72ca5640 0x557a72ca7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdf030a9740 4 client.27212 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a72b7e0f0 con 0x557a72ca5640 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.576+0000 7fdef96d4640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72ca5640 0x557a72ca7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.577+0000 7fdedd7fa640 10 client.27212 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.577+0000 7fdef96d4640 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72ca5640 0x557a72ca7a60 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d4603bb2a5f025 server_cookie=7d0609b63abc91e in_seq=0 out_seq=0 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdedd7fa640 1 -- 172.21.15.31:0/3835634607 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a72b7e0f0 con 0x557a72ca5640 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdedd7fa640 10 client.27212 handle_command_reply: tid=0 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 client.27212 shutdown 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 msgr2=0x557a72c2c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 0x557a72c2c3e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdee4000c00 tx=0x7fdee4018650 comp rx=0 tx=0).stop 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 shutdown_connections 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557a72ca5640 0x557a72ca7a60 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a728661b0 0x557a72c2ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdedd7fa640 0 client.27212 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a728454d0 0x557a72c2c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 --2- 172.21.15.31:0/3835634607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a727fa1c0 0x557a72c2c3e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.598+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 >> 172.21.15.31:0/3835634607 conn(0x557a7285a300 msgr2=0x557a72c186e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 shutdown_connections 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdf030a9740 1 -- 172.21.15.31:0/3835634607 wait complete. 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdf030a9740 20 client.27212 trim_cache size 0 max 16384 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:21:13.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:21:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdee00b67b0 msgr2=0x7fdee00b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdee00b67b0 0x7fdee00b8cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdef4051230 tx=0x7fdef40692f0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 msgr2=0x7fdeec1446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec1446e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdef002ee20 tx=0x7fdef0003890 comp rx=0 tx=0).stop 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 shutdown_connections 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdee00b67b0 0x7fdee00b8cc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdeec0afe50 0x7fdeec149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdeec0ab220 0x7fdeec1446e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 --2- 172.21.15.31:0/2679255305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeec0a4ba0 0x7fdeec144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 >> 172.21.15.31:0/2679255305 conn(0x7fdeec01cc70 msgr2=0x7fdeec0a6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 shutdown_connections 2024-09-21T06:21:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:13.599+0000 7fdefaed7640 1 -- 172.21.15.31:0/2679255305 wait complete. 2024-09-21T06:21:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:13 smithi031 ceph-mon[31237]: pgmap v615: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-09-21T06:21:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:13 smithi031 ceph-mon[31237]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:21:14.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:13 smithi110 ceph-mon[34550]: pgmap v615: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-09-21T06:21:14.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:13 smithi110 ceph-mon[34550]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:21:14.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:13 smithi107 ceph-mon[34380]: pgmap v615: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-09-21T06:21:14.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:13 smithi107 ceph-mon[34380]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:21:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 575.0 s, 1761.6 tps, lat 82.672 ms stddev 265.946 2024-09-21T06:21:15.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:15 smithi031 ceph-mon[31237]: pgmap v617: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-09-21T06:21:16.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:15 smithi110 ceph-mon[34550]: pgmap v617: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-09-21T06:21:16.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:15 smithi107 ceph-mon[34380]: pgmap v617: 97 pgs: 97 active+clean; 4.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 1.44k op/s 2024-09-21T06:21:17.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:21:17.830 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:17 smithi110 ceph-mon[34550]: pgmap v618: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2024-09-21T06:21:17.840 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:17 smithi107 ceph-mon[34380]: pgmap v618: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2024-09-21T06:21:17.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:17 smithi031 ceph-mon[31237]: pgmap v618: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 27 MiB/s wr, 1.66k op/s 2024-09-21T06:21:18.190 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:18.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.836+0000 7f87e7727640 1 -- 172.21.15.31:0/2967139593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e01083a0 msgr2=0x7f87e01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.836+0000 7f87e7727640 1 --2- 172.21.15.31:0/2967139593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e01083a0 0x7f87e01087b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009be0 tx=0x7f87d8032c80 comp rx=0 tx=0).stop 2024-09-21T06:21:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.837+0000 7f87e7727640 1 -- 172.21.15.31:0/2967139593 shutdown_connections 2024-09-21T06:21:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.837+0000 7f87e7727640 1 --2- 172.21.15.31:0/2967139593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0108cf0 0x7f87e010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.837+0000 7f87e7727640 1 --2- 172.21.15.31:0/2967139593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e01083a0 0x7f87e01087b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.837+0000 7f87e7727640 1 --2- 172.21.15.31:0/2967139593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e0102a60 0x7f87e0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.837+0000 7f87e7727640 1 -- 172.21.15.31:0/2967139593 >> 172.21.15.31:0/2967139593 conn(0x7f87e0079730 msgr2=0x7f87e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:18.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.839+0000 7f87e7727640 1 -- 172.21.15.31:0/2967139593 shutdown_connections 2024-09-21T06:21:18.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.839+0000 7f87e7727640 1 -- 172.21.15.31:0/2967139593 wait complete. 2024-09-21T06:21:18.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.840+0000 7f87e7727640 1 Processor -- start 2024-09-21T06:21:18.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.840+0000 7f87e7727640 1 -- start start 2024-09-21T06:21:18.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e7727640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0102a60 0x7f87e0074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e7727640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 0x7f87e006e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e7727640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0108cf0 0x7f87e006e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e7727640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0122920 con 0x7f87e0108cf0 2024-09-21T06:21:18.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e7727640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f87e01227a0 con 0x7f87e01083a0 2024-09-21T06:21:18.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e7727640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f87e0122aa0 con 0x7f87e0102a60 2024-09-21T06:21:18.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e4cc2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 0x7f87e006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e4cc2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 0x7f87e006e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33844/0 (socket says 172.21.15.31:33844) 2024-09-21T06:21:18.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e4cc2640 1 -- 172.21.15.31:0/2456685791 learned_addr learned my addr 172.21.15.31:0/2456685791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:18.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.841+0000 7f87e54c3640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0102a60 0x7f87e0074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.846+0000 7f87e54c3640 1 -- 172.21.15.31:0/2456685791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 msgr2=0x7f87e006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.846+0000 7f87e54c3640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 0x7f87e006e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.846+0000 7f87e54c3640 1 -- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0108cf0 msgr2=0x7f87e006e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.846+0000 7f87e54c3640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0108cf0 0x7f87e006e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.846+0000 7f87e54c3640 1 -- 172.21.15.31:0/2456685791 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e006ed10 con 0x7f87e0102a60 2024-09-21T06:21:18.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.847+0000 7f87e54c3640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0102a60 0x7f87e0074d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d6d0 tx=0x7f87dc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:18.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.849+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 0x7f87e006e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:18.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.850+0000 7f87d67fc640 1 -- 172.21.15.31:0/2456685791 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc017070 con 0x7f87e0102a60 2024-09-21T06:21:18.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.850+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e00739f0 con 0x7f87e0102a60 2024-09-21T06:21:18.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.850+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e0073ed0 con 0x7f87e0102a60 2024-09-21T06:21:18.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.850+0000 7f87d67fc640 1 -- 172.21.15.31:0/2456685791 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87dc010460 con 0x7f87e0102a60 2024-09-21T06:21:18.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.850+0000 7f87d67fc640 1 -- 172.21.15.31:0/2456685791 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00f620 con 0x7f87e0102a60 2024-09-21T06:21:18.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.854+0000 7f87edc9b740 1 Processor -- start 2024-09-21T06:21:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.857+0000 7f87d67fc640 1 -- 172.21.15.31:0/2456685791 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f87dc01a070 con 0x7f87e0102a60 2024-09-21T06:21:18.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87d67fc640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87cc0b6800 0x7f87cc0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 -- start start 2024-09-21T06:21:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702db1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e11b0 0x55d702db1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 0x55d702db20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d702d27e50 con 0x55d7029e11b0 2024-09-21T06:21:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d702d17280 con 0x55d702a0a070 2024-09-21T06:21:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.860+0000 7f87edc9b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d702d1b090 con 0x55d7029e9390 2024-09-21T06:21:18.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.861+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87cc0b6800 0x7f87cc0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.861+0000 7f87d67fc640 1 -- 172.21.15.31:0/2456685791 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f87dc0a8fd0 con 0x7f87e0102a60 2024-09-21T06:21:18.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 0x55d702db20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 0x55d702db20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34202/0 (socket says 172.21.15.31:34202) 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 -- 172.21.15.31:0/1843703271 learned_addr learned my addr 172.21.15.31:0/1843703271 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e54c3640 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702db1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 -- 172.21.15.31:0/1843703271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 msgr2=0x55d702db1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702db1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 -- 172.21.15.31:0/1843703271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e11b0 msgr2=0x55d702db1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e11b0 0x55d702db1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.862+0000 7f87e5cc4640 1 -- 172.21.15.31:0/1843703271 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d702a75650 con 0x55d7029e9390 2024-09-21T06:21:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.863+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 0x55d702db20d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f87d0013980 tx=0x7f87d0013e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21f43183c62569b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:18.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.864+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87cc0b6800 0x7f87cc0b8d10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87d0010740 tx=0x7f87d001c000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:18.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.866+0000 7f87cb7fe640 1 -- 172.21.15.31:0/1843703271 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0005020 con 0x55d7029e9390 2024-09-21T06:21:18.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.866+0000 7f87cb7fe640 1 -- 172.21.15.31:0/1843703271 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d00117b0 con 0x55d7029e9390 2024-09-21T06:21:18.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.867+0000 7f87edc9b740 1 -- 172.21.15.31:0/1843703271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 msgr2=0x55d702db20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.867+0000 7f87edc9b740 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 0x55d702db20d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f87d0013980 tx=0x7f87d0013e40 comp rx=0 tx=0).stop 2024-09-21T06:21:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.869+0000 7f87edc9b740 1 -- 172.21.15.31:0/1843703271 shutdown_connections 2024-09-21T06:21:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.869+0000 7f87edc9b740 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e9390 0x55d702db20d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.869+0000 7f87edc9b740 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e11b0 0x55d702db1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.869+0000 7f87edc9b740 1 --2- 172.21.15.31:0/1843703271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702db1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.869+0000 7f87edc9b740 1 -- 172.21.15.31:0/1843703271 >> 172.21.15.31:0/1843703271 conn(0x55d7029fe460 msgr2=0x55d702daf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.871+0000 7f87edc9b740 1 -- 172.21.15.31:0/1843703271 shutdown_connections 2024-09-21T06:21:18.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.871+0000 7f87edc9b740 1 -- 172.21.15.31:0/1843703271 wait complete. 2024-09-21T06:21:18.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.871+0000 7f87edc9b740 1 -- start start 2024-09-21T06:21:18.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.872+0000 7f87edc9b740 1 Processor -- start 2024-09-21T06:21:18.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 0x55d702dbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702dbd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d702a0a070 0x55d702dbd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d702b78700 con 0x55d7029e9390 2024-09-21T06:21:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d702cb2110 con 0x55d7029e11b0 2024-09-21T06:21:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d702dbfe20 con 0x55d702a0a070 2024-09-21T06:21:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.874+0000 7f87edc9b740 10 client.0 _ll_register_callbacks cb 0x55d702ccb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 0x55d702dbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 0x55d702dbcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33864/0 (socket says 172.21.15.31:33864) 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 -- 172.21.15.31:0/3011495553 learned_addr learned my addr 172.21.15.31:0/3011495553 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 -- 172.21.15.31:0/3011495553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d702a0a070 msgr2=0x55d702dbd680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d702a0a070 0x55d702dbd680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 -- 172.21.15.31:0/3011495553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 msgr2=0x55d702dbd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702dbd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 -- 172.21.15.31:0/3011495553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d702cdfc40 con 0x55d7029e11b0 2024-09-21T06:21:18.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.875+0000 7f87e54c3640 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 0x55d702dbcc00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0a9890 tx=0x7f87dc0b5c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4927802c5c03e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.876+0000 7f87c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.876+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3011495553 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0a9550 con 0x55d7029e11b0 2024-09-21T06:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.876+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d00117b0 con 0x55d7029e11b0 2024-09-21T06:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.876+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3011495553 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f87dc0a8fd0 con 0x55d7029e11b0 2024-09-21T06:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.876+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3011495553 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f87dc0a96f0 con 0x55d7029e11b0 2024-09-21T06:21:18.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.877+0000 7f87edc9b740 10 client.27224 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:18.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.877+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d702dcbab0 con 0x55d7029e11b0 2024-09-21T06:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.877+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3011495553 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d00117b0 con 0x55d7029e11b0 2024-09-21T06:21:18.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.877+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3011495553 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f87dc0d41b0 con 0x55d7029e11b0 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87edc9b740 10 client.27224 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87edc9b740 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87edc9b740 20 client.27224 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7029c89f0 0x55d702dcefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87edc9b740 4 client.27224 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d702d29da0 con 0x55d7029c89f0 2024-09-21T06:21:18.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.879+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7029c89f0 0x55d702dcefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.880+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7029c89f0 0x55d702dcefa0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9df0a343cf0cf8 server_cookie=75f63325155878f0 in_seq=0 out_seq=0 2024-09-21T06:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.880+0000 7f87c8ff9640 10 client.27224 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.882+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3011495553 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d702d29da0 con 0x55d7029c89f0 2024-09-21T06:21:18.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.882+0000 7f87c8ff9640 10 client.27224 handle_command_reply: tid=0 2024-09-21T06:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.884+0000 7f87edc9b740 1 client.27224 shutdown 2024-09-21T06:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 msgr2=0x55d702dbcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 0x55d702dbcc00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0a9890 tx=0x7f87dc0b5c50 comp rx=0 tx=0).stop 2024-09-21T06:21:18.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 shutdown_connections 2024-09-21T06:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7029c89f0 0x55d702dcefa0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d702a0a070 0x55d702dbd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702dbd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3011495553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e11b0 0x55d702dbcc00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 >> 172.21.15.31:0/3011495553 conn(0x55d7029fe460 msgr2=0x55d702db47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:18.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.885+0000 7f87c8ff9640 0 client.27224 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:18.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.886+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 shutdown_connections 2024-09-21T06:21:18.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.886+0000 7f87edc9b740 1 -- 172.21.15.31:0/3011495553 wait complete. 2024-09-21T06:21:18.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.886+0000 7f87edc9b740 20 client.27224 trim_cache size 0 max 16384 2024-09-21T06:21:18.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.912+0000 7f87edc9b740 1 Processor -- start 2024-09-21T06:21:18.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 -- start start 2024-09-21T06:21:18.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dbeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702db2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702dc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87c400c910 con 0x55d7029e9390 2024-09-21T06:21:18.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f87c400f170 con 0x55d702a0a070 2024-09-21T06:21:18.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87edc9b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f87c400cf60 con 0x55d7029e11b0 2024-09-21T06:21:18.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87e4cc2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702db2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87e4cc2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702db2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58356/0 (socket says 172.21.15.31:58356) 2024-09-21T06:21:18.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87e4cc2640 1 -- 172.21.15.31:0/123311832 learned_addr learned my addr 172.21.15.31:0/123311832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:18.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87e54c3640 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dbeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.913+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702dc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e4cc2640 1 -- 172.21.15.31:0/123311832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 msgr2=0x55d702dbeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dbeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e4cc2640 1 -- 172.21.15.31:0/123311832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 msgr2=0x55d702dc87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702dc87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e4cc2640 1 -- 172.21.15.31:0/123311832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d702db3940 con 0x55d7029e9390 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702dc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702db2680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f87d8033270 tx=0x7f87d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a129e66c070891b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87caffd640 1 -- 172.21.15.31:0/123311832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d803a3e0 con 0x55d7029e9390 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87caffd640 1 -- 172.21.15.31:0/123311832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d8047020 con 0x55d7029e9390 2024-09-21T06:21:18.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.914+0000 7f87caffd640 1 -- 172.21.15.31:0/123311832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d803a8a0 con 0x55d7029e9390 2024-09-21T06:21:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.916+0000 7f87edc9b740 1 -- 172.21.15.31:0/123311832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 msgr2=0x55d702db2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.916+0000 7f87edc9b740 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702db2680 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f87d8033270 tx=0x7f87d8005fd0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 -- 172.21.15.31:0/123311832 shutdown_connections 2024-09-21T06:21:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d702a0a070 0x55d702dc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7029e9390 0x55d702db2680 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 --2- 172.21.15.31:0/123311832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dbeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 -- 172.21.15.31:0/123311832 >> 172.21.15.31:0/123311832 conn(0x55d7029fe460 msgr2=0x55d702dc80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 -- 172.21.15.31:0/123311832 shutdown_connections 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 -- 172.21.15.31:0/123311832 wait complete. 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.917+0000 7f87edc9b740 1 -- start start 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.918+0000 7f87edc9b740 1 Processor -- start 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.918+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dc5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.918+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e9390 0x55d702dc1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.918+0000 7f87edc9b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d702a0a070 0x55d702dc1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.918+0000 7f87e54c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dc5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.918+0000 7f87e54c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dc5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34240/0 (socket says 172.21.15.31:34240) 2024-09-21T06:21:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e4cc2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e9390 0x55d702dc1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e4cc2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e9390 0x55d702dc1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33874/0 (socket says 172.21.15.31:33874) 2024-09-21T06:21:18.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e54c3640 1 -- 172.21.15.31:0/3706450742 learned_addr learned my addr 172.21.15.31:0/3706450742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87c400d630 con 0x55d702a0a070 2024-09-21T06:21:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f87c400eaa0 con 0x55d7029e9390 2024-09-21T06:21:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f87c400dd00 con 0x55d7029e11b0 2024-09-21T06:21:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87edc9b740 10 client.0 _ll_register_callbacks cb 0x7f87dc071510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e5cc4640 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d702a0a070 0x55d702dc1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e54c3640 1 -- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e9390 msgr2=0x55d702dc1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e54c3640 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e9390 0x55d702dc1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e54c3640 1 -- 172.21.15.31:0/3706450742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d702a0a070 msgr2=0x55d702dc1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e54c3640 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d702a0a070 0x55d702dc1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.919+0000 7f87e54c3640 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d702e389a0 con 0x55d7029e11b0 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87e54c3640 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dc5b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d6a0 tx=0x7f87dc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46520e28d0f81fef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:21:18.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3706450742 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc021070 con 0x55d7029e11b0 2024-09-21T06:21:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3706450742 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f87dc0ac4d0 con 0x55d7029e11b0 2024-09-21T06:21:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d8047020 con 0x55d7029e11b0 2024-09-21T06:21:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3706450742 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f87dc0a98c0 con 0x55d7029e11b0 2024-09-21T06:21:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87edc9b740 10 client.27262 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.920+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55d702e39aa0 con 0x55d7029e11b0 2024-09-21T06:21:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3706450742 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d8047020 con 0x55d7029e11b0 2024-09-21T06:21:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3706450742 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f87dc0b1240 con 0x55d7029e11b0 2024-09-21T06:21:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87edc9b740 10 client.27262 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87edc9b740 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87edc9b740 20 client.27262 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d70299dda0 0x55d702e3c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87edc9b740 4 client.27262 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d702d25850 con 0x55d70299dda0 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.921+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d70299dda0 0x55d702e3c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.922+0000 7f87c8ff9640 10 client.27262 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.922+0000 7f87e4cc2640 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d70299dda0 0x55d702e3c230 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90451b7a910c0070 server_cookie=a26e521ce7eb058a in_seq=0 out_seq=0 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.922+0000 7f87c8ff9640 1 -- 172.21.15.31:0/3706450742 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6906 (crc 0 0 0) 0x55d702d25850 con 0x55d70299dda0 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.922+0000 7f87c8ff9640 10 client.27262 handle_command_reply: tid=0 2024-09-21T06:21:18.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.923+0000 7f87edc9b740 1 client.27262 shutdown 2024-09-21T06:21:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.923+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 msgr2=0x55d702dc5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.923+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dc5b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d6a0 tx=0x7f87dc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.923+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 shutdown_connections 2024-09-21T06:21:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.923+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d70299dda0 0x55d702e3c230 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.923+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d702a0a070 0x55d702dc1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87c8ff9640 0 client.27262 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7029e9390 0x55d702dc1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87edc9b740 1 --2- 172.21.15.31:0/3706450742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7029e11b0 0x55d702dc5b20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 >> 172.21.15.31:0/3706450742 conn(0x55d7029fe460 msgr2=0x55d702dbe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 shutdown_connections 2024-09-21T06:21:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87edc9b740 1 -- 172.21.15.31:0/3706450742 wait complete. 2024-09-21T06:21:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.924+0000 7f87edc9b740 20 client.27262 trim_cache size 0 max 16384 2024-09-21T06:21:18.940 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:21:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:21:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:21:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:21:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:21:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:21:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:21:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:21:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:21:18.957 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:21:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.925+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87cc0b6800 msgr2=0x7f87cc0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.925+0000 7f87e7727640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87cc0b6800 0x7f87cc0b8d10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87d0010740 tx=0x7f87d001c000 comp rx=0 tx=0).stop 2024-09-21T06:21:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.925+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0102a60 msgr2=0x7f87e0074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0102a60 0x7f87e0074d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d6d0 tx=0x7f87dc00db90 comp rx=0 tx=0).stop 2024-09-21T06:21:18.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 shutdown_connections 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f87cc0b6800 0x7f87cc0b8d10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87e0108cf0 0x7f87e006e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87e01083a0 0x7f87e006e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 --2- 172.21.15.31:0/2456685791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e0102a60 0x7f87e0074d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 >> 172.21.15.31:0/2456685791 conn(0x7f87e0079730 msgr2=0x7f87e0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 shutdown_connections 2024-09-21T06:21:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:18.926+0000 7f87e7727640 1 -- 172.21.15.31:0/2456685791 wait complete. 2024-09-21T06:21:19.777 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:21:19.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:21:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:19 smithi107 ceph-mon[34380]: pgmap v619: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-09-21T06:21:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:19 smithi031 ceph-mon[31237]: pgmap v619: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-09-21T06:21:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:19 smithi110 ceph-mon[34550]: pgmap v619: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.75k op/s 2024-09-21T06:21:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 580.0 s, 1913.4 tps, lat 52.376 ms stddev 67.195 2024-09-21T06:21:20.211 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 -- 172.21.15.31:0/3477244928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34108cf0 msgr2=0x7f6b3406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/3477244928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34108cf0 0x7f6b3406ae60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009ca0 tx=0x7f6b2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:21:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 -- 172.21.15.31:0/3477244928 shutdown_connections 2024-09-21T06:21:20.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/3477244928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34108cf0 0x7f6b3406ae60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/3477244928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b341083a0 0x7f6b341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:20.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/3477244928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b34102a60 0x7f6b34102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.808+0000 7f6b38e6b640 1 -- 172.21.15.31:0/3477244928 >> 172.21.15.31:0/3477244928 conn(0x7f6b34079730 msgr2=0x7f6b34079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.809+0000 7f6b38e6b640 1 -- 172.21.15.31:0/3477244928 shutdown_connections 2024-09-21T06:21:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.809+0000 7f6b38e6b640 1 -- 172.21.15.31:0/3477244928 wait complete. 2024-09-21T06:21:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.810+0000 7f6b38e6b640 1 Processor -- start 2024-09-21T06:21:20.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.810+0000 7f6b38e6b640 1 -- start start 2024-09-21T06:21:20.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.810+0000 7f6b38e6b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 0x7f6b3419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:20.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.810+0000 7f6b38e6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b341083a0 0x7f6b3419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:20.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.810+0000 7f6b38e6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 0x7f6b341a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:20.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b38e6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34115cc0 con 0x7f6b341083a0 2024-09-21T06:21:20.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b38e6b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34115b40 con 0x7f6b34108cf0 2024-09-21T06:21:20.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b32d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 0x7f6b341a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:20.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 0x7f6b3419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:20.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 0x7f6b3419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33868/0 (socket says 172.21.15.31:33868) 2024-09-21T06:21:20.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 -- 172.21.15.31:0/1800461641 learned_addr learned my addr 172.21.15.31:0/1800461641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b32d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 0x7f6b341a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34770/0 (socket says 172.21.15.31:34770) 2024-09-21T06:21:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6b34115e40 con 0x7f6b34102a60 2024-09-21T06:21:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b31d9b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b341083a0 0x7f6b3419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:20.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 -- 172.21.15.31:0/1800461641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 msgr2=0x7f6b341a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 0x7f6b341a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 -- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b341083a0 msgr2=0x7f6b3419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:20.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b341083a0 0x7f6b3419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b3259c640 1 -- 172.21.15.31:0/1800461641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b341a2570 con 0x7f6b34102a60 2024-09-21T06:21:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.811+0000 7f6b32d9d640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 0x7f6b341a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.812+0000 7f6b3259c640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 0x7f6b3419c600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400d950 tx=0x7f6b2400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.813+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24017070 con 0x7f6b34102a60 2024-09-21T06:21:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.813+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b24009c30 con 0x7f6b34102a60 2024-09-21T06:21:20.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.814+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400f620 con 0x7f6b34102a60 2024-09-21T06:21:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.814+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b341a2860 con 0x7f6b34102a60 2024-09-21T06:21:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.814+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b341a2c60 con 0x7f6b34102a60 2024-09-21T06:21:20.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.816+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b34103ce0 con 0x7f6b34102a60 2024-09-21T06:21:20.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.817+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6b24010460 con 0x7f6b34102a60 2024-09-21T06:21:20.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.818+0000 7f6b237fe640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b1c0b66b0 0x7f6b1c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:20.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.819+0000 7f6b31d9b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b1c0b66b0 0x7f6b1c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:20.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.819+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f6b240e6ee0 con 0x7f6b34102a60 2024-09-21T06:21:20.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.819+0000 7f6b31d9b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b1c0b66b0 0x7f6b1c0b8bc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b34107ec0 tx=0x7f6b2800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:20.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:20.822+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b24020070 con 0x7f6b34102a60 2024-09-21T06:21:21.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.163+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b341a8fc0 con 0x7f6b34102a60 2024-09-21T06:21:21.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.166+0000 7f6b237fe640 1 -- 172.21.15.31:0/1800461641 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11850 (secure 0 0 0) 0x7f6b240ad4c0 con 0x7f6b34102a60 2024-09-21T06:21:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:21:21.171 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-21T06:09:46:301719+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:09:45.967301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24686":{"gid":24686,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6837/2795950298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2795950298},{"type":"v1","addr":"172.21.15.107:6837","nonce":2795950298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:21:21.171 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-21T06:21:21.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b1c0b66b0 msgr2=0x7f6b1c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:21.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b1c0b66b0 0x7f6b1c0b8bc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b34107ec0 tx=0x7f6b2800b390 comp rx=0 tx=0).stop 2024-09-21T06:21:21.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 msgr2=0x7f6b3419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:21.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 0x7f6b3419c600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400d950 tx=0x7f6b2400de10 comp rx=0 tx=0).stop 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 shutdown_connections 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b34108cf0 0x7f6b341a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6b1c0b66b0 0x7f6b1c0b8bc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b341083a0 0x7f6b3419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 --2- 172.21.15.31:0/1800461641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b34102a60 0x7f6b3419c600 secure :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400d950 tx=0x7f6b2400de10 comp rx=0 tx=0).stop 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 >> 172.21.15.31:0/1800461641 conn(0x7f6b34079730 msgr2=0x7f6b34101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:21.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 shutdown_connections 2024-09-21T06:21:21.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:21.176+0000 7f6b38e6b640 1 -- 172.21.15.31:0/1800461641 wait complete. 2024-09-21T06:21:21.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:21 smithi031 ceph-mon[31237]: pgmap v620: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2024-09-21T06:21:21.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:21 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1800461641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:21.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:21:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:21 smithi110 ceph-mon[34550]: pgmap v620: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2024-09-21T06:21:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:21 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1800461641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:21 smithi107 ceph-mon[34380]: pgmap v620: 97 pgs: 97 active+clean; 4.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2024-09-21T06:21:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:21 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1800461641' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:21:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:22.705 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:22 smithi031 ceph-mon[31237]: pgmap v621: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 885 op/s 2024-09-21T06:21:23.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:22 smithi110 ceph-mon[34550]: pgmap v621: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 885 op/s 2024-09-21T06:21:23.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:22 smithi107 ceph-mon[34380]: pgmap v621: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 19 MiB/s wr, 885 op/s 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 -- 172.21.15.31:0/2830251925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102a60 msgr2=0x7fdaf8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 --2- 172.21.15.31:0/2830251925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102a60 0x7fdaf8102e70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009900 tx=0x7fdaf402a440 comp rx=0 tx=0).stop 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 -- 172.21.15.31:0/2830251925 shutdown_connections 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 --2- 172.21.15.31:0/2830251925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf8108cf0 0x7fdaf810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 --2- 172.21.15.31:0/2830251925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaf81083a0 0x7fdaf81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 --2- 172.21.15.31:0/2830251925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8102a60 0x7fdaf8102e70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.198+0000 7fdaff317640 1 -- 172.21.15.31:0/2830251925 >> 172.21.15.31:0/2830251925 conn(0x7fdaf8079730 msgr2=0x7fdaf8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:23.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.199+0000 7fdaff317640 1 -- 172.21.15.31:0/2830251925 shutdown_connections 2024-09-21T06:21:23.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.199+0000 7fdaff317640 1 -- 172.21.15.31:0/2830251925 wait complete. 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.199+0000 7fdaff317640 1 Processor -- start 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.199+0000 7fdaff317640 1 -- start start 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdaff317640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaf8102a60 0x7fdaf8104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdaff317640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf81083a0 0x7fdaf8104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdaff317640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 0x7fdaf8104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdaff317640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112a40 con 0x7fdaf8108cf0 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdaff317640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81128c0 con 0x7fdaf8102a60 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdaff317640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112bc0 con 0x7fdaf81083a0 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 0x7fdaf8104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 0x7fdaf8104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55430/0 (socket says 172.21.15.31:55430) 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 -- 172.21.15.31:0/893185811 learned_addr learned my addr 172.21.15.31:0/893185811 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:23.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaf8102a60 0x7fdaf8104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf81083a0 0x7fdaf8104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 -- 172.21.15.31:0/893185811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf81083a0 msgr2=0x7fdaf8104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf81083a0 0x7fdaf8104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 -- 172.21.15.31:0/893185811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaf8102a60 msgr2=0x7fdaf8104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaf8102a60 0x7fdaf8104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.200+0000 7fdafd8b4640 1 -- 172.21.15.31:0/893185811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a87e0 con 0x7fdaf8108cf0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.201+0000 7fdafd8b4640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 0x7fdaf8104cd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b970 tx=0x7fdaf000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.201+0000 7fdaee7fc640 1 -- 172.21.15.31:0/893185811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0019070 con 0x7fdaf8108cf0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.201+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a8ad0 con 0x7fdaf8108cf0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.201+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a9010 con 0x7fdaf8108cf0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.201+0000 7fdaee7fc640 1 -- 172.21.15.31:0/893185811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf0014440 con 0x7fdaf8108cf0 2024-09-21T06:21:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.201+0000 7fdaee7fc640 1 -- 172.21.15.31:0/893185811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00075d0 con 0x7fdaf8108cf0 2024-09-21T06:21:23.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.205+0000 7fdb0588b740 1 Processor -- start 2024-09-21T06:21:23.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.206+0000 7fdaee7fc640 1 -- 172.21.15.31:0/893185811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdaf001c070 con 0x7fdaf8108cf0 2024-09-21T06:21:23.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.206+0000 7fdb0588b740 1 -- start start 2024-09-21T06:21:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdb0588b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618dd330ea0 con 0x5618dcfea1b0 2024-09-21T06:21:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdb0588b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618dd3202d0 con 0x5618dcff2390 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdb0588b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618dd3240e0 con 0x5618dd013070 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55440/0 (socket says 172.21.15.31:55440) 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 -- 172.21.15.31:0/1125647866 learned_addr learned my addr 172.21.15.31:0/1125647866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 -- 172.21.15.31:0/1125647866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 msgr2=0x5618dd3ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdaee7fc640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdae40b6700 0x7fdae40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 -- 172.21.15.31:0/1125647866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 msgr2=0x5618dd3bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.207+0000 7fdafd0b3640 1 -- 172.21.15.31:0/1125647866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd07e650 con 0x5618dcfea1b0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.208+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ba800 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402a960 tx=0x7fdaf4032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01591a4cfc5f872 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.208+0000 7fdaee7fc640 1 -- 172.21.15.31:0/893185811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7fdaf00e6810 con 0x7fdaf8108cf0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.208+0000 7fdafd8b4640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdae40b6700 0x7fdae40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.208+0000 7fdae37fe640 1 -- 172.21.15.31:0/1125647866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4035070 con 0x5618dcfea1b0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.208+0000 7fdafd8b4640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdae40b6700 0x7fdae40b8c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00e4020 tx=0x7fdaf0015fc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.208+0000 7fdae37fe640 1 -- 172.21.15.31:0/1125647866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf40316f0 con 0x5618dcfea1b0 2024-09-21T06:21:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.209+0000 7fdae37fe640 1 -- 172.21.15.31:0/1125647866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4005000 con 0x5618dcfea1b0 2024-09-21T06:21:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.209+0000 7fdb0588b740 1 -- 172.21.15.31:0/1125647866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 msgr2=0x5618dd3ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.209+0000 7fdb0588b740 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ba800 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402a960 tx=0x7fdaf4032ed0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 -- 172.21.15.31:0/1125647866 shutdown_connections 2024-09-21T06:21:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ba800 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 --2- 172.21.15.31:0/1125647866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 -- 172.21.15.31:0/1125647866 >> 172.21.15.31:0/1125647866 conn(0x5618dd007460 msgr2=0x5618dd3b81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 -- 172.21.15.31:0/1125647866 shutdown_connections 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 -- 172.21.15.31:0/1125647866 wait complete. 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 -- start start 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.210+0000 7fdb0588b740 1 Processor -- start 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5618dd181700 con 0x5618dd013070 2024-09-21T06:21:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5618dd2bb160 con 0x5618dcff2390 2024-09-21T06:21:23.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5618dd3c8a90 con 0x5618dcfea1b0 2024-09-21T06:21:23.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.211+0000 7fdb0588b740 10 client.0 _ll_register_callbacks cb 0x5618dd2d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:23.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.212+0000 7fdafc8b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.217+0000 7fdafc8b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33910/0 (socket says 172.21.15.31:33910) 2024-09-21T06:21:23.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.217+0000 7fdafc8b2640 1 -- 172.21.15.31:0/2481252107 learned_addr learned my addr 172.21.15.31:0/2481252107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:23.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.217+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.218+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.219+0000 7fdafc8b2640 1 -- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 msgr2=0x5618dd3c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.219+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.219+0000 7fdafc8b2640 1 -- 172.21.15.31:0/2481252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 msgr2=0x5618dd3c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.219+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.219+0000 7fdafc8b2640 1 -- 172.21.15.31:0/2481252107 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd2e8c90 con 0x5618dcfea1b0 2024-09-21T06:21:23.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.222+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c5880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdae801ada0 tx=0x7fdae8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa6fb6146ea0946 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:23.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.222+0000 7fdae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:21:23.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.222+0000 7fdae0ff9640 1 -- 172.21.15.31:0/2481252107 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8017e80 con 0x5618dcfea1b0 2024-09-21T06:21:23.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.223+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf40316f0 con 0x5618dcfea1b0 2024-09-21T06:21:23.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.223+0000 7fdae0ff9640 1 -- 172.21.15.31:0/2481252107 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7fdae801a0d0 con 0x5618dcfea1b0 2024-09-21T06:21:23.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.223+0000 7fdae0ff9640 1 -- 172.21.15.31:0/2481252107 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdae802b050 con 0x5618dcfea1b0 2024-09-21T06:21:23.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.225+0000 7fdb0588b740 10 client.27277 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:23.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.225+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5618dd3d4750 con 0x5618dcfea1b0 2024-09-21T06:21:23.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.226+0000 7fdae0ff9640 1 -- 172.21.15.31:0/2481252107 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf40316f0 con 0x5618dcfea1b0 2024-09-21T06:21:23.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdae0ff9640 1 -- 172.21.15.31:0/2481252107 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdae8026420 con 0x5618dcfea1b0 2024-09-21T06:21:23.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdb0588b740 10 client.27277 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdb0588b740 10 client.27277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdb0588b740 20 client.27277 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdb0588b740 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfd19f0 0x5618dd3d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdb0588b740 4 client.27277 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:23.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.228+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618dd332df0 con 0x5618dcfd19f0 2024-09-21T06:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.232+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfd19f0 0x5618dd3d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.233+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfd19f0 0x5618dd3d8020 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd0d376e696775d server_cookie=eea8c62f5a375f2c in_seq=0 out_seq=0 2024-09-21T06:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.233+0000 7fdae0ff9640 10 client.27277 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.234+0000 7fdae0ff9640 1 -- 172.21.15.31:0/2481252107 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618dd332df0 con 0x5618dcfd19f0 2024-09-21T06:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.234+0000 7fdae0ff9640 10 client.27277 handle_command_reply: tid=0 2024-09-21T06:21:23.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.234+0000 7fdb0588b740 1 client.27277 shutdown 2024-09-21T06:21:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.235+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 msgr2=0x5618dd3c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.235+0000 7fdb0588b740 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c5880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdae801ada0 tx=0x7fdae8008040 comp rx=0 tx=0).stop 2024-09-21T06:21:23.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.237+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 shutdown_connections 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.237+0000 7fdb0588b740 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfd19f0 0x5618dd3d8020 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.237+0000 7fdb0588b740 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.237+0000 7fdb0588b740 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.237+0000 7fdb0588b740 1 --2- 172.21.15.31:0/2481252107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c5880 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.237+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 >> 172.21.15.31:0/2481252107 conn(0x5618dd007460 msgr2=0x5618dd3bbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.238+0000 7fdae0ff9640 0 client.27277 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.238+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 shutdown_connections 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.238+0000 7fdb0588b740 1 -- 172.21.15.31:0/2481252107 wait complete. 2024-09-21T06:21:23.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.238+0000 7fdb0588b740 20 client.27277 trim_cache size 0 max 16384 2024-09-21T06:21:23.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.263+0000 7fdb0588b740 1 Processor -- start 2024-09-21T06:21:23.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 -- start start 2024-09-21T06:21:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc00c6c0 con 0x5618dd013070 2024-09-21T06:21:23.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc00ee90 con 0x5618dcff2390 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdb0588b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc00cc80 con 0x5618dcfea1b0 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdafc8b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdafc8b2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33912/0 (socket says 172.21.15.31:33912) 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdafc8b2640 1 -- 172.21.15.31:0/3005469558 learned_addr learned my addr 172.21.15.31:0/3005469558 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.264+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.265+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.265+0000 7fdafd0b3640 1 -- 172.21.15.31:0/3005469558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 msgr2=0x5618dd3c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.265+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.265+0000 7fdafd0b3640 1 -- 172.21.15.31:0/3005469558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 msgr2=0x5618dd3dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.265+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.265+0000 7fdafd0b3640 1 -- 172.21.15.31:0/3005469558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd3bcb00 con 0x5618dcff2390 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.266+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c31c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005f70 tx=0x7fdaf402a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44644a431700971 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.266+0000 7fdae2ffd640 1 -- 172.21.15.31:0/3005469558 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4035040 con 0x5618dcff2390 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.266+0000 7fdae2ffd640 1 -- 172.21.15.31:0/3005469558 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf4053020 con 0x5618dcff2390 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.266+0000 7fdae2ffd640 1 -- 172.21.15.31:0/3005469558 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4004b70 con 0x5618dcff2390 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 -- 172.21.15.31:0/3005469558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 msgr2=0x5618dd3c31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c31c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005f70 tx=0x7fdaf402a440 comp rx=0 tx=0).stop 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 -- 172.21.15.31:0/3005469558 shutdown_connections 2024-09-21T06:21:23.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dd013070 0x5618dd3dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c31c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 --2- 172.21.15.31:0/3005469558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dcfea1b0 0x5618dd3c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 -- 172.21.15.31:0/3005469558 >> 172.21.15.31:0/3005469558 conn(0x5618dd007460 msgr2=0x5618dd3c4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 -- 172.21.15.31:0/3005469558 shutdown_connections 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.267+0000 7fdb0588b740 1 -- 172.21.15.31:0/3005469558 wait complete. 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 -- start start 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 Processor -- start 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc00d350 con 0x5618dcfea1b0 2024-09-21T06:21:23.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc00e7c0 con 0x5618dcff2390 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc00da20 con 0x5618dd013070 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.268+0000 7fdb0588b740 10 client.0 _ll_register_callbacks cb 0x7fdae8009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafc8b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafc8b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55476/0 (socket says 172.21.15.31:55476) 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafc8b2640 1 -- 172.21.15.31:0/924957393 learned_addr learned my addr 172.21.15.31:0/924957393 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 -- 172.21.15.31:0/924957393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 msgr2=0x5618dd3ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 -- 172.21.15.31:0/924957393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 msgr2=0x5618dd3ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 -- 172.21.15.31:0/924957393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd441a20 con 0x5618dcff2390 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdafd0b3640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c9be0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5618dd2c0710 tx=0x7fdaf402ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232f3fe0d958ffc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdae0ff9640 1 -- 172.21.15.31:0/924957393 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4035040 con 0x5618dcff2390 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.269+0000 7fdae0ff9640 1 -- 172.21.15.31:0/924957393 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7fdaf404a070 con 0x5618dcff2390 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.270+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf4053020 con 0x5618dcff2390 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.270+0000 7fdae0ff9640 1 -- 172.21.15.31:0/924957393 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdaf40316f0 con 0x5618dcff2390 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.270+0000 7fdb0588b740 10 client.27251 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:23.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.270+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5618dd442b40 con 0x5618dcff2390 2024-09-21T06:21:23.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdae0ff9640 1 -- 172.21.15.31:0/924957393 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf4053020 con 0x5618dcff2390 2024-09-21T06:21:23.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdae0ff9640 1 -- 172.21.15.31:0/924957393 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fdaf405f030 con 0x5618dcff2390 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdb0588b740 10 client.27251 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdb0588b740 10 client.27251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdb0588b740 20 client.27251 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdb0588b740 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfa6da0 0x5618dd444e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdb0588b740 4 client.27251 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.272+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618dd32e8a0 con 0x5618dcfa6da0 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.273+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfa6da0 0x5618dd444e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.273+0000 7fdae0ff9640 10 client.27251 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.273+0000 7fdafc8b2640 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfa6da0 0x5618dd444e20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28fbb00f23d93eac server_cookie=3673ab276e056b12 in_seq=0 out_seq=0 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.273+0000 7fdae0ff9640 1 -- 172.21.15.31:0/924957393 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618dd32e8a0 con 0x5618dcfa6da0 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.273+0000 7fdae0ff9640 10 client.27251 handle_command_reply: tid=0 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 client.27251 shutdown 2024-09-21T06:21:23.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 msgr2=0x5618dd3c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c9be0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x5618dd2c0710 tx=0x7fdaf402ab80 comp rx=0 tx=0).stop 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 shutdown_connections 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5618dcfa6da0 0x5618dd444e20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618dd013070 0x5618dd3ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618dcff2390 0x5618dd3c9be0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 --2- 172.21.15.31:0/924957393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5618dcfea1b0 0x5618dd3ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 >> 172.21.15.31:0/924957393 conn(0x5618dd007460 msgr2=0x5618dd3c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdae0ff9640 0 client.27251 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 shutdown_connections 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 1 -- 172.21.15.31:0/924957393 wait complete. 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.274+0000 7fdb0588b740 20 client.27251 trim_cache size 0 max 16384 2024-09-21T06:21:23.280 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:21:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdae40b6700 msgr2=0x7fdae40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdae40b6700 0x7fdae40b8c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00e4020 tx=0x7fdaf0015fc0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 msgr2=0x7fdaf8104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 0x7fdaf8104cd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b970 tx=0x7fdaf000bd70 comp rx=0 tx=0).stop 2024-09-21T06:21:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 shutdown_connections 2024-09-21T06:21:23.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdae40b6700 0x7fdae40b8c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaf8108cf0 0x7fdaf8104cd0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdaf81083a0 0x7fdaf8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 --2- 172.21.15.31:0/893185811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaf8102a60 0x7fdaf8104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 >> 172.21.15.31:0/893185811 conn(0x7fdaf8079730 msgr2=0x7fdaf8101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 shutdown_connections 2024-09-21T06:21:23.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:23.275+0000 7fdaff317640 1 -- 172.21.15.31:0/893185811 wait complete. 2024-09-21T06:21:25.071 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:21:25.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:21:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 585.0 s, 1773.6 tps, lat 56.622 ms stddev 69.705 2024-09-21T06:21:25.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:24 smithi110 ceph-mon[34550]: pgmap v622: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 833 op/s 2024-09-21T06:21:25.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:24 smithi107 ceph-mon[34380]: pgmap v622: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 833 op/s 2024-09-21T06:21:25.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:24 smithi031 ceph-mon[31237]: pgmap v622: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 18 MiB/s wr, 833 op/s 2024-09-21T06:21:25.508 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 -- 172.21.15.31:0/2632053289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38106280 msgr2=0x7f1b38106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 --2- 172.21.15.31:0/2632053289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38106280 0x7f1b38106690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009930 tx=0x7f1b2402a440 comp rx=0 tx=0).stop 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 -- 172.21.15.31:0/2632053289 shutdown_connections 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 --2- 172.21.15.31:0/2632053289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106bd0 0x7f1b38110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 --2- 172.21.15.31:0/2632053289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38106280 0x7f1b38106690 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 --2- 172.21.15.31:0/2632053289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38102030 0x7f1b38102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 -- 172.21.15.31:0/2632053289 >> 172.21.15.31:0/2632053289 conn(0x7f1b38077c60 msgr2=0x7f1b380780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.108+0000 7f1b3f258640 1 -- 172.21.15.31:0/2632053289 shutdown_connections 2024-09-21T06:21:26.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.109+0000 7f1b3f258640 1 -- 172.21.15.31:0/2632053289 wait complete. 2024-09-21T06:21:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.110+0000 7f1b3f258640 1 Processor -- start 2024-09-21T06:21:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.110+0000 7f1b3f258640 1 -- start start 2024-09-21T06:21:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.110+0000 7f1b3f258640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38102030 0x7f1b381a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3f258640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106280 0x7f1b381a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3cff4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38102030 0x7f1b381a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3cff4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38102030 0x7f1b381a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33950/0 (socket says 172.21.15.31:33950) 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106280 0x7f1b381a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3f258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38106bd0 0x7f1b3810fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106280 0x7f1b381a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55484/0 (socket says 172.21.15.31:55484) 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3cff4640 1 -- 172.21.15.31:0/4060606824 learned_addr learned my addr 172.21.15.31:0/4060606824 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38113af0 con 0x7f1b38106280 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38106bd0 0x7f1b3810fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38113970 con 0x7f1b38106bd0 2024-09-21T06:21:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.111+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38113c70 con 0x7f1b38102030 2024-09-21T06:21:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b3d7f5640 1 -- 172.21.15.31:0/4060606824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38102030 msgr2=0x7f1b381a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38102030 0x7f1b381a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b3d7f5640 1 -- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106280 msgr2=0x7f1b381a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106280 0x7f1b381a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b3d7f5640 1 -- 172.21.15.31:0/4060606824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b38110480 con 0x7f1b38106bd0 2024-09-21T06:21:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38106bd0 0x7f1b3810fcb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009fd0 tx=0x7f1b3000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b35ffb640 1 -- 172.21.15.31:0/4060606824 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000ce20 con 0x7f1b38106bd0 2024-09-21T06:21:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b35ffb640 1 -- 172.21.15.31:0/4060606824 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b30005020 con 0x7f1b38106bd0 2024-09-21T06:21:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.112+0000 7f1b35ffb640 1 -- 172.21.15.31:0/4060606824 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30010690 con 0x7f1b38106bd0 2024-09-21T06:21:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.114+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b38110770 con 0x7f1b38106bd0 2024-09-21T06:21:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.114+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3819a860 con 0x7f1b38106bd0 2024-09-21T06:21:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.117+0000 7f1b35ffb640 1 -- 172.21.15.31:0/4060606824 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1b30010830 con 0x7f1b38106bd0 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 Processor -- start 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 -- start start 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 0x562a421dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e0d490 0x562a421dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421ddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562a42153e60 con 0x562a41e0d490 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a42143290 con 0x562a41e36350 2024-09-21T06:21:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.118+0000 7f1b457cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a421470a0 con 0x562a41e15670 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 0x562a421dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 0x562a421dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34826/0 (socket says 172.21.15.31:34826) 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 -- 172.21.15.31:0/1524318112 learned_addr learned my addr 172.21.15.31:0/1524318112 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421ddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e0d490 0x562a421dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 -- 172.21.15.31:0/1524318112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 msgr2=0x562a421ddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421ddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 -- 172.21.15.31:0/1524318112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e0d490 msgr2=0x562a421dd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e0d490 0x562a421dd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 -- 172.21.15.31:0/1524318112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a41ea1930 con 0x562a41e36350 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e0d490 0x562a421dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 0x562a421dd430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b38075ab0 tx=0x7f1b24000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19175e1835e817f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b35ffb640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b280b6700 0x7f1b280b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b22ffd640 1 -- 172.21.15.31:0/1524318112 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24035030 con 0x562a41e36350 2024-09-21T06:21:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.119+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b280b6700 0x7f1b280b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.120+0000 7f1b22ffd640 1 -- 172.21.15.31:0/1524318112 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b240054d0 con 0x562a41e36350 2024-09-21T06:21:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.120+0000 7f1b22ffd640 1 -- 172.21.15.31:0/1524318112 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24004cd0 con 0x562a41e36350 2024-09-21T06:21:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.120+0000 7f1b35ffb640 1 -- 172.21.15.31:0/4060606824 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f1b300e7dc0 con 0x7f1b38106bd0 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b280b6700 0x7f1b280b8c10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x562a421dc0b0 tx=0x7f1b2c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 -- 172.21.15.31:0/1524318112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 msgr2=0x562a421dd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 0x562a421dd430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b38075ab0 tx=0x7f1b24000bc0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 -- 172.21.15.31:0/1524318112 shutdown_connections 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421ddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e0d490 0x562a421dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1524318112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e36350 0x562a421dd430 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 -- 172.21.15.31:0/1524318112 >> 172.21.15.31:0/1524318112 conn(0x562a41e2a740 msgr2=0x562a421db320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 -- 172.21.15.31:0/1524318112 shutdown_connections 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 -- 172.21.15.31:0/1524318112 wait complete. 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.121+0000 7f1b457cc740 1 -- start start 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.122+0000 7f1b457cc740 1 Processor -- start 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e0d490 0x562a421e9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 0x562a421e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562a41fa4890 con 0x562a41e36350 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562a420ddd40 con 0x562a41e15670 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a421ebbb0 con 0x562a41e0d490 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 10 client.0 _ll_register_callbacks cb 0x562a42141c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b3d7f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 0x562a421e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b3d7f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55504/0 (socket says 172.21.15.31:55504) 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b3d7f5640 1 -- 172.21.15.31:0/1313943909 learned_addr learned my addr 172.21.15.31:0/1313943909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 0x562a421e95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34830/0 (socket says 172.21.15.31:34830) 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e0d490 0x562a421e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 -- 172.21.15.31:0/1313943909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e0d490 msgr2=0x562a421e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e0d490 0x562a421e9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 -- 172.21.15.31:0/1313943909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 msgr2=0x562a421f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 -- 172.21.15.31:0/1313943909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a421f54f0 con 0x562a41e15670 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e0d490 0x562a421e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b37fff640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 0x562a421e95d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009850 tx=0x7f1b24005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6774213454bdc7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b17fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b17fff640 1 -- 172.21.15.31:0/1313943909 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24035040 con 0x562a41e15670 2024-09-21T06:21:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b240054d0 con 0x562a41e15670 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b17fff640 1 -- 172.21.15.31:0/1313943909 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f1b24005230 con 0x562a41e15670 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b17fff640 1 -- 172.21.15.31:0/1313943909 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1b24057050 con 0x562a41e15670 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 10 client.27263 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a421f7d90 con 0x562a41e15670 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.124+0000 7f1b17fff640 1 -- 172.21.15.31:0/1313943909 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b240054d0 con 0x562a41e15670 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b17fff640 1 -- 172.21.15.31:0/1313943909 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1b240041c0 con 0x562a41e15670 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b457cc740 10 client.27263 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b457cc740 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b457cc740 20 client.27263 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41df4cd0 0x562a421fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b457cc740 4 client.27263 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a42155db0 con 0x562a41df4cd0 2024-09-21T06:21:26.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.125+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41df4cd0 0x562a421fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41df4cd0 0x562a421fb0f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27713ea7d9cc7eac server_cookie=9223ae9500a91784 in_seq=0 out_seq=0 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b17fff640 10 client.27263 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b17fff640 1 -- 172.21.15.31:0/1313943909 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a42155db0 con 0x562a41df4cd0 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b17fff640 10 client.27263 handle_command_reply: tid=0 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b457cc740 1 client.27263 shutdown 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 msgr2=0x562a421e95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 0x562a421e95d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009850 tx=0x7f1b24005850 comp rx=0 tx=0).stop 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 shutdown_connections 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41df4cd0 0x562a421fb0f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.126+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e15670 0x562a421e95d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b457cc740 1 --2- 172.21.15.31:0/1313943909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e0d490 0x562a421e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b17fff640 0 client.27263 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 >> 172.21.15.31:0/1313943909 conn(0x562a41e2a740 msgr2=0x562a421e05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 shutdown_connections 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b457cc740 1 -- 172.21.15.31:0/1313943909 wait complete. 2024-09-21T06:21:26.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.127+0000 7f1b457cc740 20 client.27263 trim_cache size 0 max 16384 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.150+0000 7f1b457cc740 1 Processor -- start 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.150+0000 7f1b457cc740 1 -- start start 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b457cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381bc400 con 0x562a41e36350 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b457cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381bec60 con 0x562a41e0d490 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b457cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381bca50 con 0x562a41e15670 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421e2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34838/0 (socket says 172.21.15.31:34838) 2024-09-21T06:21:26.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 -- 172.21.15.31:0/2566105050 learned_addr learned my addr 172.21.15.31:0/2566105050 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b37fff640 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 -- 172.21.15.31:0/2566105050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 msgr2=0x562a421deb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421deb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 -- 172.21.15.31:0/2566105050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 msgr2=0x562a421e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.151+0000 7f1b3cff4640 1 -- 172.21.15.31:0/2566105050 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a421ea130 con 0x562a41e0d490 2024-09-21T06:21:26.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.152+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421e2bb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x562a421ebcb0 tx=0x7f1b2c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20114175997cb0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.152+0000 7f1b227fc640 1 -- 172.21.15.31:0/2566105050 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c02f070 con 0x562a41e0d490 2024-09-21T06:21:26.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.153+0000 7f1b227fc640 1 -- 172.21.15.31:0/2566105050 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c0220d0 con 0x562a41e0d490 2024-09-21T06:21:26.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.153+0000 7f1b227fc640 1 -- 172.21.15.31:0/2566105050 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c020850 con 0x562a41e0d490 2024-09-21T06:21:26.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 -- 172.21.15.31:0/2566105050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 msgr2=0x562a421e2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421e2bb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x562a421ebcb0 tx=0x7f1b2c016000 comp rx=0 tx=0).stop 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 -- 172.21.15.31:0/2566105050 shutdown_connections 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e36350 0x562a421e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e15670 0x562a421deb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 --2- 172.21.15.31:0/2566105050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421e2bb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 -- 172.21.15.31:0/2566105050 >> 172.21.15.31:0/2566105050 conn(0x562a41e2a740 msgr2=0x562a421de460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 -- 172.21.15.31:0/2566105050 shutdown_connections 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 -- 172.21.15.31:0/2566105050 wait complete. 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 -- start start 2024-09-21T06:21:26.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.154+0000 7f1b457cc740 1 Processor -- start 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421fc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e15670 0x562a421fc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e36350 0x562a421fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381bd120 con 0x562a41e15670 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381be590 con 0x562a41e0d490 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b3cff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381bd7f0 con 0x562a41e36350 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b3cff4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34846/0 (socket says 172.21.15.31:34846) 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b3cff4640 1 -- 172.21.15.31:0/872754121 learned_addr learned my addr 172.21.15.31:0/872754121 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b457cc740 10 client.0 _ll_register_callbacks cb 0x7f1b24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b37fff640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e15670 0x562a421fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.155+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e36350 0x562a421fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b37fff640 1 -- 172.21.15.31:0/872754121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e36350 msgr2=0x562a421fcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b37fff640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e36350 0x562a421fcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b37fff640 1 -- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 msgr2=0x562a421fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b37fff640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421fc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b37fff640 1 -- 172.21.15.31:0/872754121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a42264870 con 0x562a41e15670 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b3d7f5640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e36350 0x562a421fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b37fff640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e15670 0x562a421fc650 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1b240036e0 tx=0x7f1b24005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98c0f408e4dd9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b17fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.156+0000 7f1b17fff640 1 -- 172.21.15.31:0/872754121 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24060060 con 0x562a41e15670 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.157+0000 7f1b17fff640 1 -- 172.21.15.31:0/872754121 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f1b24005230 con 0x562a41e15670 2024-09-21T06:21:26.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.157+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c0220d0 con 0x562a41e15670 2024-09-21T06:21:26.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.157+0000 7f1b17fff640 1 -- 172.21.15.31:0/872754121 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1b24035040 con 0x562a41e15670 2024-09-21T06:21:26.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.158+0000 7f1b457cc740 10 client.17469 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:26.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.158+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a42265b60 con 0x562a41e15670 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b17fff640 1 -- 172.21.15.31:0/872754121 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c0220d0 con 0x562a41e15670 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b17fff640 1 -- 172.21.15.31:0/872754121 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1b24003c00 con 0x562a41e15670 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b457cc740 10 client.17469 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b457cc740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b457cc740 20 client.17469 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b457cc740 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41dca080 0x562a42268320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b457cc740 4 client.17469 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.159+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a42151860 con 0x562a41dca080 2024-09-21T06:21:26.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.160+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41dca080 0x562a42268320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.160+0000 7f1b17fff640 10 client.17469 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.160+0000 7f1b3cff4640 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41dca080 0x562a42268320 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0319767aed37285 server_cookie=f2f9e4d79c5a3829 in_seq=0 out_seq=0 2024-09-21T06:21:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b17fff640 1 -- 172.21.15.31:0/872754121 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a42151860 con 0x562a41dca080 2024-09-21T06:21:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b17fff640 10 client.17469 handle_command_reply: tid=0 2024-09-21T06:21:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 client.17469 shutdown 2024-09-21T06:21:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e15670 msgr2=0x562a421fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e15670 0x562a421fc650 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f1b240036e0 tx=0x7f1b24005e30 comp rx=0 tx=0).stop 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 shutdown_connections 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562a41dca080 0x562a42268320 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b17fff640 0 client.17469 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a41e36350 0x562a421fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562a41e15670 0x562a421fc650 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 --2- 172.21.15.31:0/872754121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562a41e0d490 0x562a421fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 >> 172.21.15.31:0/872754121 conn(0x562a41e2a740 msgr2=0x562a421de3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 shutdown_connections 2024-09-21T06:21:26.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.162+0000 7f1b457cc740 1 -- 172.21.15.31:0/872754121 wait complete. 2024-09-21T06:21:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.163+0000 7f1b457cc740 20 client.17469 trim_cache size 0 max 16384 2024-09-21T06:21:26.167 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:21:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:21:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "71aa8c73-a180-4b01-a247-118653b8cea4", 2024-09-21T06:21:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:21:26.167 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:21:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.164+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b280b6700 msgr2=0x7f1b280b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.164+0000 7f1b3f258640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b280b6700 0x7f1b280b8c10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x562a421dc0b0 tx=0x7f1b2c005f90 comp rx=0 tx=0).stop 2024-09-21T06:21:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.164+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38106bd0 msgr2=0x7f1b3810fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.164+0000 7f1b3f258640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38106bd0 0x7f1b3810fcb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009fd0 tx=0x7f1b3000ef10 comp rx=0 tx=0).stop 2024-09-21T06:21:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 shutdown_connections 2024-09-21T06:21:26.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b38106bd0 0x7f1b3810fcb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1b280b6700 0x7f1b280b8c10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b38106280 0x7f1b381a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 --2- 172.21.15.31:0/4060606824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b38102030 0x7f1b381a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 >> 172.21.15.31:0/4060606824 conn(0x7f1b38077c60 msgr2=0x7f1b38109410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 shutdown_connections 2024-09-21T06:21:26.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:26.165+0000 7f1b3f258640 1 -- 172.21.15.31:0/4060606824 wait complete. 2024-09-21T06:21:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.733+0000 7f4c8d178640 1 -- 172.21.15.31:0/1881891387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 msgr2=0x7f4c8009a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.733+0000 7f4c8d178640 1 --2- 172.21.15.31:0/1881891387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 0x7f4c8009a6d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4c88066480 tx=0x7f4c880980a0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.736+0000 7f4c8d178640 1 -- 172.21.15.31:0/1881891387 shutdown_connections 2024-09-21T06:21:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.736+0000 7f4c8d178640 1 --2- 172.21.15.31:0/1881891387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 0x7f4c8009a6d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.736+0000 7f4c8d178640 1 --2- 172.21.15.31:0/1881891387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c800956e0 0x7f4c80099d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.736+0000 7f4c8d178640 1 --2- 172.21.15.31:0/1881891387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80094cb0 0x7f4c800950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.736+0000 7f4c8d178640 1 -- 172.21.15.31:0/1881891387 >> 172.21.15.31:0/1881891387 conn(0x7f4c8000cd80 msgr2=0x7f4c8000d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.736+0000 7f4c8d178640 1 -- 172.21.15.31:0/1881891387 shutdown_connections 2024-09-21T06:21:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.738+0000 7f4c8d178640 1 -- 172.21.15.31:0/1881891387 wait complete. 2024-09-21T06:21:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 Processor -- start 2024-09-21T06:21:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 -- start start 2024-09-21T06:21:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c80094cb0 0x7f4c80133170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 0x7f4c80133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 0x7f4c801389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c800ac6b0 con 0x7f4c8009a240 2024-09-21T06:21:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c800ac530 con 0x7f4c80094cb0 2024-09-21T06:21:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.739+0000 7f4c8d178640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c800ac830 con 0x7f4c800956e0 2024-09-21T06:21:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 0x7f4c80133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 0x7f4c80133710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33988/0 (socket says 172.21.15.31:33988) 2024-09-21T06:21:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 -- 172.21.15.31:0/3218078172 learned_addr learned my addr 172.21.15.31:0/3218078172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c8c977640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 0x7f4c801389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 -- 172.21.15.31:0/3218078172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c80094cb0 msgr2=0x7f4c80133170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c80094cb0 0x7f4c80133170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 -- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 msgr2=0x7f4c801389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 0x7f4c801389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.740+0000 7f4c877fe640 1 -- 172.21.15.31:0/3218078172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c801390a0 con 0x7f4c800956e0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.741+0000 7f4c877fe640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 0x7f4c80133710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00b450 tx=0x7f4c7c00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.741+0000 7f4c857fa640 1 -- 172.21.15.31:0/3218078172 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c015020 con 0x7f4c800956e0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.741+0000 7f4c857fa640 1 -- 172.21.15.31:0/3218078172 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7c011ce0 con 0x7f4c800956e0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.741+0000 7f4c857fa640 1 -- 172.21.15.31:0/3218078172 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c010de0 con 0x7f4c800956e0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.742+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c80139390 con 0x7f4c800956e0 2024-09-21T06:21:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.742+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8013fb20 con 0x7f4c800956e0 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.745+0000 7f4c9541f740 1 Processor -- start 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.745+0000 7f4c9541f740 1 -- start start 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.745+0000 7f4c857fa640 1 -- 172.21.15.31:0/3218078172 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4c7c018020 con 0x7f4c800956e0 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 0x5588d453eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d453f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d453f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c9541f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5588d44b36f0 con 0x5588d41784d0 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c9541f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5588d44a2b20 con 0x5588d41991b0 2024-09-21T06:21:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c9541f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5588d44a6930 con 0x5588d41702f0 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c87fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d453f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 0x5588d453eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c877fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d453f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c87fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d453f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34002/0 (socket says 172.21.15.31:34002) 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 0x5588d453eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34856/0 (socket says 172.21.15.31:34856) 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c87fff640 1 -- 172.21.15.31:0/155817223 learned_addr learned my addr 172.21.15.31:0/155817223 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 -- 172.21.15.31:0/155817223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 msgr2=0x5588d453f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d453f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 -- 172.21.15.31:0/155817223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 msgr2=0x5588d453f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d453f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c8c977640 1 -- 172.21.15.31:0/155817223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d4468530 con 0x5588d41991b0 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.746+0000 7f4c877fe640 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d453f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.747+0000 7f4c857fa640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b66b0 0x7f4c780b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.747+0000 7f4c8c977640 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 0x5588d453eba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4c880521d0 tx=0x7f4c880694a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef028be0e4545957 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.747+0000 7f4c87fff640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b66b0 0x7f4c780b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.747+0000 7f4c857fa640 1 -- 172.21.15.31:0/3218078172 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f4c7c0e6e30 con 0x7f4c800956e0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.747+0000 7f4c87fff640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b66b0 0x7f4c780b8bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5588d453d820 tx=0x7f4c74009190 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.748+0000 7f4c727fc640 1 -- 172.21.15.31:0/155817223 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880b0070 con 0x5588d41991b0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.748+0000 7f4c727fc640 1 -- 172.21.15.31:0/155817223 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c8809c050 con 0x5588d41991b0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.748+0000 7f4c727fc640 1 -- 172.21.15.31:0/155817223 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880aba00 con 0x5588d41991b0 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 -- 172.21.15.31:0/155817223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 msgr2=0x5588d453eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 0x5588d453eba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4c880521d0 tx=0x7f4c880694a0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 -- 172.21.15.31:0/155817223 shutdown_connections 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d453f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d453f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 --2- 172.21.15.31:0/155817223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41991b0 0x5588d453eba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 -- 172.21.15.31:0/155817223 >> 172.21.15.31:0/155817223 conn(0x5588d418d300 msgr2=0x5588d453ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 -- 172.21.15.31:0/155817223 shutdown_connections 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.749+0000 7f4c9541f740 1 -- 172.21.15.31:0/155817223 wait complete. 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 -- start start 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 Processor -- start 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41784d0 0x5588d454a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41991b0 0x5588d4556670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5588d43daab0 con 0x5588d41991b0 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5588d4448730 con 0x5588d41702f0 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5588d454d320 con 0x5588d41784d0 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c9541f740 10 client.0 _ll_register_callbacks cb 0x5588d449c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c877fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41784d0 0x5588d454a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41991b0 0x5588d4556670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34866/0 (socket says 172.21.15.31:34866) 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.750+0000 7f4c87fff640 1 -- 172.21.15.31:0/403732180 learned_addr learned my addr 172.21.15.31:0/403732180 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c877fe640 1 -- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 msgr2=0x5588d454a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c877fe640 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c877fe640 1 -- 172.21.15.31:0/403732180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41991b0 msgr2=0x5588d4556670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c877fe640 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41991b0 0x5588d4556670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c877fe640 1 -- 172.21.15.31:0/403732180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d4204790 con 0x5588d41784d0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c877fe640 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41784d0 0x5588d454a830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0e4590 tx=0x7f4c7c009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce677e279f7c1220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c677fe640 1 -- 172.21.15.31:0/403732180 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0a8b30 con 0x5588d41784d0 2024-09-21T06:21:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c677fe640 1 -- 172.21.15.31:0/403732180 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f4c7c0d00e0 con 0x5588d41784d0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.751+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c8809c050 con 0x5588d41784d0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c677fe640 1 -- 172.21.15.31:0/403732180 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c0a8400 con 0x5588d41784d0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 10 client.27310 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5588d427d620 con 0x5588d41784d0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c677fe640 1 -- 172.21.15.31:0/403732180 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c8809c050 con 0x5588d41784d0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c677fe640 1 -- 172.21.15.31:0/403732180 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4c7c0e4ce0 con 0x5588d41784d0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 10 client.27310 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 10 client.27310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 20 client.27310 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d4157d80 0x5588d455c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 4 client.27310 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.752+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588d44b5640 con 0x5588d4157d80 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.753+0000 7f4c87fff640 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d4157d80 0x5588d455c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.754+0000 7f4c87fff640 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d4157d80 0x5588d455c530 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7a454b58d06662 server_cookie=8d2a9058962d5f6d in_seq=0 out_seq=0 2024-09-21T06:21:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.754+0000 7f4c677fe640 10 client.27310 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.760+0000 7f4c677fe640 1 -- 172.21.15.31:0/403732180 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588d44b5640 con 0x5588d4157d80 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.760+0000 7f4c677fe640 10 client.27310 handle_command_reply: tid=0 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.760+0000 7f4c9541f740 1 client.27310 shutdown 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41784d0 msgr2=0x5588d454a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41784d0 0x5588d454a830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0e4590 tx=0x7f4c7c009390 comp rx=0 tx=0).stop 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 shutdown_connections 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d4157d80 0x5588d455c530 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c677fe640 0 client.27310 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41991b0 0x5588d4556670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41784d0 0x5588d454a830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 --2- 172.21.15.31:0/403732180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 >> 172.21.15.31:0/403732180 conn(0x5588d418d300 msgr2=0x5588d453fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 shutdown_connections 2024-09-21T06:21:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 1 -- 172.21.15.31:0/403732180 wait complete. 2024-09-21T06:21:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.761+0000 7f4c9541f740 20 client.27310 trim_cache size 0 max 16384 2024-09-21T06:21:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 Processor -- start 2024-09-21T06:21:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 -- start start 2024-09-21T06:21:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d4157d80 0x5588d454a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d45402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d4556570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80146380 con 0x5588d41784d0 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80148be0 con 0x5588d4157d80 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c9541f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801469d0 con 0x5588d41702f0 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d4556570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d4157d80 0x5588d454a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d4556570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55550/0 (socket says 172.21.15.31:55550) 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c8c977640 1 -- 172.21.15.31:0/844712543 learned_addr learned my addr 172.21.15.31:0/844712543 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.780+0000 7f4c877fe640 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d45402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c8c977640 1 -- 172.21.15.31:0/844712543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 msgr2=0x5588d45402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c8c977640 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d45402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c8c977640 1 -- 172.21.15.31:0/844712543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d4157d80 msgr2=0x5588d454a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c8c977640 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d4157d80 0x5588d454a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c8c977640 1 -- 172.21.15.31:0/844712543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d4536df0 con 0x5588d41784d0 2024-09-21T06:21:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c877fe640 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d45402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c8c977640 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d4556570 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4c88066480 tx=0x7f4c88098760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2813be21b03afcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c71ffb640 1 -- 172.21.15.31:0/844712543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880b0040 con 0x5588d41784d0 2024-09-21T06:21:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c71ffb640 1 -- 172.21.15.31:0/844712543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c880a1bc0 con 0x5588d41784d0 2024-09-21T06:21:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.781+0000 7f4c71ffb640 1 -- 172.21.15.31:0/844712543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880abb40 con 0x5588d41784d0 2024-09-21T06:21:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 -- 172.21.15.31:0/844712543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 msgr2=0x5588d4556570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d4556570 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4c88066480 tx=0x7f4c88098760 comp rx=0 tx=0).stop 2024-09-21T06:21:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 -- 172.21.15.31:0/844712543 shutdown_connections 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d4556570 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d41702f0 0x5588d45402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 --2- 172.21.15.31:0/844712543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d4157d80 0x5588d454a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 -- 172.21.15.31:0/844712543 >> 172.21.15.31:0/844712543 conn(0x5588d418d300 msgr2=0x5588d453fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 -- 172.21.15.31:0/844712543 shutdown_connections 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.782+0000 7f4c9541f740 1 -- 172.21.15.31:0/844712543 wait complete. 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 -- start start 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 Processor -- start 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d4157d80 0x5588d4552c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d454e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801470a0 con 0x5588d41784d0 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80148510 con 0x5588d41702f0 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80147770 con 0x5588d4157d80 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c9541f740 10 client.0 _ll_register_callbacks cb 0x7f4c7c010fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d454e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c8c977640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d454e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55556/0 (socket says 172.21.15.31:55556) 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c8c977640 1 -- 172.21.15.31:0/939898302 learned_addr learned my addr 172.21.15.31:0/939898302 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c877fe640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.783+0000 7f4c87fff640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d4157d80 0x5588d4552c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c8c977640 1 -- 172.21.15.31:0/939898302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d4157d80 msgr2=0x5588d4552c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c8c977640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d4157d80 0x5588d4552c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c8c977640 1 -- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 msgr2=0x5588d454e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c8c977640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c8c977640 1 -- 172.21.15.31:0/939898302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d454ecd0 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c8c977640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d454e660 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4c88066c70 tx=0x7f4c880a18a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cff5628aae75941 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c677fe640 1 -- 172.21.15.31:0/939898302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c880b0070 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c677fe640 1 -- 172.21.15.31:0/939898302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7f4c880ab860 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c880a1bc0 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c677fe640 1 -- 172.21.15.31:0/939898302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4c880a1a10 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c9541f740 10 client.17487 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5588d45c6d20 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.784+0000 7f4c677fe640 1 -- 172.21.15.31:0/939898302 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c880a1bc0 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c677fe640 1 -- 172.21.15.31:0/939898302 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4c880b4970 con 0x5588d41784d0 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c9541f740 10 client.17487 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c9541f740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c9541f740 20 client.17487 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c9541f740 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d41991b0 0x5588d45c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c9541f740 4 client.17487 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588d4537f80 con 0x5588d41991b0 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c87fff640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d41991b0 0x5588d45c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.785+0000 7f4c677fe640 10 client.17487 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.786+0000 7f4c87fff640 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d41991b0 0x5588d45c8f00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7641f4dbf347ff server_cookie=1e80abbba9133949 in_seq=0 out_seq=0 2024-09-21T06:21:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c677fe640 1 -- 172.21.15.31:0/939898302 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588d4537f80 con 0x5588d41991b0 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c677fe640 10 client.17487 handle_command_reply: tid=0 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 client.17487 shutdown 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 msgr2=0x5588d454e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d454e660 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4c88066c70 tx=0x7f4c880a18a0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 shutdown_connections 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5588d41991b0 0x5588d45c8f00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5588d41784d0 0x5588d454e660 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c677fe640 0 client.17487 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588d41702f0 0x5588d454e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 --2- 172.21.15.31:0/939898302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5588d4157d80 0x5588d4552c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 >> 172.21.15.31:0/939898302 conn(0x5588d418d300 msgr2=0x5588d454c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 shutdown_connections 2024-09-21T06:21:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 1 -- 172.21.15.31:0/939898302 wait complete. 2024-09-21T06:21:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.840+0000 7f4c9541f740 20 client.17487 trim_cache size 0 max 16384 2024-09-21T06:21:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:21:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:21:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:21:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:21:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b66b0 msgr2=0x7f4c780b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b66b0 0x7f4c780b8bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x5588d453d820 tx=0x7f4c74009190 comp rx=0 tx=0).stop 2024-09-21T06:21:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 msgr2=0x7f4c80133710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 0x7f4c80133710 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00b450 tx=0x7f4c7c00b910 comp rx=0 tx=0).stop 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 shutdown_connections 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c780b66b0 0x7f4c780b8bc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8009a240 0x7f4c801389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c800956e0 0x7f4c80133710 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 --2- 172.21.15.31:0/3218078172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c80094cb0 0x7f4c80133170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 >> 172.21.15.31:0/3218078172 conn(0x7f4c8000cd80 msgr2=0x7f4c80097c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 shutdown_connections 2024-09-21T06:21:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:26.841+0000 7f4c8d178640 1 -- 172.21.15.31:0/3218078172 wait complete. 2024-09-21T06:21:26.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:21:27.268 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:27.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:26 smithi110 ceph-mon[34550]: pgmap v623: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 18 MiB/s wr, 872 op/s 2024-09-21T06:21:27.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:26 smithi107 ceph-mon[34380]: pgmap v623: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 18 MiB/s wr, 872 op/s 2024-09-21T06:21:27.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h[63527]: ignoring --setuser ceph since I am not root 2024-09-21T06:21:27.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h[63527]: ignoring --setgroup ceph since I am not root 2024-09-21T06:21:27.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-mds[63532]: -- 172.21.15.107:0/3483379606 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55daae26b860 con 0x55dab2413a80 2024-09-21T06:21:27.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h[63527]: starting mds.h at 2024-09-21T06:21:27.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:26 smithi031 ceph-mon[31237]: pgmap v623: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 18 MiB/s wr, 872 op/s 2024-09-21T06:21:27.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 -- 172.21.15.31:0/3457859945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1410a9e0 msgr2=0x7efc1410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 --2- 172.21.15.31:0/3457859945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1410a9e0 0x7efc1410adf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009900 tx=0x7efbfc02a400 comp rx=0 tx=0).stop 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 -- 172.21.15.31:0/3457859945 shutdown_connections 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 --2- 172.21.15.31:0/3457859945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1411cc30 0x7efc1411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 --2- 172.21.15.31:0/3457859945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1410a9e0 0x7efc1410adf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 --2- 172.21.15.31:0/3457859945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 0x7efc1410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 -- 172.21.15.31:0/3457859945 >> 172.21.15.31:0/3457859945 conn(0x7efc14070ec0 msgr2=0x7efc140712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 -- 172.21.15.31:0/3457859945 shutdown_connections 2024-09-21T06:21:27.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.833+0000 7efc18c78640 1 -- 172.21.15.31:0/3457859945 wait complete. 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 Processor -- start 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 -- start start 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 0x7efc141a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 0x7efc141a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 0x7efc141aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc14122900 con 0x7efc1410a9e0 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc14122780 con 0x7efc14111f40 2024-09-21T06:21:27.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.834+0000 7efc18c78640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc14122a80 con 0x7efc1411cc30 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 0x7efc141aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc1259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 0x7efc141a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 0x7efc141a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 0x7efc141aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34052/0 (socket says 172.21.15.31:34052) 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 0x7efc141a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34916/0 (socket says 172.21.15.31:34916) 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc1259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 0x7efc141a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55580/0 (socket says 172.21.15.31:55580) 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 -- 172.21.15.31:0/3249024742 learned_addr learned my addr 172.21.15.31:0/3249024742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 -- 172.21.15.31:0/3249024742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 msgr2=0x7efc141a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 0x7efc141a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 -- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 msgr2=0x7efc141a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 0x7efc141a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 -- 172.21.15.31:0/3249024742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc141af180 con 0x7efc1411cc30 2024-09-21T06:21:27.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc1259c640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 0x7efc141a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:27.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc12d9d640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 0x7efc141aeb10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c970 tx=0x7efc0c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:27.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.835+0000 7efc0b7fe640 1 -- 172.21.15.31:0/3249024742 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c007bb0 con 0x7efc1411cc30 2024-09-21T06:21:27.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.836+0000 7efc0b7fe640 1 -- 172.21.15.31:0/3249024742 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0c007d50 con 0x7efc1411cc30 2024-09-21T06:21:27.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.836+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc141af470 con 0x7efc1411cc30 2024-09-21T06:21:27.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.836+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc141b5c90 con 0x7efc1411cc30 2024-09-21T06:21:27.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.837+0000 7efc0b7fe640 1 -- 172.21.15.31:0/3249024742 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c018770 con 0x7efc1411cc30 2024-09-21T06:21:27.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.840+0000 7efc1f1ec740 1 Processor -- start 2024-09-21T06:21:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.840+0000 7efc0b7fe640 1 -- 172.21.15.31:0/3249024742 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efc0c0213f0 con 0x7efc1411cc30 2024-09-21T06:21:27.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.840+0000 7efc1f1ec740 1 -- start start 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.840+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.840+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 0x560057a6f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576a7390 0x560057a6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5600579e5e50 con 0x5600576c8070 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5600579d5280 con 0x56005769f1b0 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5600579d9090 con 0x5600576a7390 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 0x560057a6f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 0x560057a6f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34928/0 (socket says 172.21.15.31:34928) 2024-09-21T06:21:27.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 -- 172.21.15.31:0/3527857580 learned_addr learned my addr 172.21.15.31:0/3527857580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc0b7fe640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc000b6730 0x7efc000b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 -- 172.21.15.31:0/3527857580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576a7390 msgr2=0x560057a6fcf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576a7390 0x560057a6fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 -- 172.21.15.31:0/3527857580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 msgr2=0x560057a6f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc0b7fe640 1 -- 172.21.15.31:0/3249024742 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7efc0c0e7660 con 0x7efc1411cc30 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc11d9b640 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.841+0000 7efc1259c640 1 -- 172.21.15.31:0/3527857580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560057733650 con 0x56005769f1b0 2024-09-21T06:21:27.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc12d9d640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc000b6730 0x7efc000b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc11d9b640 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:21:27.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc1259c640 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 0x560057a6f7b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efc04005f40 tx=0x7efc04011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca5cd241fe96647 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:27.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc12d9d640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc000b6730 0x7efc000b8c40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc0c004770 tx=0x7efc0c0046c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc08ff9640 1 -- 172.21.15.31:0/3527857580 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04011330 con 0x56005769f1b0 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc08ff9640 1 -- 172.21.15.31:0/3527857580 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0400fe50 con 0x56005769f1b0 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.842+0000 7efc08ff9640 1 -- 172.21.15.31:0/3527857580 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04011330 con 0x56005769f1b0 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.843+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3527857580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 msgr2=0x560057a6f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.843+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 0x560057a6f7b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efc04005f40 tx=0x7efc04011d30 comp rx=0 tx=0).stop 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.843+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3527857580 shutdown_connections 2024-09-21T06:21:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.843+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576a7390 0x560057a6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56005769f1b0 0x560057a6f7b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3527857580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3527857580 >> 172.21.15.31:0/3527857580 conn(0x5600576bc460 msgr2=0x560057a6d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3527857580 shutdown_connections 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3527857580 wait complete. 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 -- start start 2024-09-21T06:21:27.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.844+0000 7efc1f1ec740 1 Processor -- start 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56005769f1b0 0x560057a7a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a7ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576c8070 0x560057a7b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560057836700 con 0x56005769f1b0 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56005769f1b0 0x560057a7a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560057970110 con 0x5600576a7390 2024-09-21T06:21:27.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560057a7da50 con 0x5600576c8070 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a7ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a7ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34932/0 (socket says 172.21.15.31:34932) 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56005769f1b0 0x560057a7a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55596/0 (socket says 172.21.15.31:55596) 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 -- 172.21.15.31:0/2420803004 learned_addr learned my addr 172.21.15.31:0/2420803004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc11d9b640 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576c8070 0x560057a7b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 -- 172.21.15.31:0/2420803004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576c8070 msgr2=0x560057a7b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576c8070 0x560057a7b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 -- 172.21.15.31:0/2420803004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56005769f1b0 msgr2=0x560057a7a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56005769f1b0 0x560057a7a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1259c640 1 -- 172.21.15.31:0/2420803004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005799dc40 con 0x5600576a7390 2024-09-21T06:21:27.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 10 client.0 _ll_register_callbacks cb 0x560057989990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.845+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc0400fe50 con 0x5600576a7390 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.846+0000 7efbf9ffb640 10 client.27299 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.846+0000 7efc1259c640 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a7ae70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efc040099d0 tx=0x7efc0400fc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2378ffa4abcbbabc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.847+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2420803004 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0401f040 con 0x5600576a7390 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.847+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2420803004 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7efc04004b00 con 0x5600576a7390 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.847+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2420803004 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efc04011330 con 0x5600576a7390 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.847+0000 7efc1f1ec740 10 client.27299 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.847+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=19}) -- 0x560057a895d0 con 0x5600576a7390 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2420803004 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efc04018070 con 0x5600576a7390 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc1f1ec740 10 client.27299 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc1f1ec740 10 client.27299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc1f1ec740 20 client.27299 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5600576869f0 0x560057a8c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc1f1ec740 4 client.27299 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600579e7da0 con 0x5600576869f0 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.848+0000 7efc11d9b640 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5600576869f0 0x560057a8c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.849+0000 7efc11d9b640 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5600576869f0 0x560057a8c930 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461556fd2dafea9e server_cookie=62f151269649d65d in_seq=0 out_seq=0 2024-09-21T06:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.849+0000 7efbf9ffb640 10 client.27299 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.849+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2420803004 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5600579e7da0 con 0x5600576869f0 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.849+0000 7efbf9ffb640 10 client.27299 handle_command_reply: tid=0 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 client.27299 shutdown 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 msgr2=0x560057a7ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a7ae70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efc040099d0 tx=0x7efc0400fc50 comp rx=0 tx=0).stop 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 shutdown_connections 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5600576869f0 0x560057a8c930 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5600576c8070 0x560057a7b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a7ae70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2420803004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56005769f1b0 0x560057a7a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 >> 172.21.15.31:0/2420803004 conn(0x5600576bc460 msgr2=0x560057a72450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efbf9ffb640 0 client.27299 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 shutdown_connections 2024-09-21T06:21:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2420803004 wait complete. 2024-09-21T06:21:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.850+0000 7efc1f1ec740 20 client.27299 trim_cache size 0 max 16384 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 Processor -- start 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 -- start start 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a70d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a6c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc141bc4f0 con 0x5600576c8070 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc141bed50 con 0x5600576a7390 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.872+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc141bcb40 con 0x56005769f1b0 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55608/0 (socket says 172.21.15.31:55608) 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 -- 172.21.15.31:0/3309934473 learned_addr learned my addr 172.21.15.31:0/3309934473 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a70d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc1259c640 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a6c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 -- 172.21.15.31:0/3309934473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 msgr2=0x560057a70d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a70d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 -- 172.21.15.31:0/3309934473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 msgr2=0x560057a6c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a6c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc11d9b640 1 -- 172.21.15.31:0/3309934473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560057a7cd00 con 0x5600576c8070 2024-09-21T06:21:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.873+0000 7efc1259c640 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a6c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.874+0000 7efc11d9b640 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6cc50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005ab0 tx=0x7efbfc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1d99e5e652f2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.874+0000 7efbfbfff640 1 -- 172.21.15.31:0/3309934473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc04d030 con 0x5600576c8070 2024-09-21T06:21:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.874+0000 7efbfbfff640 1 -- 172.21.15.31:0/3309934473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc04e040 con 0x5600576c8070 2024-09-21T06:21:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.874+0000 7efbfbfff640 1 -- 172.21.15.31:0/3309934473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc035040 con 0x5600576c8070 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.875+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3309934473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 msgr2=0x560057a6cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.875+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6cc50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efbfc005ab0 tx=0x7efbfc009260 comp rx=0 tx=0).stop 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3309934473 shutdown_connections 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a6cc50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a6c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/3309934473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a70d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3309934473 >> 172.21.15.31:0/3309934473 conn(0x5600576bc460 msgr2=0x560057a86690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3309934473 shutdown_connections 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 -- 172.21.15.31:0/3309934473 wait complete. 2024-09-21T06:21:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 -- start start 2024-09-21T06:21:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.876+0000 7efc1f1ec740 1 Processor -- start 2024-09-21T06:21:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a8ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a8d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a8d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc141bd210 con 0x5600576c8070 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc141be680 con 0x5600576a7390 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efc141bd8e0 con 0x56005769f1b0 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1f1ec740 10 client.0 _ll_register_callbacks cb 0x7efc04011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a8ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc11d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a8ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34084/0 (socket says 172.21.15.31:34084) 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc11d9b640 1 -- 172.21.15.31:0/2919631344 learned_addr learned my addr 172.21.15.31:0/2919631344 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1259c640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a8d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc11d9b640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a8d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1259c640 1 -- 172.21.15.31:0/2919631344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 msgr2=0x560057a8ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1259c640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a8ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1259c640 1 -- 172.21.15.31:0/2919631344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 msgr2=0x560057a8d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1259c640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a8d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.877+0000 7efc1259c640 1 -- 172.21.15.31:0/2919631344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560057a8dea0 con 0x5600576a7390 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.878+0000 7efc1259c640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a8d3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc04000f80 tx=0x7efc04009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af5a3327d866e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.878+0000 7efbf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.878+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2919631344 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04010ae0 con 0x5600576a7390 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.878+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2919631344 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8499+0+0 (secure 0 0 0) 0x7efc04005000 con 0x5600576a7390 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.878+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc04e040 con 0x5600576a7390 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2919631344 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7efc040114d0 con 0x5600576a7390 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 10 client.27305 fetch_fsmap learned FSMap version 19 2024-09-21T06:21:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560057af6dc0 con 0x5600576a7390 2024-09-21T06:21:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2919631344 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbfc04e040 con 0x5600576a7390 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2919631344 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7efc04018050 con 0x5600576a7390 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 10 client.27305 fetch_fsmap finished waiting for FSMap version 19 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 20 client.27305 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56005765bda0 0x560057af9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 4 client.27305 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.879+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600579e3850 con 0x56005765bda0 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.880+0000 7efc11d9b640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56005765bda0 0x560057af9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.880+0000 7efbf9ffb640 10 client.27305 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.880+0000 7efc11d9b640 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56005765bda0 0x560057af9530 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6907977e3eb8f866 server_cookie=30274f3d08e9d353 in_seq=0 out_seq=0 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efbf9ffb640 1 -- 172.21.15.31:0/2919631344 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7386 (crc 0 0 0) 0x5600579e3850 con 0x56005765bda0 2024-09-21T06:21:27.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efbf9ffb640 10 client.27305 handle_command_reply: tid=0 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 client.27305 shutdown 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 msgr2=0x560057a8d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a8d3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc04000f80 tx=0x7efc04009500 comp rx=0 tx=0).stop 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 shutdown_connections 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56005765bda0 0x560057af9530 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5600576c8070 0x560057a8d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600576a7390 0x560057a8d3f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 --2- 172.21.15.31:0/2919631344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56005769f1b0 0x560057a8ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 >> 172.21.15.31:0/2919631344 conn(0x5600576bc460 msgr2=0x560057a86460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efbf9ffb640 0 client.27305 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 shutdown_connections 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 1 -- 172.21.15.31:0/2919631344 wait complete. 2024-09-21T06:21:27.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.881+0000 7efc1f1ec740 20 client.27305 trim_cache size 0 max 16384 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "71aa8c73-a180-4b01-a247-118653b8cea4", 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:21:27.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.888 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:21:27.889 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.890 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:21:27.891 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.892 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.893 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:21:27.894 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.896 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.897 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.898 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:21:27.899 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.901 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:21:27.902 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:21:27.903 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc000b6730 msgr2=0x7efc000b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc000b6730 0x7efc000b8c40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efc0c004770 tx=0x7efc0c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 msgr2=0x7efc141aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 0x7efc141aeb10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c970 tx=0x7efc0c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 shutdown_connections 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc1411cc30 0x7efc141aeb10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc14111f40 0x7efc141a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efc000b6730 0x7efc000b8c40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 --2- 172.21.15.31:0/3249024742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc1410a9e0 0x7efc141a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:27.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.882+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 >> 172.21.15.31:0/3249024742 conn(0x7efc14070ec0 msgr2=0x7efc1411d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.883+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 shutdown_connections 2024-09-21T06:21:27.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:27.883+0000 7efc18c78640 1 -- 172.21.15.31:0/3249024742 wait complete. 2024-09-21T06:21:28.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:27 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:21:28.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:27 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:28.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:27 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:21:28.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:27 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:28.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:27 smithi110 ceph-mon[34550]: daemon mds.h restarted 2024-09-21T06:21:28.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:21:28.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:28.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:21:28.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:28.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:27 smithi107 ceph-mon[34380]: daemon mds.h restarted 2024-09-21T06:21:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:27 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:21:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:27 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:27 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:21:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:27 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:21:28.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:27 smithi031 ceph-mon[31237]: daemon mds.h restarted 2024-09-21T06:21:28.921 INFO:tasks.cephfs.filesystem:scrub status for tag:71aa8c73-a180-4b01-a247-118653b8cea4 - {'path': '/', 'tag': '71aa8c73-a180-4b01-a247-118653b8cea4', 'options': 'recursive,force'} 2024-09-21T06:21:29.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:28 smithi031 ceph-mon[31237]: pgmap v624: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 738 op/s 2024-09-21T06:21:29.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:28 smithi031 ceph-mon[31237]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:21:29.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:28 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6836/2641406273,v1:172.21.15.107:6837/2641406273] up:boot 2024-09-21T06:21:29.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:28 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:21:29.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:28 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:21:29.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:28 smithi031 ceph-mon[31237]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:21:29.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:28 smithi110 ceph-mon[34550]: pgmap v624: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 738 op/s 2024-09-21T06:21:29.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:28 smithi110 ceph-mon[34550]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:21:29.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:28 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6836/2641406273,v1:172.21.15.107:6837/2641406273] up:boot 2024-09-21T06:21:29.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:28 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:21:29.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:28 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:21:29.279 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:28 smithi110 ceph-mon[34550]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:21:29.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:28 smithi107 ceph-mon[34380]: pgmap v624: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 738 op/s 2024-09-21T06:21:29.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:28 smithi107 ceph-mon[34380]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:21:29.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:28 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6836/2641406273,v1:172.21.15.107:6837/2641406273] up:boot 2024-09-21T06:21:29.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:28 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:21:29.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:28 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:21:29.290 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:28 smithi107 ceph-mon[34380]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T06:21:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 590.0 s, 1724.6 tps, lat 58.177 ms stddev 64.364 2024-09-21T06:21:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:30 smithi110 ceph-mon[34550]: pgmap v626: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 779 op/s 2024-09-21T06:21:31.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:30 smithi107 ceph-mon[34380]: pgmap v626: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 779 op/s 2024-09-21T06:21:31.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:30 smithi031 ceph-mon[31237]: pgmap v626: 97 pgs: 97 active+clean; 4.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 16 MiB/s wr, 779 op/s 2024-09-21T06:21:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:32 smithi110 ceph-mon[34550]: pgmap v627: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 12 MiB/s wr, 725 op/s 2024-09-21T06:21:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:32 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:21:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:32 smithi107 ceph-mon[34380]: pgmap v627: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 12 MiB/s wr, 725 op/s 2024-09-21T06:21:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:32 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:21:33.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:32 smithi031 ceph-mon[31237]: pgmap v627: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 12 MiB/s wr, 725 op/s 2024-09-21T06:21:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:32 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 595.0 s, 1478.8 tps, lat 67.739 ms stddev 114.052 2024-09-21T06:21:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:34 smithi110 ceph-mon[34550]: pgmap v628: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 12 MiB/s wr, 725 op/s 2024-09-21T06:21:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:34 smithi107 ceph-mon[34380]: pgmap v628: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 12 MiB/s wr, 725 op/s 2024-09-21T06:21:35.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:34 smithi031 ceph-mon[31237]: pgmap v628: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 12 MiB/s wr, 725 op/s 2024-09-21T06:21:37.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:36 smithi110 ceph-mon[34550]: pgmap v629: 97 pgs: 97 active+clean; 4.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2024-09-21T06:21:37.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:36 smithi107 ceph-mon[34380]: pgmap v629: 97 pgs: 97 active+clean; 4.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2024-09-21T06:21:37.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:36 smithi031 ceph-mon[31237]: pgmap v629: 97 pgs: 97 active+clean; 4.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2024-09-21T06:21:39.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:38 smithi110 ceph-mon[34550]: pgmap v630: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 26 MiB/s wr, 1.95k op/s 2024-09-21T06:21:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:38 smithi107 ceph-mon[34380]: pgmap v630: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 26 MiB/s wr, 1.95k op/s 2024-09-21T06:21:39.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:38 smithi031 ceph-mon[31237]: pgmap v630: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 26 MiB/s wr, 1.95k op/s 2024-09-21T06:21:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.003+0000 7f1009521640 1 -- 172.21.15.31:0/1447658626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 msgr2=0x7f1004111200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.003+0000 7f1009521640 1 --2- 172.21.15.31:0/1447658626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 0x7f1004111200 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009900 tx=0x7f0ff402a4b0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.007+0000 7f1009521640 1 -- 172.21.15.31:0/1447658626 shutdown_connections 2024-09-21T06:21:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.007+0000 7f1009521640 1 --2- 172.21.15.31:0/1447658626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406c3a0 0x7f100410f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.007+0000 7f1009521640 1 --2- 172.21.15.31:0/1447658626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100406ba00 0x7f100406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.007+0000 7f1009521640 1 --2- 172.21.15.31:0/1447658626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 0x7f1004111200 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.007+0000 7f1009521640 1 -- 172.21.15.31:0/1447658626 >> 172.21.15.31:0/1447658626 conn(0x7f1004074e40 msgr2=0x7f1004070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.010+0000 7f1009521640 1 -- 172.21.15.31:0/1447658626 shutdown_connections 2024-09-21T06:21:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.011+0000 7f1009521640 1 -- 172.21.15.31:0/1447658626 wait complete. 2024-09-21T06:21:40.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.012+0000 7f1009521640 1 Processor -- start 2024-09-21T06:21:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.013+0000 7f1009521640 1 -- start start 2024-09-21T06:21:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.013+0000 7f1009521640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 0x7f10041a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.013+0000 7f1009521640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100406c3a0 0x7f10041a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.013+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 0x7f10041a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.013+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 0x7f10041a9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34504/0 (socket says 172.21.15.31:34504) 2024-09-21T06:21:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.013+0000 7f1002ffd640 1 -- 172.21.15.31:0/34227599 learned_addr learned my addr 172.21.15.31:0/34227599 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f10027fc640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100406c3a0 0x7f10041a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 0x7f10041aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10041228c0 con 0x7f1004110df0 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1004122740 con 0x7f100406c3a0 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1004122a40 con 0x7f100406ba00 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1002ffd640 1 -- 172.21.15.31:0/34227599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100406c3a0 msgr2=0x7f10041a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1002ffd640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100406c3a0 0x7f10041a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1002ffd640 1 -- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 msgr2=0x7f10041aeb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1002ffd640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 0x7f10041aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.014+0000 7f1002ffd640 1 -- 172.21.15.31:0/34227599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10041af250 con 0x7f100406ba00 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.015+0000 7f1002ffd640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 0x7f10041a9320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005ab0 tx=0x7f0ff4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.015+0000 7f0febfff640 1 -- 172.21.15.31:0/34227599 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff403f070 con 0x7f100406ba00 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.015+0000 7f0febfff640 1 -- 172.21.15.31:0/34227599 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff4031b50 con 0x7f100406ba00 2024-09-21T06:21:40.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.015+0000 7f0febfff640 1 -- 172.21.15.31:0/34227599 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4035030 con 0x7f100406ba00 2024-09-21T06:21:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.015+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10041af4e0 con 0x7f100406ba00 2024-09-21T06:21:40.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.016+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10041af940 con 0x7f100406ba00 2024-09-21T06:21:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.020+0000 7f0febfff640 1 -- 172.21.15.31:0/34227599 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0ff4042070 con 0x7f100406ba00 2024-09-21T06:21:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.020+0000 7f1010488740 1 Processor -- start 2024-09-21T06:21:40.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 -- start start 2024-09-21T06:21:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 0x563f774d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7710a4d0 0x563f774d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f774456f0 con 0x563f7712b1b0 2024-09-21T06:21:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f77434b20 con 0x563f771022f0 2024-09-21T06:21:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.021+0000 7f1010488740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f77438930 con 0x563f7710a4d0 2024-09-21T06:21:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10037fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 0x563f774d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10037fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 0x563f774d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51554/0 (socket says 172.21.15.31:51554) 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10037fe640 1 -- 172.21.15.31:0/3946746836 learned_addr learned my addr 172.21.15.31:0/3946746836 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7710a4d0 0x563f774d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 -- 172.21.15.31:0/3946746836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 msgr2=0x563f774d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 -- 172.21.15.31:0/3946746836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 msgr2=0x563f774d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 0x563f774d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 -- 172.21.15.31:0/3946746836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f773fa530 con 0x563f7710a4d0 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10037fe640 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 0x563f774d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.022+0000 7f10027fc640 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7710a4d0 0x563f774d13f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f100406b850 tx=0x7f0ffc00c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9edd9a7b54dd71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.023+0000 7f0fe97fa640 1 -- 172.21.15.31:0/3946746836 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc015020 con 0x563f7710a4d0 2024-09-21T06:21:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.023+0000 7f0fe97fa640 1 -- 172.21.15.31:0/3946746836 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc011ae0 con 0x563f7710a4d0 2024-09-21T06:21:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.023+0000 7f0fe97fa640 1 -- 172.21.15.31:0/3946746836 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0108f0 con 0x563f7710a4d0 2024-09-21T06:21:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.023+0000 7f0febfff640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0fec0b67b0 0x7f0fec0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.024+0000 7f0febfff640 1 -- 172.21.15.31:0/34227599 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f0ff4107f20 con 0x7f100406ba00 2024-09-21T06:21:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 -- 172.21.15.31:0/3946746836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7710a4d0 msgr2=0x563f774d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7710a4d0 0x563f774d13f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f100406b850 tx=0x7f0ffc00c560 comp rx=0 tx=0).stop 2024-09-21T06:21:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f10037fe640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0fec0b67b0 0x7f0fec0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 -- 172.21.15.31:0/3946746836 shutdown_connections 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7710a4d0 0x563f774d13f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 --2- 172.21.15.31:0/3946746836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7712b1b0 0x563f774d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 -- 172.21.15.31:0/3946746836 >> 172.21.15.31:0/3946746836 conn(0x563f7711f300 msgr2=0x563f774ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.026+0000 7f1010488740 1 -- 172.21.15.31:0/3946746836 shutdown_connections 2024-09-21T06:21:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.027+0000 7f1010488740 1 -- 172.21.15.31:0/3946746836 wait complete. 2024-09-21T06:21:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.027+0000 7f10037fe640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0fec0b67b0 0x7f0fec0b8cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009a50 tx=0x7f0ff8009720 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.027+0000 7f1010488740 1 -- start start 2024-09-21T06:21:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.027+0000 7f1010488740 1 Processor -- start 2024-09-21T06:21:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f771022f0 0x563f774dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f7710a4d0 0x563f774dc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 0x563f774e8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f7736cab0 con 0x563f771022f0 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f773da730 con 0x563f7710a4d0 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f774df0f0 con 0x563f7712b1b0 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1010488740 10 client.0 _ll_register_callbacks cb 0x563f7742e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f771022f0 0x563f774dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f10037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 0x563f774e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f10037fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 0x563f774e8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34522/0 (socket says 172.21.15.31:34522) 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f10037fe640 1 -- 172.21.15.31:0/1273812970 learned_addr learned my addr 172.21.15.31:0/1273812970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.028+0000 7f10027fc640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f7710a4d0 0x563f774dc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10027fc640 1 -- 172.21.15.31:0/1273812970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 msgr2=0x563f774e8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10027fc640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 0x563f774e8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10027fc640 1 -- 172.21.15.31:0/1273812970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f771022f0 msgr2=0x563f774dc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10027fc640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f771022f0 0x563f774dc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10027fc640 1 -- 172.21.15.31:0/1273812970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f77196790 con 0x563f7710a4d0 2024-09-21T06:21:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10037fe640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 0x563f774e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f1002ffd640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f771022f0 0x563f774dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f10027fc640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f7710a4d0 0x563f774dc610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc010470 tx=0x7f0ffc007a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a91297e76342428 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f0fe6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.029+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/1273812970 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc029060 con 0x563f7710a4d0 2024-09-21T06:21:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/1273812970 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f0ffc010a80 con 0x563f7710a4d0 2024-09-21T06:21:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ffc011ae0 con 0x563f7710a4d0 2024-09-21T06:21:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/1273812970 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0ffc015020 con 0x563f7710a4d0 2024-09-21T06:21:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 10 client.27308 fetch_fsmap learned FSMap version 21 2024-09-21T06:21:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f7720f620 con 0x563f7710a4d0 2024-09-21T06:21:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/1273812970 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ffc011ae0 con 0x563f7710a4d0 2024-09-21T06:21:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/1273812970 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0ffc023030 con 0x563f7710a4d0 2024-09-21T06:21:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 10 client.27308 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:21:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 10 client.27308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 20 client.27308 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f770e9d80 0x563f774ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 4 client.27308 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.030+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f77447640 con 0x563f770e9d80 2024-09-21T06:21:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.031+0000 7f1002ffd640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f770e9d80 0x563f774ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.031+0000 7f0fe6ff5640 10 client.27308 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.033+0000 7f1002ffd640 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f770e9d80 0x563f774ee080 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=387864996e485e18 server_cookie=844468323d1a07ab in_seq=0 out_seq=0 2024-09-21T06:21:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.033+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/1273812970 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f77447640 con 0x563f770e9d80 2024-09-21T06:21:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.033+0000 7f0fe6ff5640 10 client.27308 handle_command_reply: tid=0 2024-09-21T06:21:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.033+0000 7f1010488740 1 client.27308 shutdown 2024-09-21T06:21:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f7710a4d0 msgr2=0x563f774dc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f7710a4d0 0x563f774dc610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc010470 tx=0x7f0ffc007a70 comp rx=0 tx=0).stop 2024-09-21T06:21:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 shutdown_connections 2024-09-21T06:21:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f770e9d80 0x563f774ee080 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f7712b1b0 0x563f774e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f7710a4d0 0x563f774dc610 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f0fe6ff5640 0 client.27308 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 --2- 172.21.15.31:0/1273812970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f771022f0 0x563f774dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 >> 172.21.15.31:0/1273812970 conn(0x563f7711f300 msgr2=0x563f774d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 shutdown_connections 2024-09-21T06:21:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 1 -- 172.21.15.31:0/1273812970 wait complete. 2024-09-21T06:21:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.034+0000 7f1010488740 20 client.27308 trim_cache size 0 max 16384 2024-09-21T06:21:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.055+0000 7f1010488740 1 Processor -- start 2024-09-21T06:21:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 -- start start 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f774e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10041bc4a0 con 0x563f7710a4d0 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10041bed20 con 0x563f771022f0 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1010488740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10041bcb10 con 0x563f770e9d80 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.056+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774dc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34532/0 (socket says 172.21.15.31:34532) 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 -- 172.21.15.31:0/2366208196 learned_addr learned my addr 172.21.15.31:0/2366208196 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f10027fc640 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f10037fe640 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f774e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 -- 172.21.15.31:0/2366208196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 msgr2=0x563f774d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 -- 172.21.15.31:0/2366208196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 msgr2=0x563f774e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f774e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 -- 172.21.15.31:0/2366208196 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f774cf5f0 con 0x563f770e9d80 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f10037fe640 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f774e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.057+0000 7f1002ffd640 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774dc5b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000c00 tx=0x7f0ff40f1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87af2f21de0124 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.058+0000 7f0fe8ff9640 1 -- 172.21.15.31:0/2366208196 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40b9070 con 0x563f770e9d80 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.058+0000 7f0fe8ff9640 1 -- 172.21.15.31:0/2366208196 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff40d2070 con 0x563f770e9d80 2024-09-21T06:21:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.058+0000 7f0fe8ff9640 1 -- 172.21.15.31:0/2366208196 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4049040 con 0x563f770e9d80 2024-09-21T06:21:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.058+0000 7f1010488740 1 -- 172.21.15.31:0/2366208196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 msgr2=0x563f774dc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.058+0000 7f1010488740 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774dc5b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000c00 tx=0x7f0ff40f1040 comp rx=0 tx=0).stop 2024-09-21T06:21:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.058+0000 7f1010488740 1 -- 172.21.15.31:0/2366208196 shutdown_connections 2024-09-21T06:21:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f774e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 --2- 172.21.15.31:0/2366208196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774dc5b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- 172.21.15.31:0/2366208196 >> 172.21.15.31:0/2366208196 conn(0x563f7711f300 msgr2=0x563f774d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- 172.21.15.31:0/2366208196 shutdown_connections 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- 172.21.15.31:0/2366208196 wait complete. 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- start start 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 Processor -- start 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774dff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f775568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10041bd1e0 con 0x563f7710a4d0 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10041be650 con 0x563f771022f0 2024-09-21T06:21:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.059+0000 7f1010488740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f10041bd8b0 con 0x563f770e9d80 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1010488740 10 client.0 _ll_register_callbacks cb 0x7f0ff8009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f10027fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34534/0 (socket says 172.21.15.31:34534) 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f10027fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774dff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:45882/0 (socket says 172.21.15.31:45882) 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 -- 172.21.15.31:0/4004011249 learned_addr learned my addr 172.21.15.31:0/4004011249 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f10037fe640 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f775568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 -- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 msgr2=0x563f774dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774dff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 -- 172.21.15.31:0/4004011249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 msgr2=0x563f775568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f775568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 -- 172.21.15.31:0/4004011249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f77556e10 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1002ffd640 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774df960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40cdce0 tx=0x7f0ff40b91e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757ac59cc89e3722 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f0fe6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/4004011249 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4049070 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.060+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff40d2070 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/4004011249 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f0ff40c9fc0 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/4004011249 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0ff40b9040 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f1010488740 10 client.27355 fetch_fsmap learned FSMap version 21 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f77557fc0 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/4004011249 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff40d2070 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/4004011249 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0ff40d72e0 con 0x563f770e9d80 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f1010488740 10 client.27355 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:21:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f1010488740 10 client.27355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f1010488740 20 client.27355 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.061+0000 7f1010488740 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f7712b1b0 0x563f7755a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.062+0000 7f1010488740 4 client.27355 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.062+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f774430f0 con 0x563f7712b1b0 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.062+0000 7f10027fc640 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f7712b1b0 0x563f7755a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.063+0000 7f0fe6ff5640 10 client.27355 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.063+0000 7f10027fc640 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f7712b1b0 0x563f7755a630 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d601eb8a5819618b server_cookie=62fe1eb4a72d6ed in_seq=0 out_seq=0 2024-09-21T06:21:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f0fe6ff5640 1 -- 172.21.15.31:0/4004011249 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f774430f0 con 0x563f7712b1b0 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f0fe6ff5640 10 client.27355 handle_command_reply: tid=0 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 client.27355 shutdown 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 msgr2=0x563f774df960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774df960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40cdce0 tx=0x7f0ff40b91e0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 shutdown_connections 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f7712b1b0 0x563f7755a630 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f7710a4d0 0x563f775568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f0fe6ff5640 0 client.27355 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f771022f0 0x563f774dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 --2- 172.21.15.31:0/4004011249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f770e9d80 0x563f774df960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.098+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 >> 172.21.15.31:0/4004011249 conn(0x563f7711f300 msgr2=0x563f774d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 shutdown_connections 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1010488740 1 -- 172.21.15.31:0/4004011249 wait complete. 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1010488740 20 client.27355 trim_cache size 0 max 16384 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:21:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:21:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0fec0b67b0 msgr2=0x7f0fec0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0fec0b67b0 0x7f0fec0b8cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009a50 tx=0x7f0ff8009720 comp rx=0 tx=0).stop 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 msgr2=0x7f10041a9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 0x7f10041a9320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005ab0 tx=0x7f0ff4003c20 comp rx=0 tx=0).stop 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 shutdown_connections 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0fec0b67b0 0x7f0fec0b8cc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1004110df0 0x7f10041aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100406c3a0 0x7f10041a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 --2- 172.21.15.31:0/34227599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f100406ba00 0x7f10041a9320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.099+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 >> 172.21.15.31:0/34227599 conn(0x7f1004074e40 msgr2=0x7f100410d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.101+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 shutdown_connections 2024-09-21T06:21:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:40.101+0000 7f1009521640 1 -- 172.21.15.31:0/34227599 wait complete. 2024-09-21T06:21:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 600.0 s, 1661.8 tps, lat 58.476 ms stddev 69.455 2024-09-21T06:21:41.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:40 smithi110 ceph-mon[34550]: pgmap v631: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 23 MiB/s wr, 1.66k op/s 2024-09-21T06:21:41.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:40 smithi107 ceph-mon[34380]: pgmap v631: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 23 MiB/s wr, 1.66k op/s 2024-09-21T06:21:41.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:40 smithi031 ceph-mon[31237]: pgmap v631: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 23 MiB/s wr, 1.66k op/s 2024-09-21T06:21:43.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:42 smithi110 ceph-mon[34550]: pgmap v632: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2024-09-21T06:21:43.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:42 smithi107 ceph-mon[34380]: pgmap v632: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2024-09-21T06:21:43.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:42 smithi031 ceph-mon[31237]: pgmap v632: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 25 MiB/s wr, 1.84k op/s 2024-09-21T06:21:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 605.0 s, 1413.6 tps, lat 72.430 ms stddev 91.493 2024-09-21T06:21:45.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:45 smithi110 ceph-mon[34550]: pgmap v633: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 1.54k op/s 2024-09-21T06:21:45.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:45 smithi107 ceph-mon[34380]: pgmap v633: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 1.54k op/s 2024-09-21T06:21:45.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:45 smithi031 ceph-mon[31237]: pgmap v633: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 21 MiB/s wr, 1.54k op/s 2024-09-21T06:21:47.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:47 smithi110 ceph-mon[34550]: pgmap v634: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 23 MiB/s wr, 1.70k op/s 2024-09-21T06:21:47.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:47 smithi107 ceph-mon[34380]: pgmap v634: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 23 MiB/s wr, 1.70k op/s 2024-09-21T06:21:47.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:47 smithi031 ceph-mon[31237]: pgmap v634: 97 pgs: 97 active+clean; 4.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 23 MiB/s wr, 1.70k op/s 2024-09-21T06:21:49.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:49 smithi110 ceph-mon[34550]: pgmap v635: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-21T06:21:49.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:49 smithi107 ceph-mon[34380]: pgmap v635: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-21T06:21:49.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:49 smithi031 ceph-mon[31237]: pgmap v635: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-21T06:21:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 610.0 s, 1383.6 tps, lat 72.636 ms stddev 138.289 2024-09-21T06:21:51.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:51 smithi110 ceph-mon[34550]: pgmap v636: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 842 op/s 2024-09-21T06:21:51.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:51 smithi107 ceph-mon[34380]: pgmap v636: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 842 op/s 2024-09-21T06:21:51.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:51 smithi031 ceph-mon[31237]: pgmap v636: 97 pgs: 97 active+clean; 4.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 842 op/s 2024-09-21T06:21:52.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:52 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:21:52.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:52 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:21:52.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:52 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:21:53.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:53 smithi031 ceph-mon[31237]: pgmap v637: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-09-21T06:21:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.255+0000 7fee998d5640 1 -- 172.21.15.31:0/623368881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 msgr2=0x7fee94109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.255+0000 7fee998d5640 1 --2- 172.21.15.31:0/623368881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee94109810 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009ca0 tx=0x7fee8c02a8f0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 -- 172.21.15.31:0/623368881 shutdown_connections 2024-09-21T06:21:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 --2- 172.21.15.31:0/623368881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94109d50 0x7fee94110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 --2- 172.21.15.31:0/623368881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee94109810 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 --2- 172.21.15.31:0/623368881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee94103ad0 0x7fee94103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 -- 172.21.15.31:0/623368881 >> 172.21.15.31:0/623368881 conn(0x7fee940ff510 msgr2=0x7fee94101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 -- 172.21.15.31:0/623368881 shutdown_connections 2024-09-21T06:21:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.262+0000 7fee998d5640 1 -- 172.21.15.31:0/623368881 wait complete. 2024-09-21T06:21:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.263+0000 7fee998d5640 1 Processor -- start 2024-09-21T06:21:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 -- start start 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee94103ad0 0x7fee9410dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee9410e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94109d50 0x7fee941ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee94077220 con 0x7fee94103ad0 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee940770a0 con 0x7fee94109d50 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee998d5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee940773a0 con 0x7fee94109400 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee927fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee9410e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee927fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee9410e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44138/0 (socket says 172.21.15.31:44138) 2024-09-21T06:21:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.264+0000 7fee927fc640 1 -- 172.21.15.31:0/3475292354 learned_addr learned my addr 172.21.15.31:0/3475292354 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee927fc640 1 -- 172.21.15.31:0/3475292354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94109d50 msgr2=0x7fee941ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee927fc640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94109d50 0x7fee941ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee927fc640 1 -- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee94103ad0 msgr2=0x7fee9410dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee927fc640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee94103ad0 0x7fee9410dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee927fc640 1 -- 172.21.15.31:0/3475292354 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee941acd70 con 0x7fee94109400 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee927fc640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee9410e1f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fee8c003ce0 tx=0x7fee8c003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee988d3640 1 -- 172.21.15.31:0/3475292354 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0307f0 con 0x7fee94109400 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee988d3640 1 -- 172.21.15.31:0/3475292354 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c02d070 con 0x7fee94109400 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee941acf40 con 0x7fee94109400 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.265+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee941ad480 con 0x7fee94109400 2024-09-21T06:21:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.266+0000 7fee988d3640 1 -- 172.21.15.31:0/3475292354 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c03b410 con 0x7fee94109400 2024-09-21T06:21:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.270+0000 7feea083e740 1 Processor -- start 2024-09-21T06:21:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.271+0000 7feea083e740 1 -- start start 2024-09-21T06:21:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7fee988d3640 1 -- 172.21.15.31:0/3475292354 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fee8c03b5b0 con 0x7fee94109400 2024-09-21T06:21:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7feea083e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 0x561606d269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7feea083e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069582f0 0x561606d26ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7feea083e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d27430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7feea083e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561606c9b6f0 con 0x5616069604d0 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7feea083e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561606c8ab20 con 0x5616069582f0 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7feea083e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561606c8e930 con 0x5616069811b0 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7fee92ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069582f0 0x561606d26ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7fee937fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 0x561606d269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7fee937fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 0x561606d269b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44154/0 (socket says 172.21.15.31:44154) 2024-09-21T06:21:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7fee937fe640 1 -- 172.21.15.31:0/1081843724 learned_addr learned my addr 172.21.15.31:0/1081843724 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.272+0000 7fee927fc640 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d27430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee937fe640 1 -- 172.21.15.31:0/1081843724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069582f0 msgr2=0x561606d26ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee937fe640 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069582f0 0x561606d26ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee937fe640 1 -- 172.21.15.31:0/1081843724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 msgr2=0x561606d27430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee937fe640 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d27430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee937fe640 1 -- 172.21.15.31:0/1081843724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561606c50530 con 0x5616069811b0 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee937fe640 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 0x561606d269b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee840097c0 tx=0x7fee84009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82f5ce4da3d87d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee79ffb640 1 -- 172.21.15.31:0/1081843724 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84015020 con 0x5616069811b0 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee79ffb640 1 -- 172.21.15.31:0/1081843724 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84011420 con 0x5616069811b0 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.273+0000 7fee79ffb640 1 -- 172.21.15.31:0/1081843724 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840105e0 con 0x5616069811b0 2024-09-21T06:21:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.274+0000 7fee988d3640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fee7c0b6700 0x7fee7c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.274+0000 7fee92ffd640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fee7c0b6700 0x7fee7c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.275+0000 7fee988d3640 1 -- 172.21.15.31:0/3475292354 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fee8c108da0 con 0x7fee94109400 2024-09-21T06:21:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.275+0000 7feea083e740 1 -- 172.21.15.31:0/1081843724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 msgr2=0x561606d269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.275+0000 7feea083e740 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 0x561606d269b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee840097c0 tx=0x7fee84009c80 comp rx=0 tx=0).stop 2024-09-21T06:21:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.275+0000 7fee92ffd640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fee7c0b6700 0x7fee7c0b8c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x561606d25630 tx=0x7fee88008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.276+0000 7feea083e740 1 -- 172.21.15.31:0/1081843724 shutdown_connections 2024-09-21T06:21:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.276+0000 7feea083e740 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d27430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.276+0000 7feea083e740 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069582f0 0x561606d26ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.276+0000 7feea083e740 1 --2- 172.21.15.31:0/1081843724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069811b0 0x561606d269b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.276+0000 7feea083e740 1 -- 172.21.15.31:0/1081843724 >> 172.21.15.31:0/1081843724 conn(0x561606975300 msgr2=0x561606d248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.277+0000 7feea083e740 1 -- 172.21.15.31:0/1081843724 shutdown_connections 2024-09-21T06:21:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.277+0000 7feea083e740 1 -- 172.21.15.31:0/1081843724 wait complete. 2024-09-21T06:21:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.277+0000 7feea083e740 1 -- start start 2024-09-21T06:21:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 Processor -- start 2024-09-21T06:21:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 0x561606d32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069604d0 0x561606d32590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561606bc2ab0 con 0x5616069582f0 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561606c30730 con 0x5616069811b0 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561606d35130 con 0x5616069604d0 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.278+0000 7feea083e740 10 client.0 _ll_register_callbacks cb 0x561606c846b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 0x561606d32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 0x561606d32050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44260/0 (socket says 172.21.15.31:44260) 2024-09-21T06:21:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 -- 172.21.15.31:0/271622237 learned_addr learned my addr 172.21.15.31:0/271622237 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee927fc640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069604d0 0x561606d32590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee937fe640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 -- 172.21.15.31:0/271622237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069604d0 msgr2=0x561606d32590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069604d0 0x561606d32590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 -- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 msgr2=0x561606d3e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee92ffd640 1 -- 172.21.15.31:0/271622237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616069ec790 con 0x5616069582f0 2024-09-21T06:21:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.279+0000 7fee937fe640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:21:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.280+0000 7fee92ffd640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 0x561606d32050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee8800de10 tx=0x7fee880047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e519c7bc6ab7311 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.280+0000 7fee72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:21:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.280+0000 7fee72ffd640 1 -- 172.21.15.31:0/271622237 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88024060 con 0x5616069582f0 2024-09-21T06:21:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.280+0000 7fee72ffd640 1 -- 172.21.15.31:0/271622237 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fee8801f400 con 0x5616069582f0 2024-09-21T06:21:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.280+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee84011420 con 0x5616069582f0 2024-09-21T06:21:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.280+0000 7fee72ffd640 1 -- 172.21.15.31:0/271622237 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee8801f600 con 0x5616069582f0 2024-09-21T06:21:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.281+0000 7feea083e740 10 client.17523 fetch_fsmap learned FSMap version 21 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.281+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561606a65620 con 0x5616069582f0 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.282+0000 7fee72ffd640 1 -- 172.21.15.31:0/271622237 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84011420 con 0x5616069582f0 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7fee72ffd640 1 -- 172.21.15.31:0/271622237 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fee88013070 con 0x5616069582f0 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7feea083e740 10 client.17523 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7feea083e740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7feea083e740 20 client.17523 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7feea083e740 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56160693fd80 0x561606d44780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7feea083e740 4 client.17523 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561606c9d640 con 0x56160693fd80 2024-09-21T06:21:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.283+0000 7fee927fc640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56160693fd80 0x561606d44780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.284+0000 7fee72ffd640 10 client.17523 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.284+0000 7fee927fc640 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56160693fd80 0x561606d44780 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deeb3beacfac0814 server_cookie=ae2f694a1c980d40 in_seq=0 out_seq=0 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7fee72ffd640 1 -- 172.21.15.31:0/271622237 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561606c9d640 con 0x56160693fd80 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7fee72ffd640 10 client.17523 handle_command_reply: tid=0 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 client.17523 shutdown 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 msgr2=0x561606d32050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 0x561606d32050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee8800de10 tx=0x7fee880047e0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 shutdown_connections 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56160693fd80 0x561606d44780 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069604d0 0x561606d32590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 --2- 172.21.15.31:0/271622237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069582f0 0x561606d32050 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 >> 172.21.15.31:0/271622237 conn(0x561606975300 msgr2=0x561606d279e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7fee72ffd640 0 client.17523 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:53.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.286+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 shutdown_connections 2024-09-21T06:21:53.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:53 smithi107 ceph-mon[34380]: pgmap v637: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-09-21T06:21:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.287+0000 7feea083e740 1 -- 172.21.15.31:0/271622237 wait complete. 2024-09-21T06:21:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.287+0000 7feea083e740 20 client.17523 trim_cache size 0 max 16384 2024-09-21T06:21:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 Processor -- start 2024-09-21T06:21:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 -- start start 2024-09-21T06:21:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d33c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000c760 con 0x5616069604d0 2024-09-21T06:21:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000efe0 con 0x5616069811b0 2024-09-21T06:21:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.310+0000 7feea083e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000cdd0 con 0x5616069582f0 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee927fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d33c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee927fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d33c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44270/0 (socket says 172.21.15.31:44270) 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee927fc640 1 -- 172.21.15.31:0/2214696370 learned_addr learned my addr 172.21.15.31:0/2214696370 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee92ffd640 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 -- 172.21.15.31:0/2214696370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 msgr2=0x561606d283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 -- 172.21.15.31:0/2214696370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 msgr2=0x561606d33c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d33c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 -- 172.21.15.31:0/2214696370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561606d33310 con 0x5616069811b0 2024-09-21T06:21:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee92ffd640 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:21:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.312+0000 7fee937fe640 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fee8400b4b0 tx=0x7fee84009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2033b375c3c7a4cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.313+0000 7fee797fa640 1 -- 172.21.15.31:0/2214696370 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84015030 con 0x5616069811b0 2024-09-21T06:21:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.313+0000 7fee797fa640 1 -- 172.21.15.31:0/2214696370 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84011420 con 0x5616069811b0 2024-09-21T06:21:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.313+0000 7fee797fa640 1 -- 172.21.15.31:0/2214696370 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840246d0 con 0x5616069811b0 2024-09-21T06:21:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 -- 172.21.15.31:0/2214696370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 msgr2=0x561606d3e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fee8400b4b0 tx=0x7fee84009440 comp rx=0 tx=0).stop 2024-09-21T06:21:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 -- 172.21.15.31:0/2214696370 shutdown_connections 2024-09-21T06:21:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d3e3b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d33c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 --2- 172.21.15.31:0/2214696370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 -- 172.21.15.31:0/2214696370 >> 172.21.15.31:0/2214696370 conn(0x561606975300 msgr2=0x561606d33570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 -- 172.21.15.31:0/2214696370 shutdown_connections 2024-09-21T06:21:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.314+0000 7feea083e740 1 -- 172.21.15.31:0/2214696370 wait complete. 2024-09-21T06:21:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 -- start start 2024-09-21T06:21:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 Processor -- start 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d37340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d37880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d37dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000d4a0 con 0x5616069604d0 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000e910 con 0x5616069811b0 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee8000db70 con 0x5616069582f0 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.315+0000 7feea083e740 10 client.0 _ll_register_callbacks cb 0x7fee88009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee92ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d37340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee92ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d37340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44188/0 (socket says 172.21.15.31:44188) 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee92ffd640 1 -- 172.21.15.31:0/1913879857 learned_addr learned my addr 172.21.15.31:0/1913879857 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d37dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee927fc640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d37880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 -- 172.21.15.31:0/1913879857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 msgr2=0x561606d37340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d37340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 -- 172.21.15.31:0/1913879857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 msgr2=0x561606d37880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d37880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 -- 172.21.15.31:0/1913879857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561606dad180 con 0x5616069811b0 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee92ffd640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee937fe640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d37dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fee84009cc0 tx=0x7fee840090c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9de28c8c91a257b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.316+0000 7fee72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:21:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.318+0000 7fee72ffd640 1 -- 172.21.15.31:0/1913879857 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84015040 con 0x5616069811b0 2024-09-21T06:21:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.318+0000 7fee72ffd640 1 -- 172.21.15.31:0/1913879857 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fee8400bce0 con 0x5616069811b0 2024-09-21T06:21:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.318+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee84011420 con 0x5616069811b0 2024-09-21T06:21:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.318+0000 7fee72ffd640 1 -- 172.21.15.31:0/1913879857 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fee840115d0 con 0x5616069811b0 2024-09-21T06:21:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.318+0000 7feea083e740 10 client.27323 fetch_fsmap learned FSMap version 21 2024-09-21T06:21:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.318+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561606dae370 con 0x5616069811b0 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7fee72ffd640 1 -- 172.21.15.31:0/1913879857 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84011420 con 0x5616069811b0 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7fee72ffd640 1 -- 172.21.15.31:0/1913879857 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fee8402e030 con 0x5616069811b0 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7feea083e740 10 client.27323 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7feea083e740 10 client.27323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7feea083e740 20 client.27323 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7feea083e740 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5616069151c0 0x561606db07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7feea083e740 4 client.27323 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:21:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.319+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561606c990f0 con 0x5616069151c0 2024-09-21T06:21:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.320+0000 7fee92ffd640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5616069151c0 0x561606db07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.320+0000 7fee72ffd640 10 client.27323 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.320+0000 7fee92ffd640 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5616069151c0 0x561606db07e0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1397e43a63b23e29 server_cookie=af21e6f6f8ffe93b in_seq=0 out_seq=0 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.434+0000 7fee72ffd640 1 -- 172.21.15.31:0/1913879857 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561606c990f0 con 0x5616069151c0 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7fee72ffd640 10 client.27323 handle_command_reply: tid=0 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 client.27323 shutdown 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 msgr2=0x561606d37dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d37dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fee84009cc0 tx=0x7fee840090c0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 shutdown_connections 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5616069151c0 0x561606db07e0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5616069811b0 0x561606d37dc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5616069604d0 0x561606d37880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 --2- 172.21.15.31:0/1913879857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5616069582f0 0x561606d37340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 >> 172.21.15.31:0/1913879857 conn(0x561606975300 msgr2=0x561606d28170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7fee72ffd640 0 client.27323 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.435+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 shutdown_connections 2024-09-21T06:21:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.436+0000 7feea083e740 1 -- 172.21.15.31:0/1913879857 wait complete. 2024-09-21T06:21:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.436+0000 7feea083e740 20 client.27323 trim_cache size 0 max 16384 2024-09-21T06:21:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:21:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:21:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:21:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:21:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fee7c0b6700 msgr2=0x7fee7c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fee7c0b6700 0x7fee7c0b8c10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x561606d25630 tx=0x7fee88008040 comp rx=0 tx=0).stop 2024-09-21T06:21:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 msgr2=0x7fee9410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee9410e1f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fee8c003ce0 tx=0x7fee8c003d10 comp rx=0 tx=0).stop 2024-09-21T06:21:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 shutdown_connections 2024-09-21T06:21:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94109d50 0x7fee941ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee94109400 0x7fee9410e1f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fee7c0b6700 0x7fee7c0b8c10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 --2- 172.21.15.31:0/3475292354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee94103ad0 0x7fee9410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 >> 172.21.15.31:0/3475292354 conn(0x7fee940ff510 msgr2=0x7fee94101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 shutdown_connections 2024-09-21T06:21:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:21:53.437+0000 7fee998d5640 1 -- 172.21.15.31:0/3475292354 wait complete. 2024-09-21T06:21:53.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:53 smithi110 ceph-mon[34550]: pgmap v637: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 19 MiB/s wr, 1.54k op/s 2024-09-21T06:21:55.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: pgmap v638: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-21T06:21:55.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:21:55.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:21:55.029 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:54 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: pgmap v638: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-21T06:21:55.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:21:55.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:21:55.040 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:54 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: pgmap v638: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:21:55.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:54 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:21:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 615.0 s, 1756.2 tps, lat 56.998 ms stddev 63.713 2024-09-21T06:21:57.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:56 smithi110 ceph-mon[34550]: pgmap v639: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-09-21T06:21:57.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:56 smithi107 ceph-mon[34380]: pgmap v639: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-09-21T06:21:57.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:56 smithi031 ceph-mon[31237]: pgmap v639: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 18 MiB/s wr, 1.48k op/s 2024-09-21T06:21:58.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:21:59.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:21:58 smithi110 ceph-mon[34550]: pgmap v640: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-09-21T06:21:59.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:21:58 smithi107 ceph-mon[34380]: pgmap v640: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-09-21T06:21:59.340 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:21:59.388 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:21:58 smithi031 ceph-mon[31237]: pgmap v640: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 17 MiB/s wr, 1.43k op/s 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.950+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/340403071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca010a190 msgr2=0x7f4ca010a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.950+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/340403071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca010a190 0x7f4ca010a5a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009900 tx=0x7f4c9402a440 comp rx=0 tx=0).stop 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.952+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/340403071 shutdown_connections 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.952+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/340403071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca0110420 0x7f4ca0116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.952+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/340403071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca010fad0 0x7f4ca010fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.952+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/340403071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca010a190 0x7f4ca010a5a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.952+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/340403071 >> 172.21.15.31:0/340403071 conn(0x7f4ca00780f0 msgr2=0x7f4ca01081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:59.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.953+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/340403071 shutdown_connections 2024-09-21T06:21:59.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.953+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/340403071 wait complete. 2024-09-21T06:21:59.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 Processor -- start 2024-09-21T06:21:59.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 -- start start 2024-09-21T06:21:59.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca010a190 0x7f4ca0074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 0x7f4ca006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca0110420 0x7f4ca006f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca006d6e0 con 0x7f4ca0110420 2024-09-21T06:21:59.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca006d560 con 0x7f4ca010a190 2024-09-21T06:21:59.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.954+0000 7f4ca5a3c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca006d860 con 0x7f4ca010fad0 2024-09-21T06:21:59.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.956+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca010a190 0x7f4ca0074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.958+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 0x7f4ca006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.958+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 0x7f4ca006f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38354/0 (socket says 172.21.15.31:38354) 2024-09-21T06:21:59.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.958+0000 7f4c9f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca0110420 0x7f4ca006f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.958+0000 7f4c9e7fc640 1 -- 172.21.15.31:0/3495938310 learned_addr learned my addr 172.21.15.31:0/3495938310 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:59.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.959+0000 7f4c9e7fc640 1 -- 172.21.15.31:0/3495938310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca010a190 msgr2=0x7f4ca0074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:59.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.959+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca010a190 0x7f4ca0074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.959+0000 7f4c9e7fc640 1 -- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca0110420 msgr2=0x7f4ca006f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:59.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.959+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca0110420 0x7f4ca006f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.959+0000 7f4c9e7fc640 1 -- 172.21.15.31:0/3495938310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca0081c40 con 0x7f4ca010fad0 2024-09-21T06:21:59.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.966+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 0x7f4ca006f1f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000b450 tx=0x7f4c9000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.967+0000 7f4ca4a3a640 1 -- 172.21.15.31:0/3495938310 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90015020 con 0x7f4ca010fad0 2024-09-21T06:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.967+0000 7f4ca4a3a640 1 -- 172.21.15.31:0/3495938310 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c90011d40 con 0x7f4ca010fad0 2024-09-21T06:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.967+0000 7f4ca4a3a640 1 -- 172.21.15.31:0/3495938310 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000f340 con 0x7f4ca010fad0 2024-09-21T06:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.967+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca0081f30 con 0x7f4ca010fad0 2024-09-21T06:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.967+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca00824a0 con 0x7f4ca010fad0 2024-09-21T06:21:59.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.971+0000 7f4cabfb0740 1 Processor -- start 2024-09-21T06:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.971+0000 7f4cabfb0740 1 -- start start 2024-09-21T06:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.971+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 0x564fc4b14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564fc4a8ae50 con 0x564fc47441b0 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564fc4a7a280 con 0x564fc476d070 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564fc4a7e090 con 0x564fc474c390 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 0x564fc4b14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 0x564fc4b14bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52510/0 (socket says 172.21.15.31:52510) 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 -- 172.21.15.31:0/1591959134 learned_addr learned my addr 172.21.15.31:0/1591959134 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 -- 172.21.15.31:0/1591959134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 msgr2=0x564fc4b150f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b150f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 -- 172.21.15.31:0/1591959134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 msgr2=0x564fc4b14670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:21:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b14670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.972+0000 7f4c9effd640 1 -- 172.21.15.31:0/1591959134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc47d8650 con 0x564fc47441b0 2024-09-21T06:21:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.973+0000 7f4c9effd640 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 0x564fc4b14bb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c94000c00 tx=0x7f4c94004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10d933720457c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:59.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.973+0000 7f4c85ffb640 1 -- 172.21.15.31:0/1591959134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9403f070 con 0x564fc47441b0 2024-09-21T06:21:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.974+0000 7f4c85ffb640 1 -- 172.21.15.31:0/1591959134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c94003ca0 con 0x564fc47441b0 2024-09-21T06:21:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 -- 172.21.15.31:0/1591959134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 msgr2=0x564fc4b14bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:21:59.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 0x564fc4b14bb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c94000c00 tx=0x7f4c94004310 comp rx=0 tx=0).stop 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 -- 172.21.15.31:0/1591959134 shutdown_connections 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc47441b0 0x564fc4b14bb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/1591959134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 -- 172.21.15.31:0/1591959134 >> 172.21.15.31:0/1591959134 conn(0x564fc4761460 msgr2=0x564fc4b12560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4ca4a3a640 1 -- 172.21.15.31:0/3495938310 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4c90018020 con 0x7f4ca010fad0 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 -- 172.21.15.31:0/1591959134 shutdown_connections 2024-09-21T06:21:59.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.975+0000 7f4cabfb0740 1 -- 172.21.15.31:0/1591959134 wait complete. 2024-09-21T06:21:59.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.976+0000 7f4cabfb0740 1 -- start start 2024-09-21T06:21:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 Processor -- start 2024-09-21T06:21:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b1fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b20280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc476d070 0x564fc4b2c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x564fc48db700 con 0x564fc476d070 2024-09-21T06:21:59.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564fc4a15110 con 0x564fc47441b0 2024-09-21T06:21:59.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564fc4b22e40 con 0x564fc474c390 2024-09-21T06:21:59.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.977+0000 7f4cabfb0740 10 client.0 _ll_register_callbacks cb 0x564fc4a2e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:21:59.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.978+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b20280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.978+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b20280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38370/0 (socket says 172.21.15.31:38370) 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.978+0000 7f4c9effd640 1 -- 172.21.15.31:0/3986127866 learned_addr learned my addr 172.21.15.31:0/3986127866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.978+0000 7f4c9f7fe640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b1fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.978+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc476d070 0x564fc4b2c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.979+0000 7f4c9effd640 1 -- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 msgr2=0x564fc4b1fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.979+0000 7f4c9effd640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b1fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.979+0000 7f4c9effd640 1 -- 172.21.15.31:0/3986127866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc476d070 msgr2=0x564fc4b2c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:21:59.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.979+0000 7f4c9effd640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc476d070 0x564fc4b2c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:21:59.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.979+0000 7f4c9effd640 1 -- 172.21.15.31:0/3986127866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc4a42c40 con 0x564fc474c390 2024-09-21T06:21:59.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.980+0000 7f4c9effd640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b20280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4c94031440 tx=0x7f4c9402a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11682a8ed5e3cdd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:59.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.980+0000 7f4c7effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:21:59.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.985+0000 7f4ca4a3a640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b6670 0x7f4c880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.985+0000 7f4ca4a3a640 1 -- 172.21.15.31:0/3495938310 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f4c900e7060 con 0x7f4ca010fad0 2024-09-21T06:21:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.989+0000 7f4c9f7fe640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b6670 0x7f4c880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.990+0000 7f4c7effd640 1 -- 172.21.15.31:0/3986127866 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94053060 con 0x564fc474c390 2024-09-21T06:21:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.990+0000 7f4c7effd640 1 -- 172.21.15.31:0/3986127866 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f4c94031ac0 con 0x564fc474c390 2024-09-21T06:21:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.992+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c94003ca0 con 0x564fc474c390 2024-09-21T06:21:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.993+0000 7f4c7effd640 1 -- 172.21.15.31:0/3986127866 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4c9403f070 con 0x564fc474c390 2024-09-21T06:21:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.993+0000 7f4cabfb0740 10 client.27379 fetch_fsmap learned FSMap version 21 2024-09-21T06:21:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.993+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564fc4b2ebe0 con 0x564fc474c390 2024-09-21T06:21:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4c9f7fe640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b6670 0x7f4c880b8b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x564fc4a35b60 tx=0x7f4c9800e040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4c7effd640 1 -- 172.21.15.31:0/3986127866 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c94003ca0 con 0x564fc474c390 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4c7effd640 1 -- 172.21.15.31:0/3986127866 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4c9404d030 con 0x564fc474c390 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4cabfb0740 10 client.27379 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4cabfb0740 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4cabfb0740 20 client.27379 populate_metadata read hostname 'smithi031' 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc472b9f0 0x564fc4b31f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4cabfb0740 4 client.27379 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.994+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc4a8cda0 con 0x564fc472b9f0 2024-09-21T06:21:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.995+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc472b9f0 0x564fc4b31f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:21:59.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.996+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc472b9f0 0x564fc4b31f40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e25991754369e0d server_cookie=fcaf2aef2d9ecd0f in_seq=0 out_seq=0 2024-09-21T06:21:59.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.997+0000 7f4c7effd640 10 client.27379 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:21:59.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.997+0000 7f4c7effd640 1 -- 172.21.15.31:0/3986127866 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fc4a8cda0 con 0x564fc472b9f0 2024-09-21T06:21:59.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.997+0000 7f4c7effd640 10 client.27379 handle_command_reply: tid=0 2024-09-21T06:21:59.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.997+0000 7f4cabfb0740 1 client.27379 shutdown 2024-09-21T06:22:00.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.997+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 msgr2=0x564fc4b20280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.997+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b20280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4c94031440 tx=0x7f4c9402a440 comp rx=0 tx=0).stop 2024-09-21T06:22:00.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 shutdown_connections 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc472b9f0 0x564fc4b31f40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc476d070 0x564fc4b2c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc474c390 0x564fc4b20280 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/3986127866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b1fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 >> 172.21.15.31:0/3986127866 conn(0x564fc4761460 msgr2=0x564fc4b17800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4c7effd640 0 client.27379 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:00.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:21:59.998+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 shutdown_connections 2024-09-21T06:22:00.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.000+0000 7f4cabfb0740 1 -- 172.21.15.31:0/3986127866 wait complete. 2024-09-21T06:22:00.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.000+0000 7f4cabfb0740 20 client.27379 trim_cache size 0 max 16384 2024-09-21T06:22:00.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.026+0000 7f4cabfb0740 1 Processor -- start 2024-09-21T06:22:00.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 -- start start 2024-09-21T06:22:00.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc476d070 0x564fc4b2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01c4de0 con 0x564fc474c390 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01c7660 con 0x564fc47441b0 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01c5450 con 0x564fc476d070 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b202c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39028/0 (socket says 172.21.15.31:39028) 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4c9effd640 1 -- 172.21.15.31:0/882862303 learned_addr learned my addr 172.21.15.31:0/882862303 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:00.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.028+0000 7f4c9f7fe640 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc476d070 0x564fc4b2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c9effd640 1 -- 172.21.15.31:0/882862303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc476d070 msgr2=0x564fc4b2c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c9effd640 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc476d070 0x564fc4b2c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c9effd640 1 -- 172.21.15.31:0/882862303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 msgr2=0x564fc4b21c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c9effd640 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c9effd640 1 -- 172.21.15.31:0/882862303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc4b21000 con 0x564fc47441b0 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c9effd640 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b202c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4c94044c40 tx=0x7f4c94005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82cc2b09c90c9ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c857fa640 1 -- 172.21.15.31:0/882862303 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9405d080 con 0x564fc47441b0 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c857fa640 1 -- 172.21.15.31:0/882862303 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9402ac40 con 0x564fc47441b0 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.029+0000 7f4c857fa640 1 -- 172.21.15.31:0/882862303 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94035070 con 0x564fc47441b0 2024-09-21T06:22:00.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 -- 172.21.15.31:0/882862303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 msgr2=0x564fc4b202c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b202c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4c94044c40 tx=0x7f4c94005f00 comp rx=0 tx=0).stop 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 -- 172.21.15.31:0/882862303 shutdown_connections 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc476d070 0x564fc4b2c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b21c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/882862303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc47441b0 0x564fc4b202c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 -- 172.21.15.31:0/882862303 >> 172.21.15.31:0/882862303 conn(0x564fc4761460 msgr2=0x564fc4b15650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 -- 172.21.15.31:0/882862303 shutdown_connections 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.030+0000 7f4cabfb0740 1 -- 172.21.15.31:0/882862303 wait complete. 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 -- start start 2024-09-21T06:22:00.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 Processor -- start 2024-09-21T06:22:00.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 0x564fc4b32ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b33050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01c5b20 con 0x564fc474c390 2024-09-21T06:22:00.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01c6f90 con 0x564fc476d070 2024-09-21T06:22:00.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01c61f0 con 0x564fc47441b0 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.031+0000 7f4cabfb0740 10 client.0 _ll_register_callbacks cb 0x7f4c94004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 0x564fc4b32ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b33050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 0x564fc4b32ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38392/0 (socket says 172.21.15.31:38392) 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 -- 172.21.15.31:0/861040892 learned_addr learned my addr 172.21.15.31:0/861040892 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9f7fe640 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 -- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 msgr2=0x564fc4b936e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b936e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 -- 172.21.15.31:0/861040892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 msgr2=0x564fc4b33050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b33050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 -- 172.21.15.31:0/861040892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc4b93d50 con 0x564fc47441b0 2024-09-21T06:22:00.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c9effd640 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 0x564fc4b32ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009850 tx=0x7f4c94005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571c5eb232aa170d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c7effd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c7effd640 1 -- 172.21.15.31:0/861040892 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94053030 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.032+0000 7f4c7effd640 1 -- 172.21.15.31:0/861040892 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f4c940035f0 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c9402ac40 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4c7effd640 1 -- 172.21.15.31:0/861040892 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4c94035040 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 10 client.27385 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564fc4b93d50 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4c7effd640 1 -- 172.21.15.31:0/861040892 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9402ac40 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4c7effd640 1 -- 172.21.15.31:0/861040892 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f4c9404d070 con 0x564fc47441b0 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 10 client.27385 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:00.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 20 client.27385 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:00.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc4700da0 0x564fc4b9e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:00.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 4 client.27385 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:00.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fc4a88850 con 0x564fc4700da0 2024-09-21T06:22:00.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.033+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc4700da0 0x564fc4b9e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:00.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.034+0000 7f4c7effd640 10 client.27385 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:00.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.034+0000 7f4c9e7fc640 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc4700da0 0x564fc4b9e540 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64f0ba585d1c61f server_cookie=f568fd862a18a899 in_seq=0 out_seq=0 2024-09-21T06:22:00.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4c7effd640 1 -- 172.21.15.31:0/861040892 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7190 (crc 0 0 0) 0x564fc4a88850 con 0x564fc4700da0 2024-09-21T06:22:00.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4c7effd640 10 client.27385 handle_command_reply: tid=0 2024-09-21T06:22:00.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 client.27385 shutdown 2024-09-21T06:22:00.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 msgr2=0x564fc4b32ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 0x564fc4b32ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009850 tx=0x7f4c94005fd0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 shutdown_connections 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x564fc4700da0 0x564fc4b9e540 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564fc476d070 0x564fc4b936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564fc474c390 0x564fc4b33050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 --2- 172.21.15.31:0/861040892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564fc47441b0 0x564fc4b32ab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 >> 172.21.15.31:0/861040892 conn(0x564fc4761460 msgr2=0x564fc4b16010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4c7effd640 0 client.27385 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.035+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 shutdown_connections 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.036+0000 7f4cabfb0740 1 -- 172.21.15.31:0/861040892 wait complete. 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.036+0000 7f4cabfb0740 20 client.27385 trim_cache size 0 max 16384 2024-09-21T06:22:00.041 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:22:00.042 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:22:00.044 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:22:00.047 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.051 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:22:00.053 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:22:00.056 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:00.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:00.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:00.058 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:22:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b6670 msgr2=0x7f4c880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b6670 0x7f4c880b8b80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x564fc4a35b60 tx=0x7f4c9800e040 comp rx=0 tx=0).stop 2024-09-21T06:22:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 msgr2=0x7f4ca006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 0x7f4ca006f1f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000b450 tx=0x7f4c9000b910 comp rx=0 tx=0).stop 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 shutdown_connections 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c880b6670 0x7f4c880b8b80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ca0110420 0x7f4ca006f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca010fad0 0x7f4ca006f1f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 --2- 172.21.15.31:0/3495938310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ca010a190 0x7f4ca0074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 >> 172.21.15.31:0/3495938310 conn(0x7f4ca00780f0 msgr2=0x7f4ca010dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 shutdown_connections 2024-09-21T06:22:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:00.037+0000 7f4ca5a3c640 1 -- 172.21.15.31:0/3495938310 wait complete. 2024-09-21T06:22:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 620.0 s, 1978.8 tps, lat 49.847 ms stddev 57.638 2024-09-21T06:22:01.185 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:22:01.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:22:01.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:01 smithi110 ceph-mon[34550]: pgmap v641: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-09-21T06:22:01.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:01 smithi107 ceph-mon[34380]: pgmap v641: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-09-21T06:22:01.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:01 smithi031 ceph-mon[31237]: pgmap v641: 97 pgs: 97 active+clean; 5.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2024-09-21T06:22:01.715 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 -- 172.21.15.31:0/3541735893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c108d10 msgr2=0x7fea3c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3541735893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c108d10 0x7fea3c10f540 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fea34009860 tx=0x7fea3402a340 comp rx=0 tx=0).stop 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 -- 172.21.15.31:0/3541735893 shutdown_connections 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3541735893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c108d10 0x7fea3c10f540 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3541735893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea3c1083c0 0x7fea3c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3541735893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea3c102a80 0x7fea3c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 -- 172.21.15.31:0/3541735893 >> 172.21.15.31:0/3541735893 conn(0x7fea3c079740 msgr2=0x7fea3c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 -- 172.21.15.31:0/3541735893 shutdown_connections 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.224+0000 7fea40dc1640 1 -- 172.21.15.31:0/3541735893 wait complete. 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.225+0000 7fea40dc1640 1 Processor -- start 2024-09-21T06:22:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.225+0000 7fea40dc1640 1 -- start start 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.225+0000 7fea40dc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 0x7fea3c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.225+0000 7fea40dc1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea3c1083c0 0x7fea3c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea40dc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea3c108d10 0x7fea3c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea40dc1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c112980 con 0x7fea3c102a80 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea40dc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c112800 con 0x7fea3c108d10 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea40dc1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c112b00 con 0x7fea3c1083c0 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 0x7fea3c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 0x7fea3c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52546/0 (socket says 172.21.15.31:52546) 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 -- 172.21.15.31:0/3704765920 learned_addr learned my addr 172.21.15.31:0/3704765920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea39d9b640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea3c1083c0 0x7fea3c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3ad9d640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea3c108d10 0x7fea3c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 -- 172.21.15.31:0/3704765920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea3c1083c0 msgr2=0x7fea3c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea3c1083c0 0x7fea3c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 -- 172.21.15.31:0/3704765920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea3c108d10 msgr2=0x7fea3c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea3c108d10 0x7fea3c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 -- 172.21.15.31:0/3704765920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c1a03b0 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea3a59c640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 0x7fea3c19a440 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00eb90 tx=0x7fea2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c004d20 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c00ee90 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c010670 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.226+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c1a06a0 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.227+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1a6e00 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.228+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c102e90 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.230+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fea2c010870 con 0x7fea3c102a80 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.231+0000 7fea2b7fe640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea240b6680 0x7fea240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:02.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.231+0000 7fea39d9b640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea240b6680 0x7fea240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.231+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fea2c0e7ec0 con 0x7fea3c102a80 2024-09-21T06:22:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.231+0000 7fea39d9b640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea240b6680 0x7fea240b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea300096c0 tx=0x7fea30009260 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:02.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.234+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea2c056710 con 0x7fea3c102a80 2024-09-21T06:22:02.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.478+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea3c19b570 con 0x7fea3c102a80 2024-09-21T06:22:02.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.478+0000 7fea2b7fe640 1 -- 172.21.15.31:0/3704765920 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7fea2c0ae450 con 0x7fea3c102a80 2024-09-21T06:22:02.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:22:02.481 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:22:02.481 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:22:02.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea240b6680 msgr2=0x7fea240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea240b6680 0x7fea240b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea300096c0 tx=0x7fea30009260 comp rx=0 tx=0).stop 2024-09-21T06:22:02.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 msgr2=0x7fea3c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:02.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 0x7fea3c19a440 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00eb90 tx=0x7fea2c00c6a0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 shutdown_connections 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea3c108d10 0x7fea3c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea3c1083c0 0x7fea3c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fea240b6680 0x7fea240b8b90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 --2- 172.21.15.31:0/3704765920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea3c102a80 0x7fea3c19a440 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 >> 172.21.15.31:0/3704765920 conn(0x7fea3c079740 msgr2=0x7fea3c06bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 shutdown_connections 2024-09-21T06:22:02.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:02.481+0000 7fea40dc1640 1 -- 172.21.15.31:0/3704765920 wait complete. 2024-09-21T06:22:03.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:03 smithi031 ceph-mon[31237]: pgmap v642: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2024-09-21T06:22:03.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:03 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3704765920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:03.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:22:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:03 smithi107 ceph-mon[34380]: pgmap v642: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2024-09-21T06:22:03.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:03 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3704765920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:03.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:03 smithi110 ceph-mon[34550]: pgmap v642: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 18 MiB/s wr, 1.52k op/s 2024-09-21T06:22:03.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:03 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3704765920' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:03.653 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:04.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 -- 172.21.15.31:0/2718115100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520102a90 msgr2=0x7fa520102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 --2- 172.21.15.31:0/2718115100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520102a90 0x7fa520102ea0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa51000dc90 tx=0x7fa51002e370 comp rx=0 tx=0).stop 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 -- 172.21.15.31:0/2718115100 shutdown_connections 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 --2- 172.21.15.31:0/2718115100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520108d20 0x7fa52010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 --2- 172.21.15.31:0/2718115100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5201083d0 0x7fa5201087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 --2- 172.21.15.31:0/2718115100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520102a90 0x7fa520102ea0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 -- 172.21.15.31:0/2718115100 >> 172.21.15.31:0/2718115100 conn(0x7fa520079730 msgr2=0x7fa520079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 -- 172.21.15.31:0/2718115100 shutdown_connections 2024-09-21T06:22:04.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.269+0000 7fa52590e640 1 -- 172.21.15.31:0/2718115100 wait complete. 2024-09-21T06:22:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.270+0000 7fa52590e640 1 Processor -- start 2024-09-21T06:22:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.270+0000 7fa52590e640 1 -- start start 2024-09-21T06:22:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.270+0000 7fa52590e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 0x7fa5201982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa52590e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5201083d0 0x7fa520198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa52590e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520108d20 0x7fa52019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa52590e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111920 con 0x7fa520102a90 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa52590e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201117a0 con 0x7fa5201083d0 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa52590e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111aa0 con 0x7fa520108d20 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 0x7fa5201982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 0x7fa5201982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52570/0 (socket says 172.21.15.31:52570) 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 -- 172.21.15.31:0/1260629641 learned_addr learned my addr 172.21.15.31:0/1260629641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51f7fe640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520108d20 0x7fa52019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa517fff640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5201083d0 0x7fa520198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 -- 172.21.15.31:0/1260629641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520108d20 msgr2=0x7fa52019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520108d20 0x7fa52019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 -- 172.21.15.31:0/1260629641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5201083d0 msgr2=0x7fa520198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5201083d0 0x7fa520198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.271+0000 7fa51effd640 1 -- 172.21.15.31:0/1260629641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52019e180 con 0x7fa520102a90 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.272+0000 7fa51effd640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 0x7fa5201982d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa510000c00 tx=0x7fa51000d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.272+0000 7fa52490c640 1 -- 172.21.15.31:0/1260629641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5100050b0 con 0x7fa520102a90 2024-09-21T06:22:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.272+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52019e410 con 0x7fa520102a90 2024-09-21T06:22:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.272+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52019e940 con 0x7fa520102a90 2024-09-21T06:22:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.272+0000 7fa52490c640 1 -- 172.21.15.31:0/1260629641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa510044070 con 0x7fa520102a90 2024-09-21T06:22:04.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.272+0000 7fa52490c640 1 -- 172.21.15.31:0/1260629641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51003f6e0 con 0x7fa520102a90 2024-09-21T06:22:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.276+0000 7fa52490c640 1 -- 172.21.15.31:0/1260629641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa51003f880 con 0x7fa520102a90 2024-09-21T06:22:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.276+0000 7fa52be82740 1 Processor -- start 2024-09-21T06:22:04.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.276+0000 7fa52be82740 1 -- start start 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52be82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d31962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52be82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d3196800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52be82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dce390 0x5634d3196d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52be82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5634d310cea0 con 0x5634d2dc61b0 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52be82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634d30fc2d0 con 0x5634d2def070 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52be82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5634d31000e0 con 0x5634d2dce390 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa51effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d3196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d31962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d31962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39072/0 (socket says 172.21.15.31:39072) 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa51f7fe640 1 -- 172.21.15.31:0/1638106660 learned_addr learned my addr 172.21.15.31:0/1638106660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa517fff640 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dce390 0x5634d3196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa517fff640 1 -- 172.21.15.31:0/1638106660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 msgr2=0x5634d31962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa517fff640 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d31962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa517fff640 1 -- 172.21.15.31:0/1638106660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 msgr2=0x5634d3196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa517fff640 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d3196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa52490c640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa50c0b6730 0x7fa50c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.277+0000 7fa517fff640 1 -- 172.21.15.31:0/1638106660 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d2e5a650 con 0x5634d2dce390 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.278+0000 7fa52490c640 1 -- 172.21.15.31:0/1260629641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fa51010b690 con 0x7fa520102a90 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.278+0000 7fa517fff640 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dce390 0x5634d3196d40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa52006b680 tx=0x7fa50800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c52a89e7c3f24e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.278+0000 7fa51f7fe640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa50c0b6730 0x7fa50c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.278+0000 7fa515ffb640 1 -- 172.21.15.31:0/1638106660 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508005220 con 0x5634d2dce390 2024-09-21T06:22:04.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.278+0000 7fa515ffb640 1 -- 172.21.15.31:0/1638106660 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508004c50 con 0x5634d2dce390 2024-09-21T06:22:04.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.278+0000 7fa515ffb640 1 -- 172.21.15.31:0/1638106660 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508020730 con 0x5634d2dce390 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa51f7fe640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa50c0b6730 0x7fa50c0b8c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa5180116b0 tx=0x7fa518009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 -- 172.21.15.31:0/1638106660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dce390 msgr2=0x5634d3196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dce390 0x5634d3196d40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa52006b680 tx=0x7fa50800f790 comp rx=0 tx=0).stop 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 -- 172.21.15.31:0/1638106660 shutdown_connections 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dce390 0x5634d3196d40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d3196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 --2- 172.21.15.31:0/1638106660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d31962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 -- 172.21.15.31:0/1638106660 >> 172.21.15.31:0/1638106660 conn(0x5634d2de3460 msgr2=0x5634d31941b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 -- 172.21.15.31:0/1638106660 shutdown_connections 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.279+0000 7fa52be82740 1 -- 172.21.15.31:0/1638106660 wait complete. 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.280+0000 7fa52be82740 1 -- start start 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.280+0000 7fa52be82740 1 Processor -- start 2024-09-21T06:22:04.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.280+0000 7fa52be82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.280+0000 7fa52be82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa52be82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2def070 0x5634d31add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa52be82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5634d2f5d700 con 0x5634d2def070 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa52be82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5634d3097160 con 0x5634d2dce390 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa52be82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5634d31a4aa0 con 0x5634d2dc61b0 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa52be82740 10 client.0 _ll_register_callbacks cb 0x5634d30b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2def070 0x5634d31add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2def070 0x5634d31add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52586/0 (socket says 172.21.15.31:52586) 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51f7fe640 1 -- 172.21.15.31:0/2234790116 learned_addr learned my addr 172.21.15.31:0/2234790116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa517fff640 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 -- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 msgr2=0x5634d31a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 -- 172.21.15.31:0/2234790116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2def070 msgr2=0x5634d31add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2def070 0x5634d31add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 -- 172.21.15.31:0/2234790116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d30c4c90 con 0x5634d2dc61b0 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa51effd640 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a19e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa510108e20 tx=0x7fa510005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610cb0b28c1816e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.281+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.282+0000 7fa4feffd640 1 -- 172.21.15.31:0/2234790116 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5100cdb70 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.283+0000 7fa4feffd640 1 -- 172.21.15.31:0/2234790116 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fa5100d0780 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.283+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa508004c50 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.283+0000 7fa4feffd640 1 -- 172.21.15.31:0/2234790116 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa5101092b0 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa52be82740 10 client.27403 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5634d31b07b0 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa4feffd640 1 -- 172.21.15.31:0/2234790116 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5100d1030 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa4feffd640 1 -- 172.21.15.31:0/2234790116 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa51010ece0 con 0x5634d2dc61b0 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa52be82740 10 client.27403 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa52be82740 10 client.27403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa52be82740 20 client.27403 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.284+0000 7fa52be82740 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2dad9f0 0x5634d31b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.285+0000 7fa52be82740 4 client.27403 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.285+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa508004c50 con 0x5634d2dad9f0 2024-09-21T06:22:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.285+0000 7fa517fff640 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2dad9f0 0x5634d31b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.285+0000 7fa4feffd640 10 client.27403 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.285+0000 7fa517fff640 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2dad9f0 0x5634d31b3c10 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7efaeec91cf03d9 server_cookie=98800ae49a987b85 in_seq=0 out_seq=0 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.286+0000 7fa4feffd640 1 -- 172.21.15.31:0/2234790116 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa508004c50 con 0x5634d2dad9f0 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.286+0000 7fa4feffd640 10 client.27403 handle_command_reply: tid=0 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.286+0000 7fa52be82740 1 client.27403 shutdown 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 msgr2=0x5634d31a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a19e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa510108e20 tx=0x7fa510005250 comp rx=0 tx=0).stop 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 shutdown_connections 2024-09-21T06:22:04.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2dad9f0 0x5634d31b3c10 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa4feffd640 0 client.27403 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2def070 0x5634d31add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 --2- 172.21.15.31:0/2234790116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a19e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 >> 172.21.15.31:0/2234790116 conn(0x5634d2de3460 msgr2=0x5634d31994a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 shutdown_connections 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 1 -- 172.21.15.31:0/2234790116 wait complete. 2024-09-21T06:22:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.287+0000 7fa52be82740 20 client.27403 trim_cache size 0 max 16384 2024-09-21T06:22:04.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 Processor -- start 2024-09-21T06:22:04.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 -- start start 2024-09-21T06:22:04.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 0x5634d3193760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d3193ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201ab410 con 0x5634d2dce390 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201adc90 con 0x5634d2def070 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa52be82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201aba80 con 0x5634d2dc61b0 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa517fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 0x5634d3193760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa51effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa517fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 0x5634d3193760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52600/0 (socket says 172.21.15.31:52600) 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.309+0000 7fa517fff640 1 -- 172.21.15.31:0/2849275164 learned_addr learned my addr 172.21.15.31:0/2849275164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa51f7fe640 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d3193ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa517fff640 1 -- 172.21.15.31:0/2849275164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 msgr2=0x5634d31a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa517fff640 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa517fff640 1 -- 172.21.15.31:0/2849275164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 msgr2=0x5634d3193ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa517fff640 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d3193ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa517fff640 1 -- 172.21.15.31:0/2849275164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d319fee0 con 0x5634d2dce390 2024-09-21T06:22:04.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa517fff640 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 0x5634d3193760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa508005690 tx=0x7fa50800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412b5240819e1197 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa5157fa640 1 -- 172.21.15.31:0/2849275164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508020440 con 0x5634d2dce390 2024-09-21T06:22:04.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa5157fa640 1 -- 172.21.15.31:0/2849275164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508020a90 con 0x5634d2dce390 2024-09-21T06:22:04.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.310+0000 7fa5157fa640 1 -- 172.21.15.31:0/2849275164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50802c950 con 0x5634d2dce390 2024-09-21T06:22:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 -- 172.21.15.31:0/2849275164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 msgr2=0x5634d3193760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 0x5634d3193760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa508005690 tx=0x7fa50800f790 comp rx=0 tx=0).stop 2024-09-21T06:22:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 -- 172.21.15.31:0/2849275164 shutdown_connections 2024-09-21T06:22:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2def070 0x5634d3193ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dce390 0x5634d3193760 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 --2- 172.21.15.31:0/2849275164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2dc61b0 0x5634d31a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 -- 172.21.15.31:0/2849275164 >> 172.21.15.31:0/2849275164 conn(0x5634d2de3460 msgr2=0x5634d31a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.311+0000 7fa52be82740 1 -- 172.21.15.31:0/2849275164 shutdown_connections 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.312+0000 7fa52be82740 1 -- 172.21.15.31:0/2849275164 wait complete. 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.312+0000 7fa52be82740 1 -- start start 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.312+0000 7fa52be82740 1 Processor -- start 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d31a6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2def070 0x5634d321d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201ac150 con 0x5634d2dc61b0 2024-09-21T06:22:04.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201ad5c0 con 0x5634d2dce390 2024-09-21T06:22:04.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201ac820 con 0x5634d2def070 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa51effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d31a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa51effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d31a6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52606/0 (socket says 172.21.15.31:52606) 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa52be82740 10 client.0 _ll_register_callbacks cb 0x7fa510005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.313+0000 7fa51effd640 1 -- 172.21.15.31:0/10453085 learned_addr learned my addr 172.21.15.31:0/10453085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa517fff640 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa51effd640 1 -- 172.21.15.31:0/10453085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2def070 msgr2=0x5634d321d5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa51effd640 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2def070 0x5634d321d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa51effd640 1 -- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 msgr2=0x5634d31a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa51effd640 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa51effd640 1 -- 172.21.15.31:0/10453085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d321db10 con 0x5634d2dc61b0 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa51effd640 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d31a6390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa510010090 tx=0x7fa5100d0420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623a60fb4482268e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.314+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:04.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.315+0000 7fa4feffd640 1 -- 172.21.15.31:0/10453085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5100f8070 con 0x5634d2dc61b0 2024-09-21T06:22:04.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.315+0000 7fa4feffd640 1 -- 172.21.15.31:0/10453085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7fa5100d73a0 con 0x5634d2dc61b0 2024-09-21T06:22:04.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.315+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa508020a90 con 0x5634d2dc61b0 2024-09-21T06:22:04.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.315+0000 7fa4feffd640 1 -- 172.21.15.31:0/10453085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa5100d0c90 con 0x5634d2dc61b0 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 10 client.17565 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5634d321ed20 con 0x5634d2dc61b0 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa4feffd640 1 -- 172.21.15.31:0/10453085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508020a90 con 0x5634d2dc61b0 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa4feffd640 1 -- 172.21.15.31:0/10453085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa5100da480 con 0x5634d2dc61b0 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 10 client.17565 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 20 client.17565 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2d82da0 0x5634d32214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 4 client.17565 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:04.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.316+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634d310edf0 con 0x5634d2d82da0 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.317+0000 7fa517fff640 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2d82da0 0x5634d32214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa4feffd640 10 client.17565 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa517fff640 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2d82da0 0x5634d32214e0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb8dc59a433dd56 server_cookie=aeffbd4773cd5cc in_seq=0 out_seq=0 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa4feffd640 1 -- 172.21.15.31:0/10453085 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634d310edf0 con 0x5634d2d82da0 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa4feffd640 10 client.17565 handle_command_reply: tid=0 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 client.17565 shutdown 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 msgr2=0x5634d31a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d31a6390 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa510010090 tx=0x7fa5100d0420 comp rx=0 tx=0).stop 2024-09-21T06:22:04.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 shutdown_connections 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5634d2d82da0 0x5634d32214e0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5634d2def070 0x5634d321d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5634d2dce390 0x5634d31a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 --2- 172.21.15.31:0/10453085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5634d2dc61b0 0x5634d31a6390 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 >> 172.21.15.31:0/10453085 conn(0x5634d2de3460 msgr2=0x5634d3197cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.318+0000 7fa4feffd640 0 client.17565 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 shutdown_connections 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52be82740 1 -- 172.21.15.31:0/10453085 wait complete. 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52be82740 20 client.17565 trim_cache size 0 max 16384 2024-09-21T06:22:04.325 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:22:04.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa50c0b6730 msgr2=0x7fa50c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa50c0b6730 0x7fa50c0b8c40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa5180116b0 tx=0x7fa518009500 comp rx=0 tx=0).stop 2024-09-21T06:22:04.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 msgr2=0x7fa5201982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 0x7fa5201982d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa510000c00 tx=0x7fa51000d580 comp rx=0 tx=0).stop 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 shutdown_connections 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa520108d20 0x7fa52019db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5201083d0 0x7fa520198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa50c0b6730 0x7fa50c0b8c40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 --2- 172.21.15.31:0/1260629641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa520102a90 0x7fa5201982d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.319+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 >> 172.21.15.31:0/1260629641 conn(0x7fa520079730 msgr2=0x7fa520106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.320+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 shutdown_connections 2024-09-21T06:22:04.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:04.320+0000 7fa52590e640 1 -- 172.21.15.31:0/1260629641 wait complete. 2024-09-21T06:22:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 625.0 s, 810.0 tps, lat 123.952 ms stddev 299.516 2024-09-21T06:22:05.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:05 smithi107 ceph-mon[34380]: pgmap v643: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-21T06:22:05.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:05 smithi031 ceph-mon[31237]: pgmap v643: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-21T06:22:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:05 smithi110 ceph-mon[34550]: pgmap v643: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 791 op/s 2024-09-21T06:22:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 -- 172.21.15.31:0/1106932038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 msgr2=0x7f121010ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 --2- 172.21.15.31:0/1106932038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f121010ec10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1208009ca0 tx=0x7f120802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 -- 172.21.15.31:0/1106932038 shutdown_connections 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 --2- 172.21.15.31:0/1106932038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f121010ec10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 --2- 172.21.15.31:0/1106932038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1210107a70 0x7f1210107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 --2- 172.21.15.31:0/1106932038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12101023e0 0x7f12101027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 -- 172.21.15.31:0/1106932038 >> 172.21.15.31:0/1106932038 conn(0x7f1210077770 msgr2=0x7f1210077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 -- 172.21.15.31:0/1106932038 shutdown_connections 2024-09-21T06:22:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.585+0000 7f12181ba640 1 -- 172.21.15.31:0/1106932038 wait complete. 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.586+0000 7f12181ba640 1 Processor -- start 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.586+0000 7f12181ba640 1 -- start start 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f12181ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 0x7f12101a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f12181ba640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1210107a70 0x7f12101a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f12181ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f12101a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f12181ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1210111820 con 0x7f12101083c0 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f12181ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12101116a0 con 0x7f12101023e0 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f12181ba640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12101119a0 con 0x7f1210107a70 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 0x7f12101a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215755640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1210107a70 0x7f12101a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1216757640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f12101a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 0x7f12101a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39114/0 (socket says 172.21.15.31:39114) 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215755640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1210107a70 0x7f12101a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38456/0 (socket says 172.21.15.31:38456) 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 -- 172.21.15.31:0/2066541208 learned_addr learned my addr 172.21.15.31:0/2066541208 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 -- 172.21.15.31:0/2066541208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1210107a70 msgr2=0x7f12101a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1210107a70 0x7f12101a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 -- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 msgr2=0x7f12101a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f12101a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 -- 172.21.15.31:0/2066541208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12101a6a00 con 0x7f12101023e0 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1216757640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f12101a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.587+0000 7f1215f56640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 0x7f12101a0ad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f120c00ea40 tx=0x7f120c00ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.588+0000 7f1206ffd640 1 -- 172.21.15.31:0/2066541208 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00cd40 con 0x7f12101023e0 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.588+0000 7f1206ffd640 1 -- 172.21.15.31:0/2066541208 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120c0054d0 con 0x7f12101023e0 2024-09-21T06:22:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.588+0000 7f1206ffd640 1 -- 172.21.15.31:0/2066541208 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c016670 con 0x7f12101023e0 2024-09-21T06:22:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.588+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12101a6cf0 con 0x7f12101023e0 2024-09-21T06:22:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.588+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12101ad480 con 0x7f12101023e0 2024-09-21T06:22:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.592+0000 7f1206ffd640 1 -- 172.21.15.31:0/2066541208 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f120c004e70 con 0x7f12101023e0 2024-09-21T06:22:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 Processor -- start 2024-09-21T06:22:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 -- start start 2024-09-21T06:22:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f1206ffd640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11fc0b66f0 0x7f11fc0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34721b0 0x559ff38179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34492f0 0x559ff3817ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34514d0 0x559ff3818430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559ff378c6f0 con 0x559ff34514d0 2024-09-21T06:22:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559ff377bb20 con 0x559ff34721b0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f121f201740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559ff377f930 con 0x559ff34492f0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f1215755640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34492f0 0x559ff3817ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f1215755640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34492f0 0x559ff3817ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38464/0 (socket says 172.21.15.31:38464) 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f1215755640 1 -- 172.21.15.31:0/2438028634 learned_addr learned my addr 172.21.15.31:0/2438028634 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.593+0000 7f1206ffd640 1 -- 172.21.15.31:0/2066541208 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f120c0e7430 con 0x7f12101023e0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1216757640 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34514d0 0x559ff3818430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1215f56640 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34721b0 0x559ff38179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1216757640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11fc0b66f0 0x7f11fc0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1215f56640 1 -- 172.21.15.31:0/2438028634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34492f0 msgr2=0x559ff3817ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1215f56640 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34492f0 0x559ff3817ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1215f56640 1 -- 172.21.15.31:0/2438028634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34514d0 msgr2=0x559ff3818430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1215f56640 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34514d0 0x559ff3818430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.594+0000 7f1215f56640 1 -- 172.21.15.31:0/2438028634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ff3741530 con 0x559ff34721b0 2024-09-21T06:22:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.595+0000 7f1216757640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11fc0b66f0 0x7f11fc0b8c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f120802a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.595+0000 7f1215f56640 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34721b0 0x559ff38179b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f120c0e4b90 tx=0x7f120c0a9af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a365bf65cc5cf8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.596+0000 7f11fbfff640 1 -- 172.21.15.31:0/2438028634 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0c8910 con 0x559ff34721b0 2024-09-21T06:22:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.596+0000 7f11fbfff640 1 -- 172.21.15.31:0/2438028634 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120c0049a0 con 0x559ff34721b0 2024-09-21T06:22:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.596+0000 7f11fbfff640 1 -- 172.21.15.31:0/2438028634 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c005160 con 0x559ff34721b0 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 -- 172.21.15.31:0/2438028634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34721b0 msgr2=0x559ff38179b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34721b0 0x559ff38179b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f120c0e4b90 tx=0x7f120c0a9af0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 -- 172.21.15.31:0/2438028634 shutdown_connections 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34514d0 0x559ff3818430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34492f0 0x559ff3817ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 --2- 172.21.15.31:0/2438028634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34721b0 0x559ff38179b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.597+0000 7f121f201740 1 -- 172.21.15.31:0/2438028634 >> 172.21.15.31:0/2438028634 conn(0x559ff3466300 msgr2=0x559ff38158a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.598+0000 7f121f201740 1 -- 172.21.15.31:0/2438028634 shutdown_connections 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.598+0000 7f121f201740 1 -- 172.21.15.31:0/2438028634 wait complete. 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.598+0000 7f121f201740 1 -- start start 2024-09-21T06:22:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.598+0000 7f121f201740 1 Processor -- start 2024-09-21T06:22:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f121f201740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff3823150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f121f201740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3823690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f121f201740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34721b0 0x559ff382f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f1215f56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff3823150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f1215f56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff3823150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39128/0 (socket says 172.21.15.31:39128) 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f1215755640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3823690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f1215755640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3823690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38474/0 (socket says 172.21.15.31:38474) 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f1215f56640 1 -- 172.21.15.31:0/4253423825 learned_addr learned my addr 172.21.15.31:0/4253423825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559ff36b3ab0 con 0x559ff34721b0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.599+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559ff3721730 con 0x559ff34492f0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559ff3826130 con 0x559ff34514d0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f1216757640 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34721b0 0x559ff382f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f121f201740 10 client.0 _ll_register_callbacks cb 0x559ff37756b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f1215f56640 1 -- 172.21.15.31:0/4253423825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 msgr2=0x559ff3823690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f1215f56640 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3823690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f1215f56640 1 -- 172.21.15.31:0/4253423825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34721b0 msgr2=0x559ff382f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f1215f56640 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34721b0 0x559ff382f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.600+0000 7f1215f56640 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ff34dd790 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f1215f56640 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff3823150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f120c0e51c0 tx=0x7f120c0a9880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfadd91d5e61025 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f11f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f11f97fa640 1 -- 172.21.15.31:0/4253423825 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0c8910 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f11f97fa640 1 -- 172.21.15.31:0/4253423825 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f120c0ca220 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559ff378e640 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f11f97fa640 1 -- 172.21.15.31:0/4253423825 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f120c0ca440 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f121f201740 10 client.27365 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ff3556620 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f11f97fa640 1 -- 172.21.15.31:0/4253423825 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559ff378e640 con 0x559ff34492f0 2024-09-21T06:22:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.601+0000 7f11f97fa640 1 -- 172.21.15.31:0/4253423825 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f120c00b040 con 0x559ff34492f0 2024-09-21T06:22:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f121f201740 10 client.27365 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f121f201740 10 client.27365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f121f201740 20 client.27365 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f121f201740 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff3430d80 0x559ff3835030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f121f201740 4 client.27365 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ff378a0f0 con 0x559ff3430d80 2024-09-21T06:22:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.602+0000 7f1215755640 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff3430d80 0x559ff3835030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.603+0000 7f11f97fa640 10 client.27365 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.603+0000 7f1215755640 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff3430d80 0x559ff3835030 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=772f0f348864052c server_cookie=3026424a7efbfdb9 in_seq=0 out_seq=0 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f11f97fa640 1 -- 172.21.15.31:0/4253423825 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ff378a0f0 con 0x559ff3430d80 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f11f97fa640 10 client.27365 handle_command_reply: tid=0 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 client.27365 shutdown 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 msgr2=0x559ff3823150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff3823150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f120c0e51c0 tx=0x7f120c0a9880 comp rx=0 tx=0).stop 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 shutdown_connections 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff3430d80 0x559ff3835030 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f11f97fa640 0 client.27365 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34721b0 0x559ff382f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3823690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 --2- 172.21.15.31:0/4253423825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff3823150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 >> 172.21.15.31:0/4253423825 conn(0x559ff3466300 msgr2=0x559ff38189e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 shutdown_connections 2024-09-21T06:22:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 1 -- 172.21.15.31:0/4253423825 wait complete. 2024-09-21T06:22:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.604+0000 7f121f201740 20 client.27365 trim_cache size 0 max 16384 2024-09-21T06:22:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.625+0000 7f121f201740 1 Processor -- start 2024-09-21T06:22:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.625+0000 7f121f201740 1 -- start start 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.625+0000 7f121f201740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3430d80 0x559ff3823630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.625+0000 7f121f201740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34492f0 0x559ff3825030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f121f201740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff38190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1215f56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3430d80 0x559ff3823630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1215755640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34492f0 0x559ff3825030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1215f56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3430d80 0x559ff3823630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39134/0 (socket says 172.21.15.31:39134) 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1215755640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34492f0 0x559ff3825030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52654/0 (socket says 172.21.15.31:52654) 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f121f201740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b3e20 con 0x559ff34492f0 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f121f201740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b66a0 con 0x559ff3430d80 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f121f201740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b4490 con 0x559ff34514d0 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff38190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff38190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38490/0 (socket says 172.21.15.31:38490) 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 -- 172.21.15.31:0/2714582776 learned_addr learned my addr 172.21.15.31:0/2714582776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 -- 172.21.15.31:0/2714582776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3430d80 msgr2=0x559ff3823630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3430d80 0x559ff3823630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 -- 172.21.15.31:0/2714582776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34492f0 msgr2=0x559ff3825030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34492f0 0x559ff3825030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.626+0000 7f1216757640 1 -- 172.21.15.31:0/2714582776 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ff3825ce0 con 0x559ff34514d0 2024-09-21T06:22:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.627+0000 7f1216757640 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff38190f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f120802af70 tx=0x7f1208036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96cc81d43dcc303 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.627+0000 7f11fb7fe640 1 -- 172.21.15.31:0/2714582776 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208003b70 con 0x559ff34514d0 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.627+0000 7f11fb7fe640 1 -- 172.21.15.31:0/2714582776 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1208051070 con 0x559ff34514d0 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.627+0000 7f11fb7fe640 1 -- 172.21.15.31:0/2714582776 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120802f9a0 con 0x559ff34514d0 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.628+0000 7f121f201740 1 -- 172.21.15.31:0/2714582776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 msgr2=0x559ff38190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.628+0000 7f121f201740 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff38190f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f120802af70 tx=0x7f1208036000 comp rx=0 tx=0).stop 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 -- 172.21.15.31:0/2714582776 shutdown_connections 2024-09-21T06:22:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff38190f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff34492f0 0x559ff3825030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 --2- 172.21.15.31:0/2714582776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff3430d80 0x559ff3823630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 -- 172.21.15.31:0/2714582776 >> 172.21.15.31:0/2714582776 conn(0x559ff3466300 msgr2=0x559ff38189e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 -- 172.21.15.31:0/2714582776 shutdown_connections 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 -- 172.21.15.31:0/2714582776 wait complete. 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 -- start start 2024-09-21T06:22:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.629+0000 7f121f201740 1 Processor -- start 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff3430d80 0x559ff3835c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff38361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3897960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b4b60 con 0x559ff3430d80 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b5fd0 con 0x559ff34492f0 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b5230 con 0x559ff34514d0 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f121f201740 10 client.0 _ll_register_callbacks cb 0x7f120c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f1215755640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff38361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f1215f56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff3430d80 0x559ff3835c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f1215755640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff38361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39140/0 (socket says 172.21.15.31:39140) 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.630+0000 7f1215755640 1 -- 172.21.15.31:0/1756108514 learned_addr learned my addr 172.21.15.31:0/1756108514 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.631+0000 7f1216757640 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3897960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.631+0000 7f1215f56640 1 -- 172.21.15.31:0/1756108514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 msgr2=0x559ff3897960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.631+0000 7f1215f56640 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3897960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.631+0000 7f1215f56640 1 -- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 msgr2=0x559ff38361d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.631+0000 7f1215f56640 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff38361d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.631+0000 7f1215f56640 1 -- 172.21.15.31:0/1756108514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ff3898000 con 0x559ff3430d80 2024-09-21T06:22:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.632+0000 7f1215f56640 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff3430d80 0x559ff3835c10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f120c005820 tx=0x7f120c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa31b12f200b005a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.632+0000 7f11f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.632+0000 7f11f97fa640 1 -- 172.21.15.31:0/1756108514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c096e20 con 0x559ff3430d80 2024-09-21T06:22:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.632+0000 7f11f97fa640 1 -- 172.21.15.31:0/1756108514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f120c0ac090 con 0x559ff3430d80 2024-09-21T06:22:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.632+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1208051070 con 0x559ff3430d80 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.633+0000 7f11f97fa640 1 -- 172.21.15.31:0/1756108514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f120c0c8910 con 0x559ff3430d80 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.633+0000 7f121f201740 10 client.17583 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.633+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559ff389f3f0 con 0x559ff3430d80 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.633+0000 7f11f97fa640 1 -- 172.21.15.31:0/1756108514 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1208051070 con 0x559ff3430d80 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.633+0000 7f11f97fa640 1 -- 172.21.15.31:0/1756108514 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f120c094ab0 con 0x559ff3430d80 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f121f201740 10 client.17583 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f121f201740 10 client.17583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f121f201740 20 client.17583 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f121f201740 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff34721b0 0x559ff38a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f121f201740 4 client.17583 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ff3787520 con 0x559ff34721b0 2024-09-21T06:22:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f1215755640 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff34721b0 0x559ff38a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.634+0000 7f11f97fa640 10 client.17583 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.635+0000 7f1215755640 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff34721b0 0x559ff38a19f0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f7c2b3e64074694 server_cookie=dd5335690e65e083 in_seq=0 out_seq=0 2024-09-21T06:22:06.694 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:22:06.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:22:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.755+0000 7f11f97fa640 1 -- 172.21.15.31:0/1756108514 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ff3787520 con 0x559ff34721b0 2024-09-21T06:22:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.755+0000 7f11f97fa640 10 client.17583 handle_command_reply: tid=0 2024-09-21T06:22:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.755+0000 7f121f201740 1 client.17583 shutdown 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff3430d80 msgr2=0x559ff3835c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff3430d80 0x559ff3835c10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f120c005820 tx=0x7f120c00cf70 comp rx=0 tx=0).stop 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 shutdown_connections 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559ff34721b0 0x559ff38a19f0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559ff34514d0 0x559ff3897960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559ff34492f0 0x559ff38361d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 --2- 172.21.15.31:0/1756108514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559ff3430d80 0x559ff3835c10 secure :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f120c005820 tx=0x7f120c00cf70 comp rx=0 tx=0).stop 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 >> 172.21.15.31:0/1756108514 conn(0x559ff3466300 msgr2=0x559ff3819150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f11f97fa640 0 client.17583 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 shutdown_connections 2024-09-21T06:22:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.756+0000 7f121f201740 1 -- 172.21.15.31:0/1756108514 wait complete. 2024-09-21T06:22:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.757+0000 7f121f201740 20 client.17583 trim_cache size 0 max 16384 2024-09-21T06:22:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:22:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:22:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:22:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:22:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.757+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11fc0b66f0 msgr2=0x7f11fc0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.757+0000 7f12181ba640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11fc0b66f0 0x7f11fc0b8c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f120802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.758+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 msgr2=0x7f12101a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.758+0000 7f12181ba640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 0x7f12101a0ad0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f120c00ea40 tx=0x7f120c00ef00 comp rx=0 tx=0).stop 2024-09-21T06:22:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.759+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 shutdown_connections 2024-09-21T06:22:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.759+0000 7f12181ba640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f11fc0b66f0 0x7f11fc0b8c00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.759+0000 7f12181ba640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12101083c0 0x7f12101a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.759+0000 7f12181ba640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1210107a70 0x7f12101a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.759+0000 7f12181ba640 1 --2- 172.21.15.31:0/2066541208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12101023e0 0x7f12101a0ad0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.760+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 >> 172.21.15.31:0/2066541208 conn(0x7f1210077770 msgr2=0x7f1210103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.760+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 shutdown_connections 2024-09-21T06:22:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:06.760+0000 7f12181ba640 1 -- 172.21.15.31:0/2066541208 wait complete. 2024-09-21T06:22:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:07.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:07 smithi107 ceph-mon[34380]: pgmap v644: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-09-21T06:22:07.320 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:07 smithi031 ceph-mon[31237]: pgmap v644: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-09-21T06:22:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:07 smithi110 ceph-mon[34550]: pgmap v644: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 17 MiB/s wr, 1.39k op/s 2024-09-21T06:22:07.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.759+0000 7f8dc0e17640 1 -- 172.21.15.31:0/2149579446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc10a9e0 msgr2=0x7f8dbc10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.759+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/2149579446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc10a9e0 0x7f8dbc10adf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009860 tx=0x7f8da402a340 comp rx=0 tx=0).stop 2024-09-21T06:22:07.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.761+0000 7f8dc0e17640 1 -- 172.21.15.31:0/2149579446 shutdown_connections 2024-09-21T06:22:07.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.761+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/2149579446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.761+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/2149579446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc10a9e0 0x7f8dbc10adf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.761+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/2149579446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8dbc111f40 0x7f8dbc10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.761+0000 7f8dc0e17640 1 -- 172.21.15.31:0/2149579446 >> 172.21.15.31:0/2149579446 conn(0x7f8dbc070ec0 msgr2=0x7f8dbc0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:07.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.764+0000 7f8dc0e17640 1 -- 172.21.15.31:0/2149579446 shutdown_connections 2024-09-21T06:22:07.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.765+0000 7f8dc0e17640 1 -- 172.21.15.31:0/2149579446 wait complete. 2024-09-21T06:22:07.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.766+0000 7f8dc0e17640 1 Processor -- start 2024-09-21T06:22:07.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.766+0000 7f8dc0e17640 1 -- start start 2024-09-21T06:22:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dc0e17640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8dbc10a9e0 0x7f8dbc11c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dc0e17640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc111f40 0x7f8dbc1167e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dc0e17640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc116e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dc0e17640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc112c80 con 0x7f8dbc10a9e0 2024-09-21T06:22:07.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dc0e17640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc112b00 con 0x7f8dbc11cc30 2024-09-21T06:22:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dc0e17640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc112e00 con 0x7f8dbc111f40 2024-09-21T06:22:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc111f40 0x7f8dbc1167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc116e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc116e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39156/0 (socket says 172.21.15.31:39156) 2024-09-21T06:22:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 -- 172.21.15.31:0/674375431 learned_addr learned my addr 172.21.15.31:0/674375431 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:07.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dba59c640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8dbc10a9e0 0x7f8dbc11c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 -- 172.21.15.31:0/674375431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc111f40 msgr2=0x7f8dbc1167e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc111f40 0x7f8dbc1167e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 -- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8dbc10a9e0 msgr2=0x7f8dbc11c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.767+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8dbc10a9e0 0x7f8dbc11c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.768+0000 7f8dbad9d640 1 -- 172.21.15.31:0/674375431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc1bdd90 con 0x7f8dbc11cc30 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.768+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc116e60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e970 tx=0x7f8db400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:07.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.768+0000 7f8db37fe640 1 -- 172.21.15.31:0/674375431 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db400ccc0 con 0x7f8dbc11cc30 2024-09-21T06:22:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.768+0000 7f8db37fe640 1 -- 172.21.15.31:0/674375431 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db40054d0 con 0x7f8dbc11cc30 2024-09-21T06:22:07.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.768+0000 7f8db37fe640 1 -- 172.21.15.31:0/674375431 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4010430 con 0x7f8dbc11cc30 2024-09-21T06:22:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.769+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc1bdf60 con 0x7f8dbc11cc30 2024-09-21T06:22:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.769+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc1be3c0 con 0x7f8dbc11cc30 2024-09-21T06:22:07.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.772+0000 7f8db37fe640 1 -- 172.21.15.31:0/674375431 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8db4005020 con 0x7f8dbc11cc30 2024-09-21T06:22:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.774+0000 7f8db37fe640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8da80b68c0 0x7f8da80b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.775+0000 7f8db37fe640 1 -- 172.21.15.31:0/674375431 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f8db40e7990 con 0x7f8dbc11cc30 2024-09-21T06:22:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.775+0000 7f8dba59c640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8da80b68c0 0x7f8da80b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.775+0000 7f8dc738b740 1 Processor -- start 2024-09-21T06:22:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 -- start start 2024-09-21T06:22:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edec9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 0x55d7edec9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb01670 0x55d7edec9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d7ede3fe60 con 0x55d7edb22350 2024-09-21T06:22:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7ede2f290 con 0x55d7edb01670 2024-09-21T06:22:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dc738b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d7ede330a0 con 0x55d7edaf9490 2024-09-21T06:22:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dba59c640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8da80b68c0 0x7f8da80b8dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc11c270 tx=0x7f8dac00d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 0x55d7edec9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 0x55d7edec9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38528/0 (socket says 172.21.15.31:38528) 2024-09-21T06:22:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8db3fff640 1 -- 172.21.15.31:0/2107470780 learned_addr learned my addr 172.21.15.31:0/2107470780 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.776+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb01670 0x55d7edec9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db3fff640 1 -- 172.21.15.31:0/2107470780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb01670 msgr2=0x55d7edec9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db3fff640 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb01670 0x55d7edec9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db3fff640 1 -- 172.21.15.31:0/2107470780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 msgr2=0x55d7edec9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db3fff640 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edec9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db3fff640 1 -- 172.21.15.31:0/2107470780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7edb8d930 con 0x55d7edaf9490 2024-09-21T06:22:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db3fff640 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 0x55d7edec9970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009830 tx=0x7f8da4030b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df625f47e934d870 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db0ff9640 1 -- 172.21.15.31:0/2107470780 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4041030 con 0x55d7edaf9490 2024-09-21T06:22:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db0ff9640 1 -- 172.21.15.31:0/2107470780 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da4030d40 con 0x55d7edaf9490 2024-09-21T06:22:07.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.777+0000 7f8db0ff9640 1 -- 172.21.15.31:0/2107470780 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4037040 con 0x55d7edaf9490 2024-09-21T06:22:07.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 -- 172.21.15.31:0/2107470780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 msgr2=0x55d7edec9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 0x55d7edec9970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009830 tx=0x7f8da4030b50 comp rx=0 tx=0).stop 2024-09-21T06:22:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 -- 172.21.15.31:0/2107470780 shutdown_connections 2024-09-21T06:22:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb01670 0x55d7edec9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edaf9490 0x55d7edec9970 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009830 tx=0x7f8da4030b50 comp rx=0 tx=0).stop 2024-09-21T06:22:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 --2- 172.21.15.31:0/2107470780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edec9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 -- 172.21.15.31:0/2107470780 >> 172.21.15.31:0/2107470780 conn(0x55d7edb16740 msgr2=0x55d7edec7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 -- 172.21.15.31:0/2107470780 shutdown_connections 2024-09-21T06:22:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.779+0000 7f8dc738b740 1 -- 172.21.15.31:0/2107470780 wait complete. 2024-09-21T06:22:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.780+0000 7f8dc738b740 1 -- start start 2024-09-21T06:22:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.780+0000 7f8dc738b740 1 Processor -- start 2024-09-21T06:22:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.780+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7eded5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb22350 0x55d7eded56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7ededae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dc738b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d7edc90890 con 0x55d7edb01670 2024-09-21T06:22:07.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dc738b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7eddc9d40 con 0x55d7edae0cd0 2024-09-21T06:22:07.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dc738b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d7eded7bb0 con 0x55d7edb22350 2024-09-21T06:22:07.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7eded5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7eded5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52682/0 (socket says 172.21.15.31:52682) 2024-09-21T06:22:07.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 -- 172.21.15.31:0/3064157336 learned_addr learned my addr 172.21.15.31:0/3064157336 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:07.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dc738b740 10 client.0 _ll_register_callbacks cb 0x55d7ede2dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:07.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7ededae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8db3fff640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb22350 0x55d7eded56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 -- 172.21.15.31:0/3064157336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb22350 msgr2=0x55d7eded56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb22350 0x55d7eded56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 -- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 msgr2=0x55d7ededae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7ededae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dba59c640 1 -- 172.21.15.31:0/3064157336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ededb4e0 con 0x55d7edb01670 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.781+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7ededae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8dba59c640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7eded5120 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8dac016e60 tx=0x7f8dac0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1e8c41989dd2be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:07.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8da1ffb640 1 -- 172.21.15.31:0/3064157336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac013e60 con 0x55d7edb01670 2024-09-21T06:22:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8da1ffb640 1 -- 172.21.15.31:0/3064157336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f8dac0205f0 con 0x55d7edb01670 2024-09-21T06:22:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da4030d40 con 0x55d7edb01670 2024-09-21T06:22:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8da1ffb640 1 -- 172.21.15.31:0/3064157336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8dac02e050 con 0x55d7edb01670 2024-09-21T06:22:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8dc738b740 10 client.17589 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.782+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d7edee3f10 con 0x55d7edb01670 2024-09-21T06:22:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8da1ffb640 1 -- 172.21.15.31:0/3064157336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da4030d40 con 0x55d7edb01670 2024-09-21T06:22:07.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8da1ffb640 1 -- 172.21.15.31:0/3064157336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8dac0299c0 con 0x55d7edb01670 2024-09-21T06:22:07.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8dc738b740 10 client.17589 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:07.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8dc738b740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:07.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8dc738b740 20 client.17589 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:07.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8dc738b740 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edab6080 0x55d7edee76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8dc738b740 4 client.17589 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7ede41db0 con 0x55d7edab6080 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.783+0000 7f8db3fff640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edab6080 0x55d7edee76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.787+0000 7f8db3fff640 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edab6080 0x55d7edee76e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db3fd8048c02302f server_cookie=8ca8088fa4f643cd in_seq=0 out_seq=0 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.787+0000 7f8da1ffb640 10 client.17589 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.787+0000 7f8da1ffb640 1 -- 172.21.15.31:0/3064157336 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7ede41db0 con 0x55d7edab6080 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.787+0000 7f8da1ffb640 10 client.17589 handle_command_reply: tid=0 2024-09-21T06:22:07.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 client.17589 shutdown 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 msgr2=0x55d7eded5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7eded5120 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8dac016e60 tx=0x7f8dac0096d0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 shutdown_connections 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edab6080 0x55d7edee76e0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7ededae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb22350 0x55d7eded56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 --2- 172.21.15.31:0/3064157336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7eded5120 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 >> 172.21.15.31:0/3064157336 conn(0x55d7edb16740 msgr2=0x55d7edeca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:07.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8da1ffb640 0 client.17589 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:07.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 shutdown_connections 2024-09-21T06:22:07.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 1 -- 172.21.15.31:0/3064157336 wait complete. 2024-09-21T06:22:07.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.788+0000 7f8dc738b740 20 client.17589 trim_cache size 0 max 16384 2024-09-21T06:22:07.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.812+0000 7f8dc738b740 1 Processor -- start 2024-09-21T06:22:07.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.812+0000 7f8dc738b740 1 -- start start 2024-09-21T06:22:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.812+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7edee1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.812+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb01670 0x55d7edecec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.812+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edeca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7edee1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dc738b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1c4be0 con 0x55d7edb22350 2024-09-21T06:22:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dc738b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1c7460 con 0x55d7edae0cd0 2024-09-21T06:22:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dc738b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1c5250 con 0x55d7edb01670 2024-09-21T06:22:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7edee1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39186/0 (socket says 172.21.15.31:39186) 2024-09-21T06:22:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 -- 172.21.15.31:0/1697531823 learned_addr learned my addr 172.21.15.31:0/1697531823 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edeca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8db3fff640 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb01670 0x55d7edecec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 -- 172.21.15.31:0/1697531823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb01670 msgr2=0x55d7edecec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb01670 0x55d7edecec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 -- 172.21.15.31:0/1697531823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 msgr2=0x55d7edeca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edeca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 -- 172.21.15.31:0/1697531823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7eded3e60 con 0x55d7edae0cd0 2024-09-21T06:22:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.813+0000 7f8dba59c640 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7edee1e20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8dac01eae0 tx=0x7f8dac028870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f477536e3a337657 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.814+0000 7f8da3fff640 1 -- 172.21.15.31:0/1697531823 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac030e30 con 0x55d7edae0cd0 2024-09-21T06:22:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.814+0000 7f8da3fff640 1 -- 172.21.15.31:0/1697531823 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dac039070 con 0x55d7edae0cd0 2024-09-21T06:22:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.814+0000 7f8da3fff640 1 -- 172.21.15.31:0/1697531823 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac009740 con 0x55d7edae0cd0 2024-09-21T06:22:07.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 -- 172.21.15.31:0/1697531823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 msgr2=0x55d7edee1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7edee1e20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8dac01eae0 tx=0x7f8dac028870 comp rx=0 tx=0).stop 2024-09-21T06:22:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 -- 172.21.15.31:0/1697531823 shutdown_connections 2024-09-21T06:22:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb22350 0x55d7edeca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edb01670 0x55d7edecec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1697531823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edae0cd0 0x55d7edee1e20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 -- 172.21.15.31:0/1697531823 >> 172.21.15.31:0/1697531823 conn(0x55d7edb16740 msgr2=0x55d7edece540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 -- 172.21.15.31:0/1697531823 shutdown_connections 2024-09-21T06:22:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 -- 172.21.15.31:0/1697531823 wait complete. 2024-09-21T06:22:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.815+0000 7f8dc738b740 1 -- start start 2024-09-21T06:22:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.816+0000 7f8dc738b740 1 Processor -- start 2024-09-21T06:22:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.816+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edae0cd0 0x55d7ededdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.816+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7edede380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dc738b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 0x55d7edf4af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dba59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edae0cd0 0x55d7ededdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dba59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edae0cd0 0x55d7ededdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38556/0 (socket says 172.21.15.31:38556) 2024-09-21T06:22:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dc738b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1c5920 con 0x55d7edb01670 2024-09-21T06:22:07.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dc738b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1c6d90 con 0x55d7edb22350 2024-09-21T06:22:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dc738b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1c5ff0 con 0x55d7edae0cd0 2024-09-21T06:22:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7edede380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 0x55d7edf4af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7edede380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52698/0 (socket says 172.21.15.31:52698) 2024-09-21T06:22:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dc738b740 10 client.0 _ll_register_callbacks cb 0x7f8dac009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:07.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 -- 172.21.15.31:0/1690974052 learned_addr learned my addr 172.21.15.31:0/1690974052 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 0x55d7edf4af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:39202/0 (socket says 172.21.15.31:39202) 2024-09-21T06:22:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 -- 172.21.15.31:0/1690974052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edae0cd0 msgr2=0x55d7ededdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edae0cd0 0x55d7ededdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 -- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 msgr2=0x55d7edf4af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 0x55d7edf4af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8db3fff640 1 -- 172.21.15.31:0/1690974052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7edf4b5a0 con 0x55d7edb01670 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.817+0000 7f8dbad9d640 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 0x55d7edf4af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8db3fff640 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7edede380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8da4030a00 tx=0x7f8da4030990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71533a48d8e3f097 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8da1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8da1ffb640 1 -- 172.21.15.31:0/1690974052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4041040 con 0x55d7edb01670 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8da1ffb640 1 -- 172.21.15.31:0/1690974052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8455+0+0 (secure 0 0 0) 0x7f8da4030650 con 0x55d7edb01670 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dac039070 con 0x55d7edb01670 2024-09-21T06:22:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8da1ffb640 1 -- 172.21.15.31:0/1690974052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8da4037040 con 0x55d7edb01670 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8dc738b740 10 client.17595 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d7edf52920 con 0x55d7edb01670 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.818+0000 7f8da1ffb640 1 -- 172.21.15.31:0/1690974052 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dac039070 con 0x55d7edb01670 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8da1ffb640 1 -- 172.21.15.31:0/1690974052 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8da4003970 con 0x55d7edb01670 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dc738b740 10 client.17595 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dc738b740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dc738b740 20 client.17595 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:07.834 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edf52b90 0x55d7edf54fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:07.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dc738b740 4 client.17595 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:22:07.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d7ede3d860 con 0x55d7edf52b90 2024-09-21T06:22:07.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dba59c640 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edf52b90 0x55d7edf54fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:07.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8da1ffb640 10 client.17595 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:07.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.819+0000 7f8dba59c640 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edf52b90 0x55d7edf54fb0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850d7db7dd3d9190 server_cookie=cb8ee52ff699312a in_seq=0 out_seq=0 2024-09-21T06:22:07.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.821+0000 7f8da1ffb640 1 -- 172.21.15.31:0/1690974052 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d7ede3d860 con 0x55d7edf52b90 2024-09-21T06:22:07.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.821+0000 7f8da1ffb640 10 client.17595 handle_command_reply: tid=0 2024-09-21T06:22:07.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 client.17595 shutdown 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 msgr2=0x55d7edede380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7edede380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8da4030a00 tx=0x7f8da4030990 comp rx=0 tx=0).stop 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 shutdown_connections 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d7edf52b90 0x55d7edf54fb0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7edb22350 0x55d7edf4af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d7edb01670 0x55d7edede380 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8da4030a00 tx=0x7f8da4030990 comp rx=0 tx=0).stop 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 --2- 172.21.15.31:0/1690974052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d7edae0cd0 0x55d7ededdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8da1ffb640 0 client.17595 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 >> 172.21.15.31:0/1690974052 conn(0x55d7edb16740 msgr2=0x55d7edece540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.822+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 shutdown_connections 2024-09-21T06:22:07.837 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:22:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:22:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "8d8a0e01-be41-453f-b15c-3b58c614b2da", 2024-09-21T06:22:07.838 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:22:07.838 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.823+0000 7f8dc738b740 1 -- 172.21.15.31:0/1690974052 wait complete. 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.823+0000 7f8dc738b740 20 client.17595 trim_cache size 0 max 16384 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8da80b68c0 msgr2=0x7f8da80b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8da80b68c0 0x7f8da80b8dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc11c270 tx=0x7f8dac00d040 comp rx=0 tx=0).stop 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 msgr2=0x7f8dbc116e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc116e60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8db400e970 tx=0x7f8db400ee30 comp rx=0 tx=0).stop 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 shutdown_connections 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dbc11cc30 0x7f8dbc116e60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8dbc111f40 0x7f8dbc1167e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8da80b68c0 0x7f8da80b8dd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 --2- 172.21.15.31:0/674375431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8dbc10a9e0 0x7f8dbc11c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:07.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 >> 172.21.15.31:0/674375431 conn(0x7f8dbc070ec0 msgr2=0x7f8dbc10ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:07.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 shutdown_connections 2024-09-21T06:22:07.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:07.824+0000 7f8dc0e17640 1 -- 172.21.15.31:0/674375431 wait complete. 2024-09-21T06:22:08.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:22:09.003 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:09.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:09 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:22:09.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:09 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:09.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:09 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:22:09.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:09 smithi031 ceph-mon[31237]: pgmap v645: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2024-09-21T06:22:09.326 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:09 smithi031 ceph-mon[31237]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:22:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:09 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:22:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:09 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:09 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:22:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:09 smithi110 ceph-mon[34550]: pgmap v645: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2024-09-21T06:22:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:09 smithi110 ceph-mon[34550]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:22:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:09 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:22:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:09 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:09 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:22:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:09 smithi107 ceph-mon[34380]: pgmap v645: 97 pgs: 97 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 1.38k op/s 2024-09-21T06:22:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:09 smithi107 ceph-mon[34380]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:22:09.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 -- 172.21.15.31:0/156640969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5228105320 msgr2=0x7f522810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 --2- 172.21.15.31:0/156640969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5228105320 0x7f522810d870 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5218009900 tx=0x7f521802a410 comp rx=0 tx=0).stop 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 -- 172.21.15.31:0/156640969 shutdown_connections 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 --2- 172.21.15.31:0/156640969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f522810de30 0x7f5228110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 --2- 172.21.15.31:0/156640969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5228105320 0x7f522810d870 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 --2- 172.21.15.31:0/156640969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f5228104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 -- 172.21.15.31:0/156640969 >> 172.21.15.31:0/156640969 conn(0x7f5228079740 msgr2=0x7f5228079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 -- 172.21.15.31:0/156640969 shutdown_connections 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.589+0000 7f522f4ea640 1 -- 172.21.15.31:0/156640969 wait complete. 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.590+0000 7f522f4ea640 1 Processor -- start 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.590+0000 7f522f4ea640 1 -- start start 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.590+0000 7f522f4ea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f522819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522f4ea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5228105320 0x7f522819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522f4ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f522810de30 0x7f522819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522f4ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5228113a90 con 0x7f522810de30 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522f4ea640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5228113910 con 0x7f5228105320 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522f4ea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5228113c10 con 0x7f52281049d0 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f522819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522da87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f522810de30 0x7f522819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f522819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:35136/0 (socket says 172.21.15.31:35136) 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 -- 172.21.15.31:0/2779131582 learned_addr learned my addr 172.21.15.31:0/2779131582 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:09.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522ca85640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5228105320 0x7f522819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 -- 172.21.15.31:0/2779131582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5228105320 msgr2=0x7f522819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5228105320 0x7f522819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 -- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f522810de30 msgr2=0x7f522819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f522810de30 0x7f522819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 -- 172.21.15.31:0/2779131582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52281a0470 con 0x7f52281049d0 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.591+0000 7f522d286640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f522819a500 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f522400e9c0 tx=0x7f522400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.592+0000 7f521e7fc640 1 -- 172.21.15.31:0/2779131582 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522400cd80 con 0x7f52281049d0 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.592+0000 7f521e7fc640 1 -- 172.21.15.31:0/2779131582 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52240035f0 con 0x7f52281049d0 2024-09-21T06:22:09.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.592+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52281a0760 con 0x7f52281049d0 2024-09-21T06:22:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.592+0000 7f521e7fc640 1 -- 172.21.15.31:0/2779131582 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224010690 con 0x7f52281049d0 2024-09-21T06:22:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.592+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52281a6ec0 con 0x7f52281049d0 2024-09-21T06:22:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.595+0000 7f5235a5e740 1 Processor -- start 2024-09-21T06:22:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.595+0000 7f5235a5e740 1 -- start start 2024-09-21T06:22:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.595+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc606650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f521e7fc640 1 -- 172.21.15.31:0/2779131582 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5224004000 con 0x7f52281049d0 2024-09-21T06:22:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc606b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f5235a5e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5654bc57ce50 con 0x5654bc2361b0 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f5235a5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654bc56c280 con 0x5654bc25f070 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f5235a5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654bc570090 con 0x5654bc23e390 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc606650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc606650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46422/0 (socket says 172.21.15.31:46422) 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 -- 172.21.15.31:0/879132538 learned_addr learned my addr 172.21.15.31:0/879132538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522d286640 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc606b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522ca85640 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 -- 172.21.15.31:0/879132538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 msgr2=0x5654bc6070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 -- 172.21.15.31:0/879132538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 msgr2=0x5654bc606b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc606b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 -- 172.21.15.31:0/879132538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bc2ca650 con 0x5654bc25f070 2024-09-21T06:22:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522d286640 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc606b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.596+0000 7f522da87640 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc606650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5220009fd0 tx=0x7f52200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc52eed220596f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.597+0000 7f521e7fc640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f52140b6680 0x7f52140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.597+0000 7f52137fe640 1 -- 172.21.15.31:0/879132538 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220004d20 con 0x5654bc25f070 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.597+0000 7f521e7fc640 1 -- 172.21.15.31:0/2779131582 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8543+0+0 (secure 0 0 0) 0x7f52240e7c60 con 0x7f52281049d0 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.597+0000 7f52137fe640 1 -- 172.21.15.31:0/879132538 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5220004050 con 0x5654bc25f070 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.597+0000 7f522d286640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f52140b6680 0x7f52140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.597+0000 7f52137fe640 1 -- 172.21.15.31:0/879132538 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220005280 con 0x5654bc25f070 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 -- 172.21.15.31:0/879132538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 msgr2=0x5654bc606650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f522d286640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f52140b6680 0x7f52140b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f52240e5450 tx=0x7f5224003e70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc606650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5220009fd0 tx=0x7f52200118d0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 -- 172.21.15.31:0/879132538 shutdown_connections 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc606b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 --2- 172.21.15.31:0/879132538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc606650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 -- 172.21.15.31:0/879132538 >> 172.21.15.31:0/879132538 conn(0x5654bc253460 msgr2=0x5654bc604540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 -- 172.21.15.31:0/879132538 shutdown_connections 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 -- 172.21.15.31:0/879132538 wait complete. 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 -- start start 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.598+0000 7f5235a5e740 1 Processor -- start 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc2361b0 0x5654bc611ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5654bc3cd700 con 0x5654bc25f070 2024-09-21T06:22:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5654bc507110 con 0x5654bc2361b0 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5654bc6192e0 con 0x5654bc23e390 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f5235a5e740 10 client.0 _ll_register_callbacks cb 0x5654bc520990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f522da87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc2361b0 0x5654bc611ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f522da87640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc2361b0 0x5654bc611ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46426/0 (socket says 172.21.15.31:46426) 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f522da87640 1 -- 172.21.15.31:0/2826021376 learned_addr learned my addr 172.21.15.31:0/2826021376 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f522ca85640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.599+0000 7f522da87640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f522ca85640 1 -- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc2361b0 msgr2=0x5654bc611ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f522ca85640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc2361b0 0x5654bc611ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f522ca85640 1 -- 172.21.15.31:0/2826021376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 msgr2=0x5654bc6225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f522ca85640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6225d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f522ca85640 1 -- 172.21.15.31:0/2826021376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bc534c40 con 0x5654bc23e390 2024-09-21T06:22:09.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f522ca85640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6121e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f521802a930 tx=0x7f5218032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1a05f030929728 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5210ff9640 1 -- 172.21.15.31:0/2826021376 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218035040 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5220004050 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5210ff9640 1 -- 172.21.15.31:0/2826021376 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8543+0+0 (secure 0 0 0) 0x7f5218031760 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5210ff9640 1 -- 172.21.15.31:0/2826021376 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5218004040 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5235a5e740 10 client.27451 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5654bc624fe0 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.600+0000 7f5210ff9640 1 -- 172.21.15.31:0/2826021376 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5220004050 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5210ff9640 1 -- 172.21.15.31:0/2826021376 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5218004750 con 0x5654bc23e390 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5235a5e740 10 client.27451 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5235a5e740 10 client.27451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5235a5e740 20 client.27451 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5235a5e740 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc21d9f0 0x5654bc628490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5235a5e740 4 client.27451 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654bc57eda0 con 0x5654bc21d9f0 2024-09-21T06:22:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f522da87640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc21d9f0 0x5654bc628490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f522da87640 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc21d9f0 0x5654bc628490 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2907281d05991cc4 server_cookie=20d22b0188c2d7e6 in_seq=0 out_seq=0 2024-09-21T06:22:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.601+0000 7f5210ff9640 10 client.27451 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5210ff9640 1 -- 172.21.15.31:0/2826021376 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5654bc57eda0 con 0x5654bc21d9f0 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5210ff9640 10 client.27451 handle_command_reply: tid=0 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 client.27451 shutdown 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 msgr2=0x5654bc6121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6121e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f521802a930 tx=0x7f5218032e90 comp rx=0 tx=0).stop 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 shutdown_connections 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc21d9f0 0x5654bc628490 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc6121e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 --2- 172.21.15.31:0/2826021376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc2361b0 0x5654bc611ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5210ff9640 0 client.27451 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.602+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 >> 172.21.15.31:0/2826021376 conn(0x5654bc253460 msgr2=0x5654bc607680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.603+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 shutdown_connections 2024-09-21T06:22:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.603+0000 7f5235a5e740 1 -- 172.21.15.31:0/2826021376 wait complete. 2024-09-21T06:22:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.603+0000 7f5235a5e740 20 client.27451 trim_cache size 0 max 16384 2024-09-21T06:22:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.622+0000 7f5235a5e740 1 Processor -- start 2024-09-21T06:22:09.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.622+0000 7f5235a5e740 1 -- start start 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 0x5654bc608040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc23e390 0x5654bc6138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f522da87640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 0x5654bc608040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f522da87640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 0x5654bc608040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:35152/0 (socket says 172.21.15.31:35152) 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f522da87640 1 -- 172.21.15.31:0/3739355537 learned_addr learned my addr 172.21.15.31:0/3739355537 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ab5c0 con 0x5654bc25f070 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ade20 con 0x5654bc23e390 2024-09-21T06:22:09.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52281abc10 con 0x5654bc2361b0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f522da87640 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.623+0000 7f522ca85640 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc23e390 0x5654bc6138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f522da87640 1 -- 172.21.15.31:0/3739355537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc23e390 msgr2=0x5654bc6138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f522da87640 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc23e390 0x5654bc6138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f522da87640 1 -- 172.21.15.31:0/3739355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 msgr2=0x5654bc6224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f522da87640 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6224d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f522da87640 1 -- 172.21.15.31:0/3739355537 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bc6143e0 con 0x5654bc2361b0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f522da87640 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 0x5654bc608040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5220014040 tx=0x7f5220005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1195c38a385a7794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f5212ffd640 1 -- 172.21.15.31:0/3739355537 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52200040b0 con 0x5654bc2361b0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f5212ffd640 1 -- 172.21.15.31:0/3739355537 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522000f4e0 con 0x5654bc2361b0 2024-09-21T06:22:09.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.624+0000 7f5212ffd640 1 -- 172.21.15.31:0/3739355537 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52200286b0 con 0x5654bc2361b0 2024-09-21T06:22:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 msgr2=0x5654bc608040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 0x5654bc608040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5220014040 tx=0x7f5220005fd0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 shutdown_connections 2024-09-21T06:22:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc25f070 0x5654bc6224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc23e390 0x5654bc6138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 --2- 172.21.15.31:0/3739355537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc2361b0 0x5654bc608040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.625+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 >> 172.21.15.31:0/3739355537 conn(0x5654bc253460 msgr2=0x5654bc6131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.626+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 shutdown_connections 2024-09-21T06:22:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.626+0000 7f5235a5e740 1 -- 172.21.15.31:0/3739355537 wait complete. 2024-09-21T06:22:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.626+0000 7f5235a5e740 1 -- start start 2024-09-21T06:22:09.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.626+0000 7f5235a5e740 1 Processor -- start 2024-09-21T06:22:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc61a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc61ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc691e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ac2e0 con 0x5654bc2361b0 2024-09-21T06:22:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ad750 con 0x5654bc25f070 2024-09-21T06:22:09.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f52281ac9b0 con 0x5654bc23e390 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f5235a5e740 10 client.0 _ll_register_callbacks cb 0x7f5218009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc61ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc61ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:35158/0 (socket says 172.21.15.31:35158) 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 -- 172.21.15.31:0/674836436 learned_addr learned my addr 172.21.15.31:0/674836436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522da87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc61a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 -- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 msgr2=0x5654bc691e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc691e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 -- 172.21.15.31:0/674836436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 msgr2=0x5654bc61a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc61a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.627+0000 7f522ca85640 1 -- 172.21.15.31:0/674836436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bc61b610 con 0x5654bc23e390 2024-09-21T06:22:09.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f522ca85640 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc61ad90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f521802a930 tx=0x7f5218004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3cd5998ae7617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5210ff9640 1 -- 172.21.15.31:0/674836436 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218035070 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5210ff9640 1 -- 172.21.15.31:0/674836436 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8543+0+0 (secure 0 0 0) 0x7f52180038c0 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522000f4e0 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5210ff9640 1 -- 172.21.15.31:0/674836436 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5218053040 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5235a5e740 10 client.27457 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.628+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5654bc693410 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5210ff9640 1 -- 172.21.15.31:0/674836436 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5218005850 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5210ff9640 1 -- 172.21.15.31:0/674836436 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f52180053d0 con 0x5654bc23e390 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5235a5e740 10 client.27457 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5235a5e740 10 client.27457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5235a5e740 20 client.27457 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5235a5e740 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc1f2da0 0x5654bc695ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5235a5e740 4 client.27457 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f522000f4e0 con 0x5654bc1f2da0 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.629+0000 7f522da87640 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc1f2da0 0x5654bc695ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5210ff9640 10 client.27457 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f522da87640 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc1f2da0 0x5654bc695ba0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c3beec89488fe4 server_cookie=8ea25c27501937c5 in_seq=0 out_seq=0 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5210ff9640 1 -- 172.21.15.31:0/674836436 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7670 (crc 0 0 0) 0x7f522000f4e0 con 0x5654bc1f2da0 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5210ff9640 10 client.27457 handle_command_reply: tid=0 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 client.27457 shutdown 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 msgr2=0x5654bc61ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc61ad90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f521802a930 tx=0x7f5218004410 comp rx=0 tx=0).stop 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 shutdown_connections 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5654bc1f2da0 0x5654bc695ba0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5654bc25f070 0x5654bc691e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5654bc23e390 0x5654bc61ad90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5210ff9640 0 client.27457 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 --2- 172.21.15.31:0/674836436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5654bc2361b0 0x5654bc61a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 >> 172.21.15.31:0/674836436 conn(0x5654bc253460 msgr2=0x5654bc607e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 shutdown_connections 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.630+0000 7f5235a5e740 1 -- 172.21.15.31:0/674836436 wait complete. 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f5235a5e740 20 client.27457 trim_cache size 0 max 16384 2024-09-21T06:22:09.636 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "8d8a0e01-be41-453f-b15c-3b58c614b2da", 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:22:09.637 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.638 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.639 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.640 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:22:09.641 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.643 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.644 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:22:09.646 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.648 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:22:09.649 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.650 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.651 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:09.652 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:22:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f52140b6680 msgr2=0x7f52140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f52140b6680 0x7f52140b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f52240e5450 tx=0x7f5224003e70 comp rx=0 tx=0).stop 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 msgr2=0x7f522819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f522819a500 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f522400e9c0 tx=0x7f522400ee80 comp rx=0 tx=0).stop 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 shutdown_connections 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f52140b6680 0x7f52140b8b90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f522810de30 0x7f522819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5228105320 0x7f522819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 --2- 172.21.15.31:0/2779131582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52281049d0 0x7f522819a500 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 >> 172.21.15.31:0/2779131582 conn(0x7f5228079740 msgr2=0x7f52281007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 shutdown_connections 2024-09-21T06:22:09.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:09.631+0000 7f522f4ea640 1 -- 172.21.15.31:0/2779131582 wait complete. 2024-09-21T06:22:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 630.0 s, 1460.1 tps, lat 46.930 ms stddev 112.682 2024-09-21T06:22:10.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:10 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:10.487 INFO:tasks.cephfs.filesystem:scrub status for tag:8d8a0e01-be41-453f-b15c-3b58c614b2da - {'path': '/', 'tag': '8d8a0e01-be41-453f-b15c-3b58c614b2da', 'options': 'recursive,force'} 2024-09-21T06:22:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:10 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:10 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:11.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:11 smithi031 ceph-mon[31237]: pgmap v647: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-09-21T06:22:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:11 smithi110 ceph-mon[34550]: pgmap v647: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-09-21T06:22:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:11 smithi107 ceph-mon[34380]: pgmap v647: 97 pgs: 8 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 85 active+clean; 5.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 18 MiB/s wr, 1.57k op/s 2024-09-21T06:22:13.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:13 smithi031 ceph-mon[31237]: pgmap v648: 97 pgs: 28 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 54 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-09-21T06:22:13.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:13 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:22:13.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:13 smithi110 ceph-mon[34550]: pgmap v648: 97 pgs: 28 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 54 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-09-21T06:22:13.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:13 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:22:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:13 smithi107 ceph-mon[34380]: pgmap v648: 97 pgs: 28 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 54 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-09-21T06:22:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:13 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:22:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 635.0 s, 1473.5 tps, lat 89.669 ms stddev 349.966 2024-09-21T06:22:15.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:15 smithi031 ceph-mon[31237]: pgmap v649: 97 pgs: 28 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 54 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-09-21T06:22:15.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:15 smithi110 ceph-mon[34550]: pgmap v649: 97 pgs: 28 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 54 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-09-21T06:22:15.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:15 smithi107 ceph-mon[34380]: pgmap v649: 97 pgs: 28 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 54 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 13 MiB/s wr, 1.29k op/s 2024-09-21T06:22:17.340 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:17 smithi107 ceph-mon[34380]: pgmap v650: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.6 MiB/s wr, 730 op/s 2024-09-21T06:22:17.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:17 smithi031 ceph-mon[31237]: pgmap v650: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.6 MiB/s wr, 730 op/s 2024-09-21T06:22:17.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:17 smithi110 ceph-mon[34550]: pgmap v650: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.6 MiB/s wr, 730 op/s 2024-09-21T06:22:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:19 smithi107 ceph-mon[34380]: pgmap v651: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 6.4 MiB/s wr, 621 op/s 2024-09-21T06:22:19.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:19 smithi107 ceph-mon[34380]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:22:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:19 smithi031 ceph-mon[31237]: pgmap v651: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 6.4 MiB/s wr, 621 op/s 2024-09-21T06:22:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:19 smithi031 ceph-mon[31237]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:22:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:19 smithi110 ceph-mon[34550]: pgmap v651: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 6.4 MiB/s wr, 621 op/s 2024-09-21T06:22:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:19 smithi110 ceph-mon[34550]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:22:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.901+0000 7f1ce1172640 1 -- 172.21.15.31:0/4247816773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc107bb0 msgr2=0x7f1cdc108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.901+0000 7f1ce1172640 1 --2- 172.21.15.31:0/4247816773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc108020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009be0 tx=0x7f1cd402a6c0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 -- 172.21.15.31:0/4247816773 shutdown_connections 2024-09-21T06:22:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 --2- 172.21.15.31:0/4247816773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc108020 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 --2- 172.21.15.31:0/4247816773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc103040 0x7f1cdc107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 --2- 172.21.15.31:0/4247816773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc1026f0 0x7f1cdc102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 -- 172.21.15.31:0/4247816773 >> 172.21.15.31:0/4247816773 conn(0x7f1cdc07a740 msgr2=0x7f1cdc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 -- 172.21.15.31:0/4247816773 shutdown_connections 2024-09-21T06:22:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.902+0000 7f1ce1172640 1 -- 172.21.15.31:0/4247816773 wait complete. 2024-09-21T06:22:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.903+0000 7f1ce1172640 1 Processor -- start 2024-09-21T06:22:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.903+0000 7f1ce1172640 1 -- start start 2024-09-21T06:22:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1ce1172640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc1026f0 0x7f1cdc1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1ce1172640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 0x7f1cdc1a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1ce1172640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc1a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1ce1172640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc119fe0 con 0x7f1cdc107bb0 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1ce1172640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc119e60 con 0x7f1cdc103040 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1ce1172640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc11a160 con 0x7f1cdc1026f0 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc1a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cda59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 0x7f1cdc1a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cda59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 0x7f1cdc1a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50194/0 (socket says 172.21.15.31:50194) 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cda59c640 1 -- 172.21.15.31:0/2884690621 learned_addr learned my addr 172.21.15.31:0/2884690621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc1026f0 0x7f1cdc1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 -- 172.21.15.31:0/2884690621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 msgr2=0x7f1cdc1a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 0x7f1cdc1a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 -- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc107bb0 msgr2=0x7f1cdc1a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc1a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 -- 172.21.15.31:0/2884690621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc1a68d0 con 0x7f1cdc1026f0 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc1a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cda59c640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 0x7f1cdc1a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.904+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc1026f0 0x7f1cdc1a09a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00e790 tx=0x7f1ccc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.905+0000 7f1cc3fff640 1 -- 172.21.15.31:0/2884690621 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc00eea0 con 0x7f1cdc1026f0 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.905+0000 7f1cc3fff640 1 -- 172.21.15.31:0/2884690621 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ccc00ccc0 con 0x7f1cdc1026f0 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.905+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc1a6bc0 con 0x7f1cdc1026f0 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.905+0000 7f1cc3fff640 1 -- 172.21.15.31:0/2884690621 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc018680 con 0x7f1cdc1026f0 2024-09-21T06:22:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.905+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc1ad350 con 0x7f1cdc1026f0 2024-09-21T06:22:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1cc3fff640 1 -- 172.21.15.31:0/2884690621 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1ccc018820 con 0x7f1cdc1026f0 2024-09-21T06:22:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 Processor -- start 2024-09-21T06:22:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 -- start start 2024-09-21T06:22:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 0x5591ff1719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff171ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff172430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5591ff0e66f0 con 0x5591feda32f0 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.910+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5591ff0d5b20 con 0x5591fedcc1b0 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591ff0d9930 con 0x5591fedab4d0 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 0x5591ff1719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff171ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 0x5591ff1719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50202/0 (socket says 172.21.15.31:50202) 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 -- 172.21.15.31:0/776035477 learned_addr learned my addr 172.21.15.31:0/776035477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cda59c640 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff172430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cc3fff640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1cc40b6680 0x7f1cc40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 -- 172.21.15.31:0/776035477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 msgr2=0x5591ff172430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff172430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 -- 172.21.15.31:0/776035477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 msgr2=0x5591ff171ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff171ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cdb59e640 1 -- 172.21.15.31:0/776035477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ff09b530 con 0x5591fedcc1b0 2024-09-21T06:22:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.911+0000 7f1cc3fff640 1 -- 172.21.15.31:0/2884690621 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f1ccc0e7920 con 0x7f1cdc1026f0 2024-09-21T06:22:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.912+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 0x5591ff1719b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4033000 tx=0x7f1cd40307d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9b82ecaef38de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.912+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1cc40b6680 0x7f1cc40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.912+0000 7f1cc17fa640 1 -- 172.21.15.31:0/776035477 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4041070 con 0x5591fedcc1b0 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.912+0000 7f1cc17fa640 1 -- 172.21.15.31:0/776035477 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd4003e30 con 0x5591fedcc1b0 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.912+0000 7f1cc17fa640 1 -- 172.21.15.31:0/776035477 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4036040 con 0x5591fedcc1b0 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 -- 172.21.15.31:0/776035477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 msgr2=0x5591ff1719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1cc40b6680 0x7f1cc40b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4043000 tx=0x7f1cd403d850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 0x5591ff1719b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4033000 tx=0x7f1cd40307d0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 -- 172.21.15.31:0/776035477 shutdown_connections 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff172430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff171ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/776035477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedcc1b0 0x5591ff1719b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 -- 172.21.15.31:0/776035477 >> 172.21.15.31:0/776035477 conn(0x5591fedc0300 msgr2=0x5591ff16f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 -- 172.21.15.31:0/776035477 shutdown_connections 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.913+0000 7f1ce81c2740 1 -- 172.21.15.31:0/776035477 wait complete. 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 -- start start 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 Processor -- start 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff17d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedcc1b0 0x5591ff189410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5591ff00dab0 con 0x5591feda32f0 2024-09-21T06:22:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5591ff07b730 con 0x5591fedab4d0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591ff180120 con 0x5591fedcc1b0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1ce81c2740 10 client.0 _ll_register_callbacks cb 0x5591ff0cf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1cda59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1cdad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff17d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1cda59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:50214/0 (socket says 172.21.15.31:50214) 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.914+0000 7f1cda59c640 1 -- 172.21.15.31:0/2265352134 learned_addr learned my addr 172.21.15.31:0/2265352134 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedcc1b0 0x5591ff189410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdad9d640 1 -- 172.21.15.31:0/2265352134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedcc1b0 msgr2=0x5591ff189410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedcc1b0 0x5591ff189410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdad9d640 1 -- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 msgr2=0x5591ff17d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdad9d640 1 -- 172.21.15.31:0/2265352134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591fee37790 con 0x5591feda32f0 2024-09-21T06:22:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cda59c640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff17d0a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0e5080 tx=0x7f1ccc0ace60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773aa6ef7a629246 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cbeff5640 1 -- 172.21.15.31:0/2265352134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0a9e80 con 0x5591feda32f0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cbeff5640 1 -- 172.21.15.31:0/2265352134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f1ccc018b10 con 0x5591feda32f0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd4003e30 con 0x5591feda32f0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cbeff5640 1 -- 172.21.15.31:0/2265352134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1ccc0acfa0 con 0x5591feda32f0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1ce81c2740 10 client.17625 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5591feeb0620 con 0x5591feda32f0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.915+0000 7f1cbeff5640 1 -- 172.21.15.31:0/2265352134 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd4003e30 con 0x5591feda32f0 2024-09-21T06:22:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1cbeff5640 1 -- 172.21.15.31:0/2265352134 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1ccc0b33f0 con 0x5591feda32f0 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1ce81c2740 10 client.17625 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1ce81c2740 10 client.17625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1ce81c2740 20 client.17625 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fed8ad80 0x5591ff18f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1ce81c2740 4 client.17625 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591ff0e8640 con 0x5591fed8ad80 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1cda59c640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fed8ad80 0x5591ff18f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.916+0000 7f1cbeff5640 10 client.17625 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.917+0000 7f1cda59c640 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fed8ad80 0x5591ff18f150 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ed66a220275380 server_cookie=ac10764aa94d0eb8 in_seq=0 out_seq=0 2024-09-21T06:22:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1cbeff5640 1 -- 172.21.15.31:0/2265352134 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591ff0e8640 con 0x5591fed8ad80 2024-09-21T06:22:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1cbeff5640 10 client.17625 handle_command_reply: tid=0 2024-09-21T06:22:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 client.17625 shutdown 2024-09-21T06:22:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 msgr2=0x5591ff17d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff17d0a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0e5080 tx=0x7f1ccc0ace60 comp rx=0 tx=0).stop 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 shutdown_connections 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fed8ad80 0x5591ff18f150 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedcc1b0 0x5591ff189410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1cbeff5640 0 client.17625 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/2265352134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff17d0a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 >> 172.21.15.31:0/2265352134 conn(0x5591fedc0300 msgr2=0x5591ff1729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 shutdown_connections 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.919+0000 7f1ce81c2740 1 -- 172.21.15.31:0/2265352134 wait complete. 2024-09-21T06:22:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.920+0000 7f1ce81c2740 20 client.17625 trim_cache size 0 max 16384 2024-09-21T06:22:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 Processor -- start 2024-09-21T06:22:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 -- start start 2024-09-21T06:22:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fed8ad80 0x5591ff1730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff189310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1b3cf0 con 0x5591feda32f0 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1b6550 con 0x5591fedab4d0 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.939+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1b4340 con 0x5591fed8ad80 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff189310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff189310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52188/0 (socket says 172.21.15.31:52188) 2024-09-21T06:22:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 -- 172.21.15.31:0/1160119771 learned_addr learned my addr 172.21.15.31:0/1160119771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fed8ad80 0x5591ff1730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 -- 172.21.15.31:0/1160119771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fed8ad80 msgr2=0x5591ff1730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fed8ad80 0x5591ff1730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 -- 172.21.15.31:0/1160119771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 msgr2=0x5591ff17a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 -- 172.21.15.31:0/1160119771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ff17d9c0 con 0x5591feda32f0 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fed8ad80 0x5591ff1730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cda59c640 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff189310 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009f30 tx=0x7f1cd0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d982f5a763932885 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cc0ff9640 1 -- 172.21.15.31:0/1160119771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000fcb0 con 0x5591feda32f0 2024-09-21T06:22:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cc0ff9640 1 -- 172.21.15.31:0/1160119771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd000fe50 con 0x5591feda32f0 2024-09-21T06:22:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.940+0000 7f1cc0ff9640 1 -- 172.21.15.31:0/1160119771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0003d80 con 0x5591feda32f0 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1160119771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 msgr2=0x5591ff189310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff189310 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009f30 tx=0x7f1cd0005f90 comp rx=0 tx=0).stop 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1160119771 shutdown_connections 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591fedab4d0 0x5591ff17a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591feda32f0 0x5591ff189310 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1160119771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fed8ad80 0x5591ff1730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1160119771 >> 172.21.15.31:0/1160119771 conn(0x5591fedc0300 msgr2=0x5591ff1729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1160119771 shutdown_connections 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.942+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1160119771 wait complete. 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.943+0000 7f1ce81c2740 1 -- start start 2024-09-21T06:22:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.943+0000 7f1ce81c2740 1 Processor -- start 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.943+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591fed8ad80 0x5591ff1858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.943+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591feda32f0 0x5591ff182db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.943+0000 7f1ce81c2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff1832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1b4a10 con 0x5591fed8ad80 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1b5e80 con 0x5591feda32f0 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1ce81c2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1b50e0 con 0x5591fedab4d0 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1ce81c2740 10 client.0 _ll_register_callbacks cb 0x7f1ccc0ed240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591fed8ad80 0x5591ff1858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591fed8ad80 0x5591ff1858f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52202/0 (socket says 172.21.15.31:52202) 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdad9d640 1 -- 172.21.15.31:0/1755892660 learned_addr learned my addr 172.21.15.31:0/1755892660 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdb59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff1832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdb59e640 1 -- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591feda32f0 msgr2=0x5591ff182db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591feda32f0 0x5591ff182db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdb59e640 1 -- 172.21.15.31:0/1755892660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591fed8ad80 msgr2=0x5591ff1858f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591fed8ad80 0x5591ff1858f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.944+0000 7f1cdb59e640 1 -- 172.21.15.31:0/1755892660 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ff183830 con 0x5591fedab4d0 2024-09-21T06:22:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.945+0000 7f1cdb59e640 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff1832f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1cd402acb0 tx=0x7f1cd4048050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ccb3f3e34247ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.945+0000 7f1cbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.954+0000 7f1cbeff5640 1 -- 172.21.15.31:0/1755892660 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4061060 con 0x5591fedab4d0 2024-09-21T06:22:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.954+0000 7f1cbeff5640 1 -- 172.21.15.31:0/1755892660 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f1cd4030b70 con 0x5591fedab4d0 2024-09-21T06:22:19.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.954+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd000fe50 con 0x5591fedab4d0 2024-09-21T06:22:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.963+0000 7f1cbeff5640 1 -- 172.21.15.31:0/1755892660 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1cd4036050 con 0x5591fedab4d0 2024-09-21T06:22:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.967+0000 7f1ce81c2740 10 client.27475 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.967+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5591ff1f97d0 con 0x5591fedab4d0 2024-09-21T06:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.973+0000 7f1cbeff5640 1 -- 172.21.15.31:0/1755892660 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd000fe50 con 0x5591fedab4d0 2024-09-21T06:22:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.973+0000 7f1cbeff5640 1 -- 172.21.15.31:0/1755892660 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1cd4050070 con 0x5591fedab4d0 2024-09-21T06:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.973+0000 7f1ce81c2740 10 client.27475 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.973+0000 7f1ce81c2740 10 client.27475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.973+0000 7f1ce81c2740 20 client.27475 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.973+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fedcc1b0 0x5591ff1fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.974+0000 7f1ce81c2740 4 client.27475 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.974+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591ff0e40f0 con 0x5591fedcc1b0 2024-09-21T06:22:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.975+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fedcc1b0 0x5591ff1fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.988+0000 7f1cdad9d640 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fedcc1b0 0x5591ff1fbb20 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7ae681dd7c7fb4 server_cookie=ad91c724450dbba2 in_seq=0 out_seq=0 2024-09-21T06:22:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:19.993+0000 7f1cbeff5640 10 client.27475 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 640.0 s, 1562.8 tps, lat 63.449 ms stddev 70.125 2024-09-21T06:22:21.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:21 smithi107 ceph-mon[34380]: pgmap v653: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-21T06:22:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:21 smithi031 ceph-mon[31237]: pgmap v653: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-21T06:22:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:21 smithi110 ceph-mon[34550]: pgmap v653: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-21T06:22:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1cbeff5640 1 -- 172.21.15.31:0/1755892660 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591ff0e40f0 con 0x5591fedcc1b0 2024-09-21T06:22:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1cbeff5640 10 client.27475 handle_command_reply: tid=0 2024-09-21T06:22:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 client.27475 shutdown 2024-09-21T06:22:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 msgr2=0x5591ff1832f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff1832f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1cd402acb0 tx=0x7f1cd4048050 comp rx=0 tx=0).stop 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 shutdown_connections 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5591fedcc1b0 0x5591ff1fbb20 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591fedab4d0 0x5591ff1832f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591feda32f0 0x5591ff182db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1cbeff5640 0 client.27475 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 --2- 172.21.15.31:0/1755892660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5591fed8ad80 0x5591ff1858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 >> 172.21.15.31:0/1755892660 conn(0x5591fedc0300 msgr2=0x5591ff16dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 shutdown_connections 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 1 -- 172.21.15.31:0/1755892660 wait complete. 2024-09-21T06:22:22.970 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.967+0000 7f1ce81c2740 20 client.27475 trim_cache size 0 max 16384 2024-09-21T06:22:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:22:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:22:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:22:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1cc40b6680 msgr2=0x7f1cc40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1cc40b6680 0x7f1cc40b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4043000 tx=0x7f1cd403d850 comp rx=0 tx=0).stop 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc1026f0 msgr2=0x7f1cdc1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc1026f0 0x7f1cdc1a09a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00e790 tx=0x7f1ccc00ec50 comp rx=0 tx=0).stop 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 shutdown_connections 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1cc40b6680 0x7f1cc40b8b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cdc107bb0 0x7f1cdc1a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cdc103040 0x7f1cdc1a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 --2- 172.21.15.31:0/2884690621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cdc1026f0 0x7f1cdc1a09a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 >> 172.21.15.31:0/2884690621 conn(0x7f1cdc07a740 msgr2=0x7f1cdc105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 shutdown_connections 2024-09-21T06:22:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:22.968+0000 7f1ce1172640 1 -- 172.21.15.31:0/2884690621 wait complete. 2024-09-21T06:22:23.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:23 smithi107 ceph-mon[34380]: pgmap v654: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 647 op/s 2024-09-21T06:22:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:23 smithi031 ceph-mon[31237]: pgmap v654: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 647 op/s 2024-09-21T06:22:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:23 smithi110 ceph-mon[34550]: pgmap v654: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 647 op/s 2024-09-21T06:22:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 645.0 s, 1283.2 tps, lat 78.417 ms stddev 136.391 2024-09-21T06:22:25.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:25 smithi107 ceph-mon[34380]: pgmap v655: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 647 op/s 2024-09-21T06:22:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:25 smithi031 ceph-mon[31237]: pgmap v655: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 647 op/s 2024-09-21T06:22:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:25 smithi110 ceph-mon[34550]: pgmap v655: 97 pgs: 97 active+clean; 4.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.5 MiB/s wr, 647 op/s 2024-09-21T06:22:27.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:27 smithi031 ceph-mon[31237]: pgmap v656: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-09-21T06:22:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:27 smithi110 ceph-mon[34550]: pgmap v656: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-09-21T06:22:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:27 smithi107 ceph-mon[34380]: pgmap v656: 97 pgs: 97 active+clean; 4.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 14 MiB/s wr, 1.30k op/s 2024-09-21T06:22:29.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:29 smithi110 ceph-mon[34550]: pgmap v657: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-09-21T06:22:29.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:29 smithi031 ceph-mon[31237]: pgmap v657: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-09-21T06:22:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:29 smithi107 ceph-mon[34380]: pgmap v657: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-09-21T06:22:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 650.0 s, 2103.4 tps, lat 47.869 ms stddev 58.139 2024-09-21T06:22:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:31 smithi031 ceph-mon[31237]: pgmap v658: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-09-21T06:22:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:31 smithi110 ceph-mon[34550]: pgmap v658: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-09-21T06:22:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:31 smithi107 ceph-mon[34380]: pgmap v658: 97 pgs: 97 active+clean; 4.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 16 MiB/s wr, 1.48k op/s 2024-09-21T06:22:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:33 smithi031 ceph-mon[31237]: pgmap v659: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.62k op/s 2024-09-21T06:22:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:33 smithi110 ceph-mon[34550]: pgmap v659: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.62k op/s 2024-09-21T06:22:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:33 smithi107 ceph-mon[34380]: pgmap v659: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 1.62k op/s 2024-09-21T06:22:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 655.0 s, 2159.4 tps, lat 46.020 ms stddev 54.037 2024-09-21T06:22:35.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:35 smithi031 ceph-mon[31237]: pgmap v660: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-21T06:22:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:35 smithi110 ceph-mon[34550]: pgmap v660: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-21T06:22:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:35 smithi107 ceph-mon[34380]: pgmap v660: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 1.33k op/s 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 -- 172.21.15.31:0/253650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1794108c30 msgr2=0x7f179410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 --2- 172.21.15.31:0/253650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1794108c30 0x7f179410f460 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f178c009ca0 tx=0x7f178c02a8d0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 -- 172.21.15.31:0/253650335 shutdown_connections 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 --2- 172.21.15.31:0/253650335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1794108c30 0x7f179410f460 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 --2- 172.21.15.31:0/253650335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17941082e0 0x7f17941086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 --2- 172.21.15.31:0/253650335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941029b0 0x7f1794102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 -- 172.21.15.31:0/253650335 >> 172.21.15.31:0/253650335 conn(0x7f179407a740 msgr2=0x7f179407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 -- 172.21.15.31:0/253650335 shutdown_connections 2024-09-21T06:22:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.116+0000 7f179ad5f640 1 -- 172.21.15.31:0/253650335 wait complete. 2024-09-21T06:22:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.117+0000 7f179ad5f640 1 Processor -- start 2024-09-21T06:22:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.117+0000 7f179ad5f640 1 -- start start 2024-09-21T06:22:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.117+0000 7f179ad5f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17941029b0 0x7f179419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.117+0000 7f179ad5f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 0x7f179419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.117+0000 7f179ad5f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1794108c30 0x7f17941a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.117+0000 7f179ad5f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f17941117f0 con 0x7f17941029b0 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f179ad5f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1794111670 con 0x7f17941082e0 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f179ad5f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1794111970 con 0x7f1794108c30 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 0x7f179419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1798afb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17941029b0 0x7f179419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 0x7f179419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:52026/0 (socket says 172.21.15.31:52026) 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 -- 172.21.15.31:0/2254896129 learned_addr learned my addr 172.21.15.31:0/2254896129 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f17992fc640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1794108c30 0x7f17941a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 -- 172.21.15.31:0/2254896129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1794108c30 msgr2=0x7f17941a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1794108c30 0x7f17941a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 -- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17941029b0 msgr2=0x7f179419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17941029b0 0x7f179419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 -- 172.21.15.31:0/2254896129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17941a2590 con 0x7f17941082e0 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1793fff640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 0x7f179419cc00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f178000e9f0 tx=0x7f178000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1791ffb640 1 -- 172.21.15.31:0/2254896129 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178000ccc0 con 0x7f17941082e0 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f1791ffb640 1 -- 172.21.15.31:0/2254896129 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17800108e0 con 0x7f17941082e0 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.118+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17941a2880 con 0x7f17941082e0 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.119+0000 7f1791ffb640 1 -- 172.21.15.31:0/2254896129 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780018860 con 0x7f17941082e0 2024-09-21T06:22:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.119+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941a9010 con 0x7f17941082e0 2024-09-21T06:22:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.122+0000 7f1791ffb640 1 -- 172.21.15.31:0/2254896129 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1780010430 con 0x7f17941082e0 2024-09-21T06:22:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.122+0000 7f17a1cc9740 1 Processor -- start 2024-09-21T06:22:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 -- start start 2024-09-21T06:22:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16df59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 0x556c16df5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16df6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556c16d6a6f0 con 0x556c16a501b0 2024-09-21T06:22:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556c16d59b20 con 0x556c16a2f4d0 2024-09-21T06:22:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556c16d5d930 con 0x556c16a272f0 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 0x556c16df5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 0x556c16df5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48022/0 (socket says 172.21.15.31:48022) 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 -- 172.21.15.31:0/2663299657 learned_addr learned my addr 172.21.15.31:0/2663299657 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1791ffb640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6670 0x7f17840b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17992fc640 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16df59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 -- 172.21.15.31:0/2663299657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 msgr2=0x556c16df6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16df6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f17992fc640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6670 0x7f17840b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 -- 172.21.15.31:0/2663299657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 msgr2=0x556c16df59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16df59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1798afb640 1 -- 172.21.15.31:0/2663299657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c16d1f530 con 0x556c16a272f0 2024-09-21T06:22:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.123+0000 7f1791ffb640 1 -- 172.21.15.31:0/2254896129 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f17800e7960 con 0x7f17941082e0 2024-09-21T06:22:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.124+0000 7f1798afb640 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 0x556c16df5ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d20 tx=0x7f1788011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6470a936af6477b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.124+0000 7f177effd640 1 -- 172.21.15.31:0/2663299657 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788004230 con 0x556c16a272f0 2024-09-21T06:22:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.124+0000 7f17992fc640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6670 0x7f17840b8b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f178c005e30 tx=0x7f178c02a8d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.124+0000 7f177effd640 1 -- 172.21.15.31:0/2663299657 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17880043d0 con 0x556c16a272f0 2024-09-21T06:22:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.124+0000 7f177effd640 1 -- 172.21.15.31:0/2663299657 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880197c0 con 0x556c16a272f0 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 -- 172.21.15.31:0/2663299657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 msgr2=0x556c16df5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 0x556c16df5ef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1788009d20 tx=0x7f1788011810 comp rx=0 tx=0).stop 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 -- 172.21.15.31:0/2663299657 shutdown_connections 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16df6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a272f0 0x556c16df5ef0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/2663299657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16df59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 -- 172.21.15.31:0/2663299657 >> 172.21.15.31:0/2663299657 conn(0x556c16a44300 msgr2=0x556c16df38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.125+0000 7f17a1cc9740 1 -- 172.21.15.31:0/2663299657 shutdown_connections 2024-09-21T06:22:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.127+0000 7f17a1cc9740 1 -- 172.21.15.31:0/2663299657 wait complete. 2024-09-21T06:22:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.128+0000 7f17a1cc9740 1 -- start start 2024-09-21T06:22:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.128+0000 7f17a1cc9740 1 Processor -- start 2024-09-21T06:22:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16e0d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556c16c91ab0 con 0x556c16a501b0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556c16cff730 con 0x556c16a272f0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556c16e04130 con 0x556c16a2f4d0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17a1cc9740 10 client.0 _ll_register_callbacks cb 0x556c16d536b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48034/0 (socket says 172.21.15.31:48034) 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 -- 172.21.15.31:0/4022995501 learned_addr learned my addr 172.21.15.31:0/4022995501 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17992fc640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16e0d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1798afb640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 -- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 msgr2=0x556c16e010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 -- 172.21.15.31:0/4022995501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 msgr2=0x556c16e0d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16e0d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 -- 172.21.15.31:0/4022995501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c16abb790 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f17992fc640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16e0d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1793fff640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e015e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f17800a9e40 tx=0x7f178000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3635f721494a855 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.129+0000 7f1773fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f1773fff640 1 -- 172.21.15.31:0/4022995501 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800acfe0 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f1773fff640 1 -- 172.21.15.31:0/4022995501 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f17800ac540 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17880043d0 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f1773fff640 1 -- 172.21.15.31:0/4022995501 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1780010720 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 10 client.27487 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556c16b34620 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f1773fff640 1 -- 172.21.15.31:0/4022995501 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17880043d0 con 0x556c16a2f4d0 2024-09-21T06:22:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f1773fff640 1 -- 172.21.15.31:0/4022995501 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f17800b6720 con 0x556c16a2f4d0 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 10 client.27487 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 10 client.27487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 20 client.27487 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a0ed80 0x556c16e13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 4 client.27487 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.130+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c16d6c640 con 0x556c16a0ed80 2024-09-21T06:22:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.131+0000 7f1798afb640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a0ed80 0x556c16e13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.131+0000 7f1773fff640 10 client.27487 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.131+0000 7f1798afb640 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a0ed80 0x556c16e13210 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948870c2412683a0 server_cookie=eb6a987318d6addb in_seq=0 out_seq=0 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.132+0000 7f1773fff640 1 -- 172.21.15.31:0/4022995501 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c16d6c640 con 0x556c16a0ed80 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.132+0000 7f1773fff640 10 client.27487 handle_command_reply: tid=0 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.132+0000 7f17a1cc9740 1 client.27487 shutdown 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.133+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 msgr2=0x556c16e015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.133+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e015e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f17800a9e40 tx=0x7f178000ce60 comp rx=0 tx=0).stop 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.133+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 shutdown_connections 2024-09-21T06:22:36.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.133+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a0ed80 0x556c16e13210 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.133+0000 7f1773fff640 0 client.27487 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.133+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a501b0 0x556c16e0d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.134+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e015e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.134+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/4022995501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.134+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 >> 172.21.15.31:0/4022995501 conn(0x556c16a44300 msgr2=0x556c16df69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.134+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 shutdown_connections 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.134+0000 7f17a1cc9740 1 -- 172.21.15.31:0/4022995501 wait complete. 2024-09-21T06:22:36.137 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.134+0000 7f17a1cc9740 20 client.27487 trim_cache size 0 max 16384 2024-09-21T06:22:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 Processor -- start 2024-09-21T06:22:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 -- start start 2024-09-21T06:22:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a0ed80 0x556c16e01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 0x556c16df25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16e0d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f1798afb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a0ed80 0x556c16e01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f1798afb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a0ed80 0x556c16e01580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:48044/0 (socket says 172.21.15.31:48044) 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f1793fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 0x556c16df25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f1793fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 0x556c16df25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49956/0 (socket says 172.21.15.31:49956) 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f1798afb640 1 -- 172.21.15.31:0/3554107453 learned_addr learned my addr 172.21.15.31:0/3554107453 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f17941af990 con 0x556c16a272f0 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17941b21f0 con 0x556c16a2f4d0 2024-09-21T06:22:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.156+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f17941affe0 con 0x556c16a0ed80 2024-09-21T06:22:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.157+0000 7f17992fc640 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16e0d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.157+0000 7f1793fff640 1 -- 172.21.15.31:0/3554107453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a0ed80 msgr2=0x556c16e01580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.157+0000 7f1793fff640 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a0ed80 0x556c16e01580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.157+0000 7f1793fff640 1 -- 172.21.15.31:0/3554107453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 msgr2=0x556c16e0d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.157+0000 7f1793fff640 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16e0d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.157+0000 7f1793fff640 1 -- 172.21.15.31:0/3554107453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c16dfb5b0 con 0x556c16a272f0 2024-09-21T06:22:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.158+0000 7f1793fff640 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 0x556c16df25b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f17800b6cb0 tx=0x7f1780000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03711b93e002021 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.158+0000 7f177e7fc640 1 -- 172.21.15.31:0/3554107453 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800a9ec0 con 0x556c16a272f0 2024-09-21T06:22:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.158+0000 7f177e7fc640 1 -- 172.21.15.31:0/3554107453 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17800ad380 con 0x556c16a272f0 2024-09-21T06:22:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.158+0000 7f177e7fc640 1 -- 172.21.15.31:0/3554107453 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800aeaf0 con 0x556c16a272f0 2024-09-21T06:22:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 msgr2=0x556c16df25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 0x556c16df25b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f17800b6cb0 tx=0x7f1780000bc0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 shutdown_connections 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a2f4d0 0x556c16e0d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a272f0 0x556c16df25b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/3554107453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a0ed80 0x556c16e01580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 >> 172.21.15.31:0/3554107453 conn(0x556c16a44300 msgr2=0x556c16df1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 shutdown_connections 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.159+0000 7f17a1cc9740 1 -- 172.21.15.31:0/3554107453 wait complete. 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.160+0000 7f17a1cc9740 1 -- start start 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.160+0000 7f17a1cc9740 1 Processor -- start 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 0x556c16e06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f17941b06b0 con 0x556c16a0ed80 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f17941b1b20 con 0x556c16a272f0 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f17941b0d80 con 0x556c16a2f4d0 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17a1cc9740 10 client.0 _ll_register_callbacks cb 0x7f1780011d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1798afb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 0x556c16e06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1798afb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 0x556c16e06120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49964/0 (socket says 172.21.15.31:49964) 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1798afb640 1 -- 172.21.15.31:0/1874791773 learned_addr learned my addr 172.21.15.31:0/1874791773 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17992fc640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 -- 172.21.15.31:0/1874791773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 msgr2=0x556c16e7bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e7bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 -- 172.21.15.31:0/1874791773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 msgr2=0x556c16e06120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 0x556c16e06120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 -- 172.21.15.31:0/1874791773 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c16e7c1c0 con 0x556c16a272f0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1798afb640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 0x556c16e06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f17992fc640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e7bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1793fff640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e066c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17800b73a0 tx=0x7f178000ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bb639f4c256847e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.161+0000 7f1773fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f1773fff640 1 -- 172.21.15.31:0/1874791773 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800c7060 con 0x556c16a272f0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f1773fff640 1 -- 172.21.15.31:0/1874791773 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f17800aecf0 con 0x556c16a272f0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17800ad380 con 0x556c16a272f0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f1773fff640 1 -- 172.21.15.31:0/1874791773 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f17800a9ec0 con 0x556c16a272f0 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f17a1cc9740 10 client.27434 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556c16e7d450 con 0x556c16a272f0 2024-09-21T06:22:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f1773fff640 1 -- 172.21.15.31:0/1874791773 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17800ad380 con 0x556c16a272f0 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f1773fff640 1 -- 172.21.15.31:0/1874791773 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f17800cf1f0 con 0x556c16a272f0 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f17a1cc9740 10 client.27434 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f17a1cc9740 10 client.27434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.162+0000 7f17a1cc9740 20 client.27434 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.163+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a501b0 0x556c16e7fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.163+0000 7f17a1cc9740 4 client.27434 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.163+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c16d680f0 con 0x556c16a501b0 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.165+0000 7f1798afb640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a501b0 0x556c16e7fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.166+0000 7f1773fff640 10 client.27434 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.166+0000 7f1798afb640 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a501b0 0x556c16e7fa40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83f2f93d79c8d72b server_cookie=8bd176c81a5708c6 in_seq=0 out_seq=0 2024-09-21T06:22:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.238+0000 7f1773fff640 1 -- 172.21.15.31:0/1874791773 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c16d680f0 con 0x556c16a501b0 2024-09-21T06:22:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.238+0000 7f1773fff640 10 client.27434 handle_command_reply: tid=0 2024-09-21T06:22:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.238+0000 7f17a1cc9740 1 client.27434 shutdown 2024-09-21T06:22:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.238+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 msgr2=0x556c16e066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.238+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e066c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17800b73a0 tx=0x7f178000ce60 comp rx=0 tx=0).stop 2024-09-21T06:22:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.238+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 shutdown_connections 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556c16a501b0 0x556c16e7fa40 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c16a2f4d0 0x556c16e7bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556c16a272f0 0x556c16e066c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f1773fff640 0 client.27434 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 --2- 172.21.15.31:0/1874791773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556c16a0ed80 0x556c16e06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 >> 172.21.15.31:0/1874791773 conn(0x556c16a44300 msgr2=0x556c16df3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 shutdown_connections 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 1 -- 172.21.15.31:0/1874791773 wait complete. 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.239+0000 7f17a1cc9740 20 client.27434 trim_cache size 0 max 16384 2024-09-21T06:22:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:22:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:22:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:22:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:22:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6670 msgr2=0x7f17840b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6670 0x7f17840b8b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f178c005e30 tx=0x7f178c02a8d0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 msgr2=0x7f179419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 0x7f179419cc00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f178000e9f0 tx=0x7f178000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 shutdown_connections 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1794108c30 0x7f17941a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17941082e0 0x7f179419cc00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f17840b6670 0x7f17840b8b80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 --2- 172.21.15.31:0/2254896129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17941029b0 0x7f179419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 >> 172.21.15.31:0/2254896129 conn(0x7f179407a740 msgr2=0x7f1794104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 shutdown_connections 2024-09-21T06:22:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:36.240+0000 7f179ad5f640 1 -- 172.21.15.31:0/2254896129 wait complete. 2024-09-21T06:22:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:37 smithi031 ceph-mon[31237]: pgmap v661: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2024-09-21T06:22:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:37 smithi110 ceph-mon[34550]: pgmap v661: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2024-09-21T06:22:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:37 smithi107 ceph-mon[34380]: pgmap v661: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 17 MiB/s wr, 1.53k op/s 2024-09-21T06:22:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:39 smithi031 ceph-mon[31237]: pgmap v662: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 933 op/s 2024-09-21T06:22:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:39 smithi110 ceph-mon[34550]: pgmap v662: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 933 op/s 2024-09-21T06:22:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:39 smithi107 ceph-mon[34380]: pgmap v662: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 933 op/s 2024-09-21T06:22:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 660.0 s, 2362.4 tps, lat 42.555 ms stddev 55.424 2024-09-21T06:22:40.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:22:40.876 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:41.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.484+0000 7f5144f75640 1 -- 172.21.15.31:0/2171115891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5140109fc0 msgr2=0x7f514010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.484+0000 7f5144f75640 1 --2- 172.21.15.31:0/2171115891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5140109fc0 0x7f514010a3d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5130009900 tx=0x7f513002a410 comp rx=0 tx=0).stop 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 -- 172.21.15.31:0/2171115891 shutdown_connections 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 --2- 172.21.15.31:0/2171115891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 0x7f5140116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 --2- 172.21.15.31:0/2171115891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514010f900 0x7f514010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 --2- 172.21.15.31:0/2171115891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5140109fc0 0x7f514010a3d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 -- 172.21.15.31:0/2171115891 >> 172.21.15.31:0/2171115891 conn(0x7f51400795c0 msgr2=0x7f51401081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 -- 172.21.15.31:0/2171115891 shutdown_connections 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 -- 172.21.15.31:0/2171115891 wait complete. 2024-09-21T06:22:41.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.485+0000 7f5144f75640 1 Processor -- start 2024-09-21T06:22:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 -- start start 2024-09-21T06:22:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5140109fc0 0x7f51401a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 0x7f51401a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 0x7f51401a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5140119f10 con 0x7f514010f900 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5140119d90 con 0x7f5140110250 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f5144f75640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f514011a090 con 0x7f5140109fc0 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f513effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 0x7f51401a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f513ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 0x7f51401a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f513effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 0x7f51401a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48794/0 (socket says 172.21.15.31:48794) 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.486+0000 7f513effd640 1 -- 172.21.15.31:0/2212236943 learned_addr learned my addr 172.21.15.31:0/2212236943 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513f7fe640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5140109fc0 0x7f51401a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513ffff640 1 -- 172.21.15.31:0/2212236943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5140109fc0 msgr2=0x7f51401a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513ffff640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5140109fc0 0x7f51401a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513ffff640 1 -- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 msgr2=0x7f51401a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513ffff640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 0x7f51401a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513ffff640 1 -- 172.21.15.31:0/2212236943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51401a6830 con 0x7f5140110250 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513f7fe640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5140109fc0 0x7f51401a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513effd640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 0x7f51401a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513ffff640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 0x7f51401a6100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5138009fd0 tx=0x7f513800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513cff9640 1 -- 172.21.15.31:0/2212236943 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513800ce20 con 0x7f5140110250 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51401a6b20 con 0x7f5140110250 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513cff9640 1 -- 172.21.15.31:0/2212236943 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5138005020 con 0x7f5140110250 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f513cff9640 1 -- 172.21.15.31:0/2212236943 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138010690 con 0x7f5140110250 2024-09-21T06:22:41.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.487+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51401ad280 con 0x7f5140110250 2024-09-21T06:22:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.491+0000 7f514c74b740 1 Processor -- start 2024-09-21T06:22:41.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.491+0000 7f513cff9640 1 -- 172.21.15.31:0/2212236943 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f51380054d0 con 0x7f5140110250 2024-09-21T06:22:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.493+0000 7f513cff9640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51280b6680 0x7f51280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.493+0000 7f514c74b740 1 -- start start 2024-09-21T06:22:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.493+0000 7f513cff9640 1 -- 172.21.15.31:0/2212236943 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f51380e7d90 con 0x7f5140110250 2024-09-21T06:22:41.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.493+0000 7f513effd640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51280b6680 0x7f51280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f514c74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x56299707a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f514c74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x56299707a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f514c74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x56299707ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f514c74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562996ff0e50 con 0x562996caa1b0 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f514c74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562996fe0280 con 0x562996cb2390 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f514c74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562996fe4090 con 0x562996cd3070 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x56299707ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x56299707ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35346/0 (socket says 172.21.15.31:35346) 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 -- 172.21.15.31:0/2424303889 learned_addr learned my addr 172.21.15.31:0/2424303889 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:41.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513f7fe640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x56299707a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 -- 172.21.15.31:0/2424303889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 msgr2=0x56299707a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x56299707a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 -- 172.21.15.31:0/2424303889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 msgr2=0x56299707a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513effd640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x56299707a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513f7fe640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x56299707a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x56299707a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513ffff640 1 -- 172.21.15.31:0/2424303889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562996d3e650 con 0x562996cb2390 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.494+0000 7f513effd640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x56299707a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:22:41.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.495+0000 7f513ffff640 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x56299707ae90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f51380aa1c0 tx=0x7f51380ad7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66abda051e3f4601 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.495+0000 7f513effd640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51280b6680 0x7f51280b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51401100c0 tx=0x7f5134011920 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.495+0000 7f5125ffb640 1 -- 172.21.15.31:0/2424303889 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380b2020 con 0x562996cb2390 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.495+0000 7f5125ffb640 1 -- 172.21.15.31:0/2424303889 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51380b7610 con 0x562996cb2390 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.495+0000 7f5125ffb640 1 -- 172.21.15.31:0/2424303889 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380b73e0 con 0x562996cb2390 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 -- 172.21.15.31:0/2424303889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 msgr2=0x56299707ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x56299707ae90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f51380aa1c0 tx=0x7f51380ad7c0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 -- 172.21.15.31:0/2424303889 shutdown_connections 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x56299707ae90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x56299707a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 --2- 172.21.15.31:0/2424303889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x56299707a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 -- 172.21.15.31:0/2424303889 >> 172.21.15.31:0/2424303889 conn(0x562996cc7460 msgr2=0x562997078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:41.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 -- 172.21.15.31:0/2424303889 shutdown_connections 2024-09-21T06:22:41.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.496+0000 7f514c74b740 1 -- 172.21.15.31:0/2424303889 wait complete. 2024-09-21T06:22:41.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 -- start start 2024-09-21T06:22:41.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 Processor -- start 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x562997085af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x562997086030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997091f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562996e41700 con 0x562996caa1b0 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562996f7b110 con 0x562996cb2390 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.497+0000 7f514c74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562997088c40 con 0x562996cd3070 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f514c74b740 10 client.0 _ll_register_callbacks cb 0x562996f94990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x562997085af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x562997085af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48812/0 (socket says 172.21.15.31:48812) 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x562997086030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 -- 172.21.15.31:0/3892179342 learned_addr learned my addr 172.21.15.31:0/3892179342 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513ffff640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997091f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 -- 172.21.15.31:0/3892179342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 msgr2=0x562997091f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997091f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 -- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 msgr2=0x562997086030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x562997086030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513f7fe640 1 -- 172.21.15.31:0/3892179342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562996fa8c40 con 0x562996caa1b0 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513ffff640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.498+0000 7f513effd640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x562997086030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.499+0000 7f513f7fe640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x562997085af0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5130009900 tx=0x7f51300316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824a295f9cb84d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:41.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.499+0000 7f511effd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.499+0000 7f511effd640 1 -- 172.21.15.31:0/3892179342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130035040 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.499+0000 7f511effd640 1 -- 172.21.15.31:0/3892179342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f5130004040 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.499+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51380b7610 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f511effd640 1 -- 172.21.15.31:0/3892179342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5130035040 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 10 client.17658 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5629970948f0 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f511effd640 1 -- 172.21.15.31:0/3892179342 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51380b7610 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f511effd640 1 -- 172.21.15.31:0/3892179342 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5130004800 con 0x562996caa1b0 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 10 client.17658 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 20 client.17658 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c919f0 0x562997097dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 4 client.17658 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562996ff2da0 con 0x562996c919f0 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.500+0000 7f513effd640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c919f0 0x562997097dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.501+0000 7f511effd640 10 client.17658 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.501+0000 7f513effd640 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c919f0 0x562997097dc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25eb743d728bbc79 server_cookie=e57ca7d81fefc9f8 in_seq=0 out_seq=0 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f511effd640 1 -- 172.21.15.31:0/3892179342 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562996ff2da0 con 0x562996c919f0 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f511effd640 10 client.17658 handle_command_reply: tid=0 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 client.17658 shutdown 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 msgr2=0x562997085af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x562997085af0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5130009900 tx=0x7f51300316d0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 shutdown_connections 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c919f0 0x562997097dc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997091f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cb2390 0x562997086030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 --2- 172.21.15.31:0/3892179342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996caa1b0 0x562997085af0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f511effd640 0 client.17658 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 >> 172.21.15.31:0/3892179342 conn(0x562996cc7460 msgr2=0x56299707d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.502+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 shutdown_connections 2024-09-21T06:22:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.503+0000 7f514c74b740 1 -- 172.21.15.31:0/3892179342 wait complete. 2024-09-21T06:22:41.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.503+0000 7f514c74b740 20 client.17658 trim_cache size 0 max 16384 2024-09-21T06:22:41.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.541+0000 7f514c74b740 1 Processor -- start 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 -- start start 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996c919f0 0x562997085fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996cb2390 0x562997077930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997077e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51401b3d50 con 0x562996cb2390 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51401b65f0 con 0x562996c919f0 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.542+0000 7f514c74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51401b43e0 con 0x562996cd3070 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.543+0000 7f513ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997077e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.544+0000 7f513f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996c919f0 0x562997085fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.544+0000 7f513ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997077e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55028/0 (socket says 172.21.15.31:55028) 2024-09-21T06:22:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.544+0000 7f513ffff640 1 -- 172.21.15.31:0/1423947597 learned_addr learned my addr 172.21.15.31:0/1423947597 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.546+0000 7f513effd640 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996cb2390 0x562997077930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.548+0000 7f513ffff640 1 -- 172.21.15.31:0/1423947597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996c919f0 msgr2=0x562997085fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.548+0000 7f513ffff640 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996c919f0 0x562997085fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.548+0000 7f513ffff640 1 -- 172.21.15.31:0/1423947597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996cb2390 msgr2=0x562997077930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.548+0000 7f513ffff640 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996cb2390 0x562997077930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.548+0000 7f513ffff640 1 -- 172.21.15.31:0/1423947597 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562997086db0 con 0x562996cd3070 2024-09-21T06:22:41.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.550+0000 7f513ffff640 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997077e70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f51380e55a0 tx=0x7f5138005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9302d1d5f0338c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:41.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.550+0000 7f51257fa640 1 -- 172.21.15.31:0/1423947597 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380b2040 con 0x562996cd3070 2024-09-21T06:22:41.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.550+0000 7f51257fa640 1 -- 172.21.15.31:0/1423947597 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51380b7610 con 0x562996cd3070 2024-09-21T06:22:41.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.550+0000 7f51257fa640 1 -- 172.21.15.31:0/1423947597 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380a9540 con 0x562996cd3070 2024-09-21T06:22:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.552+0000 7f514c74b740 1 -- 172.21.15.31:0/1423947597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 msgr2=0x562997077e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.552+0000 7f514c74b740 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997077e70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f51380e55a0 tx=0x7f5138005870 comp rx=0 tx=0).stop 2024-09-21T06:22:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.553+0000 7f514c74b740 1 -- 172.21.15.31:0/1423947597 shutdown_connections 2024-09-21T06:22:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.553+0000 7f514c74b740 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cd3070 0x562997077e70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.553+0000 7f514c74b740 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996cb2390 0x562997077930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.553+0000 7f514c74b740 1 --2- 172.21.15.31:0/1423947597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996c919f0 0x562997085fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.553+0000 7f514c74b740 1 -- 172.21.15.31:0/1423947597 >> 172.21.15.31:0/1423947597 conn(0x562996cc7460 msgr2=0x5629970858a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.553+0000 7f514c74b740 1 -- 172.21.15.31:0/1423947597 shutdown_connections 2024-09-21T06:22:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.554+0000 7f514c74b740 1 -- 172.21.15.31:0/1423947597 wait complete. 2024-09-21T06:22:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.554+0000 7f514c74b740 1 -- start start 2024-09-21T06:22:41.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.555+0000 7f514c74b740 1 Processor -- start 2024-09-21T06:22:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996c919f0 0x5629970fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cb2390 0x5629970fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f513f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996c919f0 0x5629970fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cd3070 0x56299708de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51401b4ab0 con 0x562996c919f0 2024-09-21T06:22:41.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f513effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cb2390 0x5629970fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51401b5f20 con 0x562996cd3070 2024-09-21T06:22:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51401b5180 con 0x562996cb2390 2024-09-21T06:22:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f513effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cb2390 0x5629970fc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55038/0 (socket says 172.21.15.31:55038) 2024-09-21T06:22:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f513effd640 1 -- 172.21.15.31:0/2714365887 learned_addr learned my addr 172.21.15.31:0/2714365887 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f514c74b740 10 client.0 _ll_register_callbacks cb 0x7f5130009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:41.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.556+0000 7f513ffff640 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cd3070 0x56299708de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f513ffff640 1 -- 172.21.15.31:0/2714365887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cb2390 msgr2=0x5629970fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f513ffff640 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cb2390 0x5629970fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f513ffff640 1 -- 172.21.15.31:0/2714365887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996c919f0 msgr2=0x5629970fbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f513ffff640 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996c919f0 0x5629970fbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f513ffff640 1 -- 172.21.15.31:0/2714365887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299708e3d0 con 0x562996cd3070 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f513ffff640 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cd3070 0x56299708de90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f51380b7580 tx=0x7f51380aa2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d630a6354f20d829 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.557+0000 7f511effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:22:41.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.558+0000 7f511effd640 1 -- 172.21.15.31:0/2714365887 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380b2040 con 0x562996cd3070 2024-09-21T06:22:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.558+0000 7f511effd640 1 -- 172.21.15.31:0/2714365887 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f51380a9540 con 0x562996cd3070 2024-09-21T06:22:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.558+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51380b7610 con 0x562996cd3070 2024-09-21T06:22:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.558+0000 7f511effd640 1 -- 172.21.15.31:0/2714365887 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f51380d51c0 con 0x562996cd3070 2024-09-21T06:22:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.558+0000 7f514c74b740 10 client.27452 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:41.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.558+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562997113f40 con 0x562996cd3070 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f511effd640 1 -- 172.21.15.31:0/2714365887 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51380a9740 con 0x562996cd3070 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f511effd640 1 -- 172.21.15.31:0/2714365887 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f51380a9050 con 0x562996cd3070 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f514c74b740 10 client.27452 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f514c74b740 10 client.27452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f514c74b740 20 client.27452 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f514c74b740 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c66da0 0x562997116120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f514c74b740 4 client.27452 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f51380b7610 con 0x562996c66da0 2024-09-21T06:22:41.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.559+0000 7f513f7fe640 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c66da0 0x562997116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.560+0000 7f511effd640 10 client.27452 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:41.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.560+0000 7f513f7fe640 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c66da0 0x562997116120 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f656c4519d1283 server_cookie=443068c2fb9b5de9 in_seq=0 out_seq=0 2024-09-21T06:22:41.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.560+0000 7f511effd640 1 -- 172.21.15.31:0/2714365887 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7474 (crc 0 0 0) 0x7f51380b7610 con 0x562996c66da0 2024-09-21T06:22:41.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.560+0000 7f511effd640 10 client.27452 handle_command_reply: tid=0 2024-09-21T06:22:41.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 client.27452 shutdown 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cd3070 msgr2=0x56299708de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cd3070 0x56299708de90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f51380b7580 tx=0x7f51380aa2f0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 shutdown_connections 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562996c66da0 0x562997116120 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f511effd640 0 client.27452 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562996cd3070 0x56299708de90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562996cb2390 0x5629970fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 --2- 172.21.15.31:0/2714365887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562996c919f0 0x5629970fbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 >> 172.21.15.31:0/2714365887 conn(0x562996cc7460 msgr2=0x56299707d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 shutdown_connections 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.561+0000 7f514c74b740 1 -- 172.21.15.31:0/2714365887 wait complete. 2024-09-21T06:22:41.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.562+0000 7f514c74b740 20 client.27452 trim_cache size 0 max 16384 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:22:41.566 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.567 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.568 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:22:41.569 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.572 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.573 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.574 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:22:41.575 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.576 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.577 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:22:41.578 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.580 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:22:41.581 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:41.582 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.562+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51280b6680 msgr2=0x7f51280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.562+0000 7f5144f75640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51280b6680 0x7f51280b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51401100c0 tx=0x7f5134011920 comp rx=0 tx=0).stop 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.562+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 msgr2=0x7f51401a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 0x7f51401a6100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5138009fd0 tx=0x7f513800ef10 comp rx=0 tx=0).stop 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 shutdown_connections 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5140110250 0x7f51401a6100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51280b6680 0x7f51280b8b90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514010f900 0x7f51401a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 --2- 172.21.15.31:0/2212236943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5140109fc0 0x7f51401a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:41.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 >> 172.21.15.31:0/2212236943 conn(0x7f51400795c0 msgr2=0x7f5140079cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:41.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 shutdown_connections 2024-09-21T06:22:41.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:41.563+0000 7f5144f75640 1 -- 172.21.15.31:0/2212236943 wait complete. 2024-09-21T06:22:41.584 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:41 smithi031 ceph-mon[31237]: pgmap v663: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 626 op/s 2024-09-21T06:22:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:41 smithi110 ceph-mon[34550]: pgmap v663: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 626 op/s 2024-09-21T06:22:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:41 smithi107 ceph-mon[34380]: pgmap v663: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 626 op/s 2024-09-21T06:22:42.841 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:22:42.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:22:43.290 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:43.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:43 smithi031 ceph-mon[31237]: pgmap v664: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.9 MiB/s wr, 765 op/s 2024-09-21T06:22:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:43 smithi110 ceph-mon[34550]: pgmap v664: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.9 MiB/s wr, 765 op/s 2024-09-21T06:22:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:43 smithi107 ceph-mon[34380]: pgmap v664: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.9 MiB/s wr, 765 op/s 2024-09-21T06:22:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 -- 172.21.15.31:0/1737521335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810f150 msgr2=0x7fde1810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 --2- 172.21.15.31:0/1737521335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810f150 0x7fde1810f560 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde14009860 tx=0x7fde1402a360 comp rx=0 tx=0).stop 2024-09-21T06:22:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 -- 172.21.15.31:0/1737521335 shutdown_connections 2024-09-21T06:22:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 --2- 172.21.15.31:0/1737521335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde1810ad10 0x7fde18116fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 --2- 172.21.15.31:0/1737521335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810a3c0 0x7fde1810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 --2- 172.21.15.31:0/1737521335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810f150 0x7fde1810f560 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde14009860 tx=0x7fde1402a360 comp rx=0 tx=0).stop 2024-09-21T06:22:43.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.881+0000 7fde1feb6640 1 -- 172.21.15.31:0/1737521335 >> 172.21.15.31:0/1737521335 conn(0x7fde180712f0 msgr2=0x7fde18073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.884+0000 7fde1feb6640 1 -- 172.21.15.31:0/1737521335 shutdown_connections 2024-09-21T06:22:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.884+0000 7fde1feb6640 1 -- 172.21.15.31:0/1737521335 wait complete. 2024-09-21T06:22:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 Processor -- start 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 -- start start 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810a3c0 0x7fde181a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 0x7fde181a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde181ab4d0 0x7fde181ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde1811a110 con 0x7fde181ab4d0 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde18119f90 con 0x7fde1810a3c0 2024-09-21T06:22:43.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.885+0000 7fde1feb6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde1811a290 con 0x7fde1810ad10 2024-09-21T06:22:43.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.886+0000 7fde1d451640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 0x7fde181a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.886+0000 7fde1dc52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810a3c0 0x7fde181a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.886+0000 7fde1d451640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 0x7fde181a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55066/0 (socket says 172.21.15.31:55066) 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.886+0000 7fde1d451640 1 -- 172.21.15.31:0/3438434110 learned_addr learned my addr 172.21.15.31:0/3438434110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.887+0000 7fde1d451640 1 -- 172.21.15.31:0/3438434110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810a3c0 msgr2=0x7fde181a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.887+0000 7fde1d451640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810a3c0 0x7fde181a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.887+0000 7fde1d451640 1 -- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde181ab4d0 msgr2=0x7fde181ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.887+0000 7fde1d451640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde181ab4d0 0x7fde181ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.887+0000 7fde1d451640 1 -- 172.21.15.31:0/3438434110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde181a6340 con 0x7fde1810ad10 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.887+0000 7fde1d451640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 0x7fde181a1140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00e9f0 tx=0x7fde0c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.888+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00ccc0 con 0x7fde1810ad10 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.888+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c0035f0 con 0x7fde1810ad10 2024-09-21T06:22:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.888+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c00ce60 con 0x7fde1810ad10 2024-09-21T06:22:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.893+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde181a6630 con 0x7fde1810ad10 2024-09-21T06:22:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.893+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde181a6af0 con 0x7fde1810ad10 2024-09-21T06:22:43.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.895+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1810f560 con 0x7fde1810ad10 2024-09-21T06:22:43.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.904+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fde0c004000 con 0x7fde1810ad10 2024-09-21T06:22:43.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.906+0000 7fde0affd640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde040b6880 0x7fde040b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:43.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.906+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7fde0c0e8210 con 0x7fde1810ad10 2024-09-21T06:22:43.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.907+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde0c01d060 con 0x7fde1810ad10 2024-09-21T06:22:43.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.909+0000 7fde1dc52640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde040b6880 0x7fde040b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:43.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:43.914+0000 7fde1dc52640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde040b6880 0x7fde040b8d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fde1402a8d0 tx=0x7fde14000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:44.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.267+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde181a1c30 con 0x7fde1810ad10 2024-09-21T06:22:44.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.271+0000 7fde0affd640 1 -- 172.21.15.31:0/3438434110 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7fde0c0ae790 con 0x7fde1810ad10 2024-09-21T06:22:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:22:44.273 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:22:44.273 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:22:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde040b6880 msgr2=0x7fde040b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde040b6880 0x7fde040b8d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fde1402a8d0 tx=0x7fde14000bc0 comp rx=0 tx=0).stop 2024-09-21T06:22:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 msgr2=0x7fde181a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 0x7fde181a1140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00e9f0 tx=0x7fde0c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:22:44.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 shutdown_connections 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde040b6880 0x7fde040b8d90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde181ab4d0 0x7fde181ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde1810ad10 0x7fde181a1140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 --2- 172.21.15.31:0/3438434110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1810a3c0 0x7fde181a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.277+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 >> 172.21.15.31:0/3438434110 conn(0x7fde180712f0 msgr2=0x7fde18073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.278+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 shutdown_connections 2024-09-21T06:22:44.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:44.278+0000 7fde1feb6640 1 -- 172.21.15.31:0/3438434110 wait complete. 2024-09-21T06:22:44.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:44 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3438434110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:44.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:44 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3438434110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:44.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:44 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3438434110' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:22:45.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:22:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 665.0 s, 1613.6 tps, lat 60.881 ms stddev 72.356 2024-09-21T06:22:45.509 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:45.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:45 smithi031 ceph-mon[31237]: pgmap v665: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2024-09-21T06:22:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:45 smithi110 ceph-mon[34550]: pgmap v665: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2024-09-21T06:22:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:45 smithi107 ceph-mon[34380]: pgmap v665: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2024-09-21T06:22:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 -- 172.21.15.31:0/4197551883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 msgr2=0x7f671010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 --2- 172.21.15.31:0/4197551883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f671010f550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6708009ca0 tx=0x7f670802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 -- 172.21.15.31:0/4197551883 shutdown_connections 2024-09-21T06:22:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 --2- 172.21.15.31:0/4197551883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f671010f550 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 --2- 172.21.15.31:0/4197551883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 0x7f67101087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 --2- 172.21.15.31:0/4197551883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f6710102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 -- 172.21.15.31:0/4197551883 >> 172.21.15.31:0/4197551883 conn(0x7f6710079790 msgr2=0x7f6710079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 -- 172.21.15.31:0/4197551883 shutdown_connections 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.096+0000 7f6718137640 1 -- 172.21.15.31:0/4197551883 wait complete. 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.097+0000 7f6718137640 1 Processor -- start 2024-09-21T06:22:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.097+0000 7f6718137640 1 -- start start 2024-09-21T06:22:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6718137640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f671019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6718137640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 0x7f671019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6718137640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f67101a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6718137640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67101129e0 con 0x7f6710108d20 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6718137640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6710112860 con 0x7f6710102ae0 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6718137640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6710112b60 con 0x7f67101083d0 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f67166d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f67101a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f671019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f67156d2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 0x7f671019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f671019c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35398/0 (socket says 172.21.15.31:35398) 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f67166d4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f67101a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48844/0 (socket says 172.21.15.31:48844) 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 -- 172.21.15.31:0/3454550438 learned_addr learned my addr 172.21.15.31:0/3454550438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 -- 172.21.15.31:0/3454550438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 msgr2=0x7f671019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 0x7f671019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 -- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 msgr2=0x7f67101a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f67101a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 -- 172.21.15.31:0/3454550438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67101a2560 con 0x7f6710102ae0 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f67156d2640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 0x7f671019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.098+0000 7f6715ed3640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f671019c5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f670c00e9f0 tx=0x7f670c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.099+0000 7f6706ffd640 1 -- 172.21.15.31:0/3454550438 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c00cd80 con 0x7f6710102ae0 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.099+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67101a2850 con 0x7f6710102ae0 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.099+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67101a8fb0 con 0x7f6710102ae0 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.099+0000 7f6706ffd640 1 -- 172.21.15.31:0/3454550438 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670c0108e0 con 0x7f6710102ae0 2024-09-21T06:22:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.099+0000 7f6706ffd640 1 -- 172.21.15.31:0/3454550438 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c018980 con 0x7f6710102ae0 2024-09-21T06:22:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.103+0000 7f6706ffd640 1 -- 172.21.15.31:0/3454550438 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f670c010430 con 0x7f6710102ae0 2024-09-21T06:22:46.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.104+0000 7f671e6ab740 1 Processor -- start 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 -- start start 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6706ffd640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66fc0b6700 0x7f66fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 0x5571f968d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f67166d4640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66fc0b6700 0x7f66fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92bd1b0 0x5571f968dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f968e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5571f9603ea0 con 0x5571f92c5390 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571f95f32d0 con 0x5571f92e6070 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f671e6ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5571f95f70e0 con 0x5571f92bd1b0 2024-09-21T06:22:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6706ffd640 1 -- 172.21.15.31:0/3454550438 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f670c0e7a10 con 0x7f6710102ae0 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6715ed3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 0x5571f968d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f67156d2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92bd1b0 0x5571f968dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6715ed3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 0x5571f968d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:35406/0 (socket says 172.21.15.31:35406) 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f67156d2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92bd1b0 0x5571f968dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55106/0 (socket says 172.21.15.31:55106) 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6715ed3640 1 -- 172.21.15.31:0/558248320 learned_addr learned my addr 172.21.15.31:0/558248320 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f67166d4640 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f968e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6715ed3640 1 -- 172.21.15.31:0/558248320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92bd1b0 msgr2=0x5571f968dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6715ed3640 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92bd1b0 0x5571f968dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.105+0000 7f6715ed3640 1 -- 172.21.15.31:0/558248320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 msgr2=0x5571f968e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f6715ed3640 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f968e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f6715ed3640 1 -- 172.21.15.31:0/558248320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571f9351650 con 0x5571f92e6070 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f67166d4640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66fc0b6700 0x7f66fc0b8c10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6708006010 tx=0x7f6708031bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f6715ed3640 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 0x5571f968d6a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f670c0a9e90 tx=0x7f670c0213f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b825150db4810a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f66fbfff640 1 -- 172.21.15.31:0/558248320 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c0b22f0 con 0x5571f92e6070 2024-09-21T06:22:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f66fbfff640 1 -- 172.21.15.31:0/558248320 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670c0b7230 con 0x5571f92e6070 2024-09-21T06:22:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.106+0000 7f66fbfff640 1 -- 172.21.15.31:0/558248320 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c0a9210 con 0x5571f92e6070 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 -- 172.21.15.31:0/558248320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 msgr2=0x5571f968d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 0x5571f968d6a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f670c0a9e90 tx=0x7f670c0213f0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 -- 172.21.15.31:0/558248320 shutdown_connections 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f968e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92bd1b0 0x5571f968dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 --2- 172.21.15.31:0/558248320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92e6070 0x5571f968d6a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 -- 172.21.15.31:0/558248320 >> 172.21.15.31:0/558248320 conn(0x5571f92da460 msgr2=0x5571f968b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 -- 172.21.15.31:0/558248320 shutdown_connections 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.107+0000 7f671e6ab740 1 -- 172.21.15.31:0/558248320 wait complete. 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.108+0000 7f671e6ab740 1 -- start start 2024-09-21T06:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.108+0000 7f671e6ab740 1 Processor -- start 2024-09-21T06:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.110+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92bd1b0 0x5571f9698d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.110+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f96992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92e6070 0x5571f96a5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67156d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f96992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f671e6ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5571f9454700 con 0x5571f92c5390 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f671e6ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5571f958e160 con 0x5571f92bd1b0 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f671e6ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5571f969be80 con 0x5571f92e6070 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67156d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f96992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48854/0 (socket says 172.21.15.31:48854) 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67156d2640 1 -- 172.21.15.31:0/3658490462 learned_addr learned my addr 172.21.15.31:0/3658490462 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f671e6ab740 10 client.0 _ll_register_callbacks cb 0x5571f95a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f6715ed3640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92bd1b0 0x5571f9698d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67166d4640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92e6070 0x5571f96a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67166d4640 1 -- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92bd1b0 msgr2=0x5571f9698d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67166d4640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92bd1b0 0x5571f9698d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67166d4640 1 -- 172.21.15.31:0/3658490462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 msgr2=0x5571f96992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67166d4640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f96992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.111+0000 7f67166d4640 1 -- 172.21.15.31:0/3658490462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571f95bbc90 con 0x5571f92e6070 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f67166d4640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92e6070 0x5571f96a5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6708005e30 tx=0x7f670802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d72be6c9ac4e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f66f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f66f97fa640 1 -- 172.21.15.31:0/3658490462 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670804f070 con 0x5571f92e6070 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f670c0b7230 con 0x5571f92e6070 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f66f97fa640 1 -- 172.21.15.31:0/3658490462 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f6708004450 con 0x5571f92e6070 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f66f97fa640 1 -- 172.21.15.31:0/3658490462 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6708004650 con 0x5571f92e6070 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f671e6ab740 10 client.27529 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571f96a7ba0 con 0x5571f92e6070 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.112+0000 7f66f97fa640 1 -- 172.21.15.31:0/3658490462 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670c0b7230 con 0x5571f92e6070 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f66f97fa640 1 -- 172.21.15.31:0/3658490462 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f67080305c0 con 0x5571f92e6070 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f671e6ab740 10 client.27529 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f671e6ab740 10 client.27529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f671e6ab740 20 client.27529 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f671e6ab740 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92a49f0 0x5571f96ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f671e6ab740 4 client.27529 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571f9605df0 con 0x5571f92a49f0 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.113+0000 7f6715ed3640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92a49f0 0x5571f96ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f66f97fa640 10 client.27529 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f6715ed3640 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92a49f0 0x5571f96ab060 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a4f4b980eb599a server_cookie=4bc652d8e3ba520d in_seq=0 out_seq=0 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f66f97fa640 1 -- 172.21.15.31:0/3658490462 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571f9605df0 con 0x5571f92a49f0 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f66f97fa640 10 client.27529 handle_command_reply: tid=0 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f671e6ab740 1 client.27529 shutdown 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92e6070 msgr2=0x5571f96a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.114+0000 7f671e6ab740 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92e6070 0x5571f96a5170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6708005e30 tx=0x7f670802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 shutdown_connections 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92a49f0 0x5571f96ab060 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f66f97fa640 0 client.27529 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92e6070 0x5571f96a5170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92c5390 0x5571f96992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 --2- 172.21.15.31:0/3658490462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92bd1b0 0x5571f9698d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 >> 172.21.15.31:0/3658490462 conn(0x5571f92da460 msgr2=0x5571f968e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 shutdown_connections 2024-09-21T06:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 1 -- 172.21.15.31:0/3658490462 wait complete. 2024-09-21T06:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.115+0000 7f671e6ab740 20 client.27529 trim_cache size 0 max 16384 2024-09-21T06:22:46.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 Processor -- start 2024-09-21T06:22:46.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 -- start start 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92a49f0 0x5571f9699240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f96a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92c5390 0x5571f9695f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67101ab4d0 con 0x5571f92bd1b0 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67101add10 con 0x5571f92a49f0 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.132+0000 7f671e6ab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67101abb20 con 0x5571f92c5390 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67156d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f96a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67156d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f96a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48866/0 (socket says 172.21.15.31:48866) 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67156d2640 1 -- 172.21.15.31:0/2190755442 learned_addr learned my addr 172.21.15.31:0/2190755442 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:46.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92c5390 0x5571f9695f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f6715ed3640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92a49f0 0x5571f9699240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 -- 172.21.15.31:0/2190755442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92a49f0 msgr2=0x5571f9699240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92a49f0 0x5571f9699240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 -- 172.21.15.31:0/2190755442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 msgr2=0x5571f96a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f96a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 -- 172.21.15.31:0/2190755442 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571f968be80 con 0x5571f92c5390 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67156d2640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f96a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f6715ed3640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92a49f0 0x5571f9699240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.133+0000 7f67166d4640 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92c5390 0x5571f9695f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6708030990 tx=0x7f670802f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f276411b9983af4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.134+0000 7f66fb7fe640 1 -- 172.21.15.31:0/2190755442 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708062030 con 0x5571f92c5390 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.134+0000 7f66fb7fe640 1 -- 172.21.15.31:0/2190755442 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6708036050 con 0x5571f92c5390 2024-09-21T06:22:46.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.134+0000 7f66fb7fe640 1 -- 172.21.15.31:0/2190755442 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670804f040 con 0x5571f92c5390 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 -- 172.21.15.31:0/2190755442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92c5390 msgr2=0x5571f9695f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92c5390 0x5571f9695f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6708030990 tx=0x7f670802f430 comp rx=0 tx=0).stop 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 -- 172.21.15.31:0/2190755442 shutdown_connections 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92c5390 0x5571f9695f70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f96a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 --2- 172.21.15.31:0/2190755442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92a49f0 0x5571f9699240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 -- 172.21.15.31:0/2190755442 >> 172.21.15.31:0/2190755442 conn(0x5571f92da460 msgr2=0x5571f9698b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 -- 172.21.15.31:0/2190755442 shutdown_connections 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 -- 172.21.15.31:0/2190755442 wait complete. 2024-09-21T06:22:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 -- start start 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.135+0000 7f671e6ab740 1 Processor -- start 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 0x5571f969d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f969d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92c5390 0x5571f9714a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67101ac1d0 con 0x5571f92bd1b0 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f969d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f67101ad640 con 0x5571f92c5390 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f6715ed3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 0x5571f969d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f969d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48872/0 (socket says 172.21.15.31:48872) 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f6715ed3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 0x5571f969d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55130/0 (socket says 172.21.15.31:55130) 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 -- 172.21.15.31:0/1319588425 learned_addr learned my addr 172.21.15.31:0/1319588425 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f67101ac8a0 con 0x5571f92a49f0 2024-09-21T06:22:46.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f671e6ab740 10 client.0 _ll_register_callbacks cb 0x7f6708009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67166d4640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92c5390 0x5571f9714a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 -- 172.21.15.31:0/1319588425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 msgr2=0x5571f969d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 0x5571f969d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 -- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92c5390 msgr2=0x5571f9714a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92c5390 0x5571f9714a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 -- 172.21.15.31:0/1319588425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571f969e1a0 con 0x5571f92bd1b0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f6715ed3640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 0x5571f969d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f67156d2640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f969d900 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f670000fdc0 tx=0x7f6700012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e20c7f89be9d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.136+0000 7f66f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.137+0000 7f66f97fa640 1 -- 172.21.15.31:0/1319588425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700022020 con 0x5571f92bd1b0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.137+0000 7f66f97fa640 1 -- 172.21.15.31:0/1319588425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f6700004230 con 0x5571f92bd1b0 2024-09-21T06:22:46.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.137+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6708036050 con 0x5571f92bd1b0 2024-09-21T06:22:46.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.137+0000 7f66f97fa640 1 -- 172.21.15.31:0/1319588425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6700004430 con 0x5571f92bd1b0 2024-09-21T06:22:46.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.137+0000 7f671e6ab740 10 client.17685 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:46.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.137+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5571f9715fd0 con 0x5571f92bd1b0 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.139+0000 7f66f97fa640 1 -- 172.21.15.31:0/1319588425 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6708036050 con 0x5571f92bd1b0 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f66f97fa640 1 -- 172.21.15.31:0/1319588425 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f670001e640 con 0x5571f92bd1b0 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f671e6ab740 10 client.17685 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f671e6ab740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f671e6ab740 20 client.17685 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f671e6ab740 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92e6070 0x5571f9718840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f671e6ab740 4 client.17685 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571f96018a0 con 0x5571f92e6070 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.140+0000 7f6715ed3640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92e6070 0x5571f9718840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f66f97fa640 10 client.17685 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f6715ed3640 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92e6070 0x5571f9718840 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aec9bf3dc334dcd server_cookie=e95d9a465b067aa in_seq=0 out_seq=0 2024-09-21T06:22:46.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f66f97fa640 1 -- 172.21.15.31:0/1319588425 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571f96018a0 con 0x5571f92e6070 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f66f97fa640 10 client.17685 handle_command_reply: tid=0 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 client.17685 shutdown 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 msgr2=0x5571f969d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f969d900 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f670000fdc0 tx=0x7f6700012820 comp rx=0 tx=0).stop 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 shutdown_connections 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5571f92e6070 0x5571f9718840 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5571f92c5390 0x5571f9714a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5571f92bd1b0 0x5571f969d900 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 --2- 172.21.15.31:0/1319588425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5571f92a49f0 0x5571f969d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 >> 172.21.15.31:0/1319588425 conn(0x5571f92da460 msgr2=0x5571f968e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f66f97fa640 0 client.17685 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.141+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 shutdown_connections 2024-09-21T06:22:46.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.142+0000 7f671e6ab740 1 -- 172.21.15.31:0/1319588425 wait complete. 2024-09-21T06:22:46.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.142+0000 7f671e6ab740 20 client.17685 trim_cache size 0 max 16384 2024-09-21T06:22:46.147 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66fc0b6700 msgr2=0x7f66fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66fc0b6700 0x7f66fc0b8c10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6708006010 tx=0x7f6708031bd0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 msgr2=0x7f671019c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f671019c5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f670c00e9f0 tx=0x7f670c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 shutdown_connections 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66fc0b6700 0x7f66fc0b8c10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6710108d20 0x7f67101a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67101083d0 0x7f671019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 --2- 172.21.15.31:0/3454550438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6710102ae0 0x7f671019c5f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 >> 172.21.15.31:0/3454550438 conn(0x7f6710079790 msgr2=0x7f67101061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:46.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 shutdown_connections 2024-09-21T06:22:46.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:46.143+0000 7f6718137640 1 -- 172.21.15.31:0/3454550438 wait complete. 2024-09-21T06:22:47.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:47 smithi031 ceph-mon[31237]: pgmap v666: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.9 MiB/s wr, 725 op/s 2024-09-21T06:22:47.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:47 smithi110 ceph-mon[34550]: pgmap v666: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.9 MiB/s wr, 725 op/s 2024-09-21T06:22:47.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:47 smithi107 ceph-mon[34380]: pgmap v666: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 8.9 MiB/s wr, 725 op/s 2024-09-21T06:22:48.088 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:22:48.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:22:48.469 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:49.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 -- 172.21.15.31:0/2635078658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878102450 msgr2=0x7f8878105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 --2- 172.21.15.31:0/2635078658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878102450 0x7f8878105fb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f887001acb0 tx=0x7f887003b920 comp rx=0 tx=0).stop 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 -- 172.21.15.31:0/2635078658 shutdown_connections 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 --2- 172.21.15.31:0/2635078658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878102450 0x7f8878105fb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 --2- 172.21.15.31:0/2635078658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878101b00 0x7f8878101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 --2- 172.21.15.31:0/2635078658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878105210 0x7f8878105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 -- 172.21.15.31:0/2635078658 >> 172.21.15.31:0/2635078658 conn(0x7f8878077aa0 msgr2=0x7f8878077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 -- 172.21.15.31:0/2635078658 shutdown_connections 2024-09-21T06:22:49.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.009+0000 7f887759e640 1 -- 172.21.15.31:0/2635078658 wait complete. 2024-09-21T06:22:49.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 Processor -- start 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 -- start start 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878101b00 0x7f88781980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878102450 0x7f8878198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878105210 0x7f887819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f887810b6b0 con 0x7f8878105210 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f887810b530 con 0x7f8878102450 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.011+0000 7f887759e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f887810b830 con 0x7f8878101b00 2024-09-21T06:22:49.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.012+0000 7f887659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878101b00 0x7f88781980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.012+0000 7f887659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878101b00 0x7f88781980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55146/0 (socket says 172.21.15.31:55146) 2024-09-21T06:22:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.012+0000 7f887659c640 1 -- 172.21.15.31:0/2931153010 learned_addr learned my addr 172.21.15.31:0/2931153010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.012+0000 7f886ffff640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878102450 0x7f8878198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f8876d9d640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878105210 0x7f887819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f886ffff640 1 -- 172.21.15.31:0/2931153010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878101b00 msgr2=0x7f88781980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f886ffff640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878101b00 0x7f88781980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f886ffff640 1 -- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878105210 msgr2=0x7f887819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f886ffff640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878105210 0x7f887819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f886ffff640 1 -- 172.21.15.31:0/2931153010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887819e020 con 0x7f8878102450 2024-09-21T06:22:49.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.013+0000 7f886ffff640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878102450 0x7f8878198650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f886000d6a0 tx=0x7f886000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.014+0000 7f886f7fe640 1 -- 172.21.15.31:0/2931153010 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860017070 con 0x7f8878102450 2024-09-21T06:22:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.015+0000 7f886f7fe640 1 -- 172.21.15.31:0/2931153010 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8860009820 con 0x7f8878102450 2024-09-21T06:22:49.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.015+0000 7f886f7fe640 1 -- 172.21.15.31:0/2931153010 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860004c40 con 0x7f8878102450 2024-09-21T06:22:49.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.017+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887819e310 con 0x7f8878102450 2024-09-21T06:22:49.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.017+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88781a4a70 con 0x7f8878102450 2024-09-21T06:22:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.023+0000 7f8883018740 1 Processor -- start 2024-09-21T06:22:49.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.023+0000 7f8883018740 1 -- start start 2024-09-21T06:22:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f886f7fe640 1 -- 172.21.15.31:0/2931153010 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8860005220 con 0x7f8878102450 2024-09-21T06:22:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f8883018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a009610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f8883018740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a009b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f8883018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c41670 0x55f00a00a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f8883018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f009f7fe60 con 0x55f009c62350 2024-09-21T06:22:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f8883018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f009f6f290 con 0x55f009c41670 2024-09-21T06:22:49.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f8883018740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f009f730a0 con 0x55f009c39490 2024-09-21T06:22:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f887659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a009b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f887659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a009b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:55154/0 (socket says 172.21.15.31:55154) 2024-09-21T06:22:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.024+0000 7f887659c640 1 -- 172.21.15.31:0/2482913813 learned_addr learned my addr 172.21.15.31:0/2482913813 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.025+0000 7f8876d9d640 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a009610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.025+0000 7f887659c640 1 -- 172.21.15.31:0/2482913813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c41670 msgr2=0x55f00a00a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.025+0000 7f887659c640 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c41670 0x55f00a00a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.025+0000 7f887659c640 1 -- 172.21.15.31:0/2482913813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 msgr2=0x55f00a009610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.025+0000 7f887659c640 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a009610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.025+0000 7f887659c640 1 -- 172.21.15.31:0/2482913813 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f009ccd930 con 0x55f009c39490 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.026+0000 7f887659c640 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a009b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f886800bd40 tx=0x7f88680107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf5fc78bf50591e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.026+0000 7f886cff9640 1 -- 172.21.15.31:0/2482913813 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886801c680 con 0x55f009c39490 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.026+0000 7f886cff9640 1 -- 172.21.15.31:0/2482913813 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88680118f0 con 0x55f009c39490 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.026+0000 7f886cff9640 1 -- 172.21.15.31:0/2482913813 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886801a740 con 0x55f009c39490 2024-09-21T06:22:49.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.027+0000 7f886f7fe640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88640b6850 0x7f88640b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.027+0000 7f886f7fe640 1 -- 172.21.15.31:0/2931153010 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f88600e7280 con 0x7f8878102450 2024-09-21T06:22:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.027+0000 7f8876d9d640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88640b6850 0x7f88640b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 -- 172.21.15.31:0/2482913813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 msgr2=0x55f00a009b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a009b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f886800bd40 tx=0x7f88680107a0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 -- 172.21.15.31:0/2482913813 shutdown_connections 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c41670 0x55f00a00a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a009b50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 --2- 172.21.15.31:0/2482913813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a009610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.028+0000 7f8883018740 1 -- 172.21.15.31:0/2482913813 >> 172.21.15.31:0/2482913813 conn(0x55f009c56740 msgr2=0x55f00a007500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.029+0000 7f8876d9d640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88640b6850 0x7f88640b8d60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8870006010 tx=0x7f887001a7f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.030+0000 7f8883018740 1 -- 172.21.15.31:0/2482913813 shutdown_connections 2024-09-21T06:22:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.030+0000 7f8883018740 1 -- 172.21.15.31:0/2482913813 wait complete. 2024-09-21T06:22:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.030+0000 7f8883018740 1 -- start start 2024-09-21T06:22:49.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 Processor -- start 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a015260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a0157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c62350 0x55f00a021110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f009dd0890 con 0x55f009c41670 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f009f09d40 con 0x55f009c62350 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f00a017df0 con 0x55f009c39490 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.031+0000 7f8883018740 10 client.0 _ll_register_callbacks cb 0x55f009f6dc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.032+0000 7f886ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a0157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.032+0000 7f886ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a0157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48882/0 (socket says 172.21.15.31:48882) 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.032+0000 7f886ffff640 1 -- 172.21.15.31:0/1758769487 learned_addr learned my addr 172.21.15.31:0/1758769487 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.032+0000 7f887659c640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a015260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.032+0000 7f8876d9d640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c62350 0x55f00a021110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.033+0000 7f887659c640 1 -- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c62350 msgr2=0x55f00a021110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.033+0000 7f887659c640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c62350 0x55f00a021110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.033+0000 7f887659c640 1 -- 172.21.15.31:0/1758769487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 msgr2=0x55f00a0157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.033+0000 7f887659c640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a0157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.033+0000 7f887659c640 1 -- 172.21.15.31:0/1758769487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00a021720 con 0x55f009c39490 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.034+0000 7f887659c640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a015260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8868005f70 tx=0x7f8868011670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c8632dc2ad35137 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.034+0000 7f885dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.034+0000 7f885dffb640 1 -- 172.21.15.31:0/1758769487 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868005040 con 0x55f009c39490 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.034+0000 7f885dffb640 1 -- 172.21.15.31:0/1758769487 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f88680051e0 con 0x55f009c39490 2024-09-21T06:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.034+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88680118f0 con 0x55f009c39490 2024-09-21T06:22:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.034+0000 7f885dffb640 1 -- 172.21.15.31:0/1758769487 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f88680053e0 con 0x55f009c39490 2024-09-21T06:22:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.035+0000 7f8883018740 10 client.27547 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.035+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f00a024050 con 0x55f009c39490 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.038+0000 7f885dffb640 1 -- 172.21.15.31:0/1758769487 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88680118f0 con 0x55f009c39490 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.038+0000 7f885dffb640 1 -- 172.21.15.31:0/1758769487 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8868024070 con 0x55f009c39490 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.038+0000 7f8883018740 10 client.27547 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.038+0000 7f8883018740 10 client.27547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.038+0000 7f8883018740 20 client.27547 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.038+0000 7f8883018740 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009c20cd0 0x55f00a0277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.039+0000 7f8883018740 4 client.27547 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.039+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f009f81db0 con 0x55f009c20cd0 2024-09-21T06:22:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.039+0000 7f886ffff640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009c20cd0 0x55f00a0277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.040+0000 7f885dffb640 10 client.27547 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.041+0000 7f886ffff640 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009c20cd0 0x55f00a0277a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42276da74c8371ba server_cookie=cb38aa01bd6349e0 in_seq=0 out_seq=0 2024-09-21T06:22:49.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.044+0000 7f885dffb640 1 -- 172.21.15.31:0/1758769487 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f009f81db0 con 0x55f009c20cd0 2024-09-21T06:22:49.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f885dffb640 10 client.27547 handle_command_reply: tid=0 2024-09-21T06:22:49.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 client.27547 shutdown 2024-09-21T06:22:49.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 msgr2=0x55f00a015260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a015260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8868005f70 tx=0x7f8868011670 comp rx=0 tx=0).stop 2024-09-21T06:22:49.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 shutdown_connections 2024-09-21T06:22:49.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009c20cd0 0x55f00a0277a0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c62350 0x55f00a021110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a0157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 --2- 172.21.15.31:0/1758769487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c39490 0x55f00a015260 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.045+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 >> 172.21.15.31:0/1758769487 conn(0x55f009c56740 msgr2=0x55f00a00c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.046+0000 7f885dffb640 0 client.27547 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.046+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 shutdown_connections 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.046+0000 7f8883018740 1 -- 172.21.15.31:0/1758769487 wait complete. 2024-09-21T06:22:49.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.047+0000 7f8883018740 20 client.27547 trim_cache size 0 max 16384 2024-09-21T06:22:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.070+0000 7f8883018740 1 Processor -- start 2024-09-21T06:22:49.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.070+0000 7f8883018740 1 -- start start 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8883018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a00e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8883018740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 0x55f00a00ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8883018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a006b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8883018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88781ab520 con 0x55f009c62350 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8883018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88781add80 con 0x55f009c39490 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8883018740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88781abb70 con 0x55f009c41670 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 0x55f00a00ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 0x55f00a00ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33764/0 (socket says 172.21.15.31:33764) 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 -- 172.21.15.31:0/672063165 learned_addr learned my addr 172.21.15.31:0/672063165 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f887659c640 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a00e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8876d9d640 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a006b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 -- 172.21.15.31:0/672063165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 msgr2=0x55f00a00e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a00e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 -- 172.21.15.31:0/672063165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 msgr2=0x55f00a006b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a006b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f886ffff640 1 -- 172.21.15.31:0/672063165 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f00a016320 con 0x55f009c41670 2024-09-21T06:22:49.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.071+0000 7f8876d9d640 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a006b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.072+0000 7f886ffff640 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 0x55f00a00ec60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f886000b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510f9c0d6b5f5e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.072+0000 7f885ffff640 1 -- 172.21.15.31:0/672063165 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600d4070 con 0x55f009c41670 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.072+0000 7f885ffff640 1 -- 172.21.15.31:0/672063165 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88600acc10 con 0x55f009c41670 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.072+0000 7f885ffff640 1 -- 172.21.15.31:0/672063165 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600099d0 con 0x55f009c41670 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.073+0000 7f8883018740 1 -- 172.21.15.31:0/672063165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 msgr2=0x55f00a00ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.073+0000 7f8883018740 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 0x55f00a00ec60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f886000b400 comp rx=0 tx=0).stop 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.073+0000 7f8883018740 1 -- 172.21.15.31:0/672063165 shutdown_connections 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.073+0000 7f8883018740 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c62350 0x55f00a006b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.073+0000 7f8883018740 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c41670 0x55f00a00ec60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.073+0000 7f8883018740 1 --2- 172.21.15.31:0/672063165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a00e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.074+0000 7f8883018740 1 -- 172.21.15.31:0/672063165 >> 172.21.15.31:0/672063165 conn(0x55f009c56740 msgr2=0x55f00a0138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.074+0000 7f8883018740 1 -- 172.21.15.31:0/672063165 shutdown_connections 2024-09-21T06:22:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.074+0000 7f8883018740 1 -- 172.21.15.31:0/672063165 wait complete. 2024-09-21T06:22:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.075+0000 7f8883018740 1 -- start start 2024-09-21T06:22:49.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.075+0000 7f8883018740 1 Processor -- start 2024-09-21T06:22:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.075+0000 7f8883018740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a018840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.075+0000 7f8883018740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a018d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f8883018740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c62350 0x55f00a090150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f8883018740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88781ac240 con 0x55f009c41670 2024-09-21T06:22:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f8883018740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88781ad6b0 con 0x55f009c39490 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f8883018740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f88781ac910 con 0x55f009c62350 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a018840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a018840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34328/0 (socket says 172.21.15.31:34328) 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 -- 172.21.15.31:0/2844959572 learned_addr learned my addr 172.21.15.31:0/2844959572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f886ffff640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a018d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f8883018740 10 client.0 _ll_register_callbacks cb 0x7f886800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f8876d9d640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c62350 0x55f00a090150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 -- 172.21.15.31:0/2844959572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c62350 msgr2=0x55f00a090150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c62350 0x55f00a090150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 -- 172.21.15.31:0/2844959572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 msgr2=0x55f00a018d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a018d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 -- 172.21.15.31:0/2844959572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00a090690 con 0x55f009c39490 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f886ffff640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a018d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f887659c640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a018840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f886800b770 tx=0x7f886801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba0049f7ea37d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.076+0000 7f885dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f885dffb640 1 -- 172.21.15.31:0/2844959572 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868031040 con 0x55f009c39490 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f885dffb640 1 -- 172.21.15.31:0/2844959572 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f8868011760 con 0x55f009c39490 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88600acc10 con 0x55f009c39490 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f885dffb640 1 -- 172.21.15.31:0/2844959572 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f886801c9a0 con 0x55f009c39490 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f8883018740 10 client.27491 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:49.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f00a091890 con 0x55f009c39490 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.077+0000 7f885dffb640 1 -- 172.21.15.31:0/2844959572 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886800b480 con 0x55f009c39490 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f885dffb640 1 -- 172.21.15.31:0/2844959572 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8868024040 con 0x55f009c39490 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f8883018740 10 client.27491 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f8883018740 10 client.27491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f8883018740 20 client.27491 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f8883018740 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009bf6080 0x55f00a093f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f8883018740 4 client.27491 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f88600acc10 con 0x55f009bf6080 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f886ffff640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009bf6080 0x55f00a093f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f886ffff640 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009bf6080 0x55f00a093f30 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6a39f4505e363c server_cookie=29781a76f243393c in_seq=0 out_seq=0 2024-09-21T06:22:49.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.079+0000 7f885dffb640 10 client.27491 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f885dffb640 1 -- 172.21.15.31:0/2844959572 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f88600acc10 con 0x55f009bf6080 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f885dffb640 10 client.27491 handle_command_reply: tid=0 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 client.27491 shutdown 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 msgr2=0x55f00a018840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a018840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f886800b770 tx=0x7f886801c900 comp rx=0 tx=0).stop 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 shutdown_connections 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f009bf6080 0x55f00a093f30 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f885dffb640 0 client.27491 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f009c62350 0x55f00a090150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f009c41670 0x55f00a018d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 --2- 172.21.15.31:0/2844959572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f009c39490 0x55f00a018840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 >> 172.21.15.31:0/2844959572 conn(0x55f009c56740 msgr2=0x55f00a00e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 shutdown_connections 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.082+0000 7f8883018740 1 -- 172.21.15.31:0/2844959572 wait complete. 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f8883018740 20 client.27491 trim_cache size 0 max 16384 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571", 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:22:49.088 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88640b6850 msgr2=0x7f88640b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88640b6850 0x7f88640b8d60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8870006010 tx=0x7f887001a7f0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878102450 msgr2=0x7f8878198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878102450 0x7f8878198650 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f886000d6a0 tx=0x7f886000db60 comp rx=0 tx=0).stop 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 shutdown_connections 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f88640b6850 0x7f88640b8d60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8878105210 0x7f887819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8878102450 0x7f8878198650 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 --2- 172.21.15.31:0/2931153010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8878101b00 0x7f88781980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.083+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 >> 172.21.15.31:0/2931153010 conn(0x7f8878077aa0 msgr2=0x7f8878100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.084+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 shutdown_connections 2024-09-21T06:22:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:49.084+0000 7f887759e640 1 -- 172.21.15.31:0/2931153010 wait complete. 2024-09-21T06:22:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.396+0000 7f0d023bb640 1 -- 172.21.15.31:0/1358153203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 msgr2=0x7f0cfc10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.396+0000 7f0d023bb640 1 --2- 172.21.15.31:0/1358153203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc10a3c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009ca0 tx=0x7f0cf002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.400+0000 7f0d023bb640 1 -- 172.21.15.31:0/1358153203 shutdown_connections 2024-09-21T06:22:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.400+0000 7f0d023bb640 1 --2- 172.21.15.31:0/1358153203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cfc11cba0 0x7f0cfc11efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.400+0000 7f0d023bb640 1 --2- 172.21.15.31:0/1358153203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc10a900 0x7f0cfc10ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.400+0000 7f0d023bb640 1 --2- 172.21.15.31:0/1358153203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc10a3c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.400+0000 7f0d023bb640 1 -- 172.21.15.31:0/1358153203 >> 172.21.15.31:0/1358153203 conn(0x7f0cfc070c00 msgr2=0x7f0cfc071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.406+0000 7f0d023bb640 1 -- 172.21.15.31:0/1358153203 shutdown_connections 2024-09-21T06:22:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.407+0000 7f0d023bb640 1 -- 172.21.15.31:0/1358153203 wait complete. 2024-09-21T06:22:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.408+0000 7f0d023bb640 1 Processor -- start 2024-09-21T06:22:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 -- start start 2024-09-21T06:22:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cfc10a900 0x7f0cfc1a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc1a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc11cba0 0x7f0cfc1ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1226f0 con 0x7f0cfc111ca0 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc122570 con 0x7f0cfc10a900 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.409+0000 7f0d023bb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc122870 con 0x7f0cfc11cba0 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d013b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cfc10a900 0x7f0cfc1a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc1a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc1a96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54080/0 (socket says 172.21.15.31:54080) 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 -- 172.21.15.31:0/68334485 learned_addr learned my addr 172.21.15.31:0/68334485 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d01bba640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc11cba0 0x7f0cfc1ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 -- 172.21.15.31:0/68334485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc11cba0 msgr2=0x7f0cfc1ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc11cba0 0x7f0cfc1ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 -- 172.21.15.31:0/68334485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cfc10a900 msgr2=0x7f0cfc1a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cfc10a900 0x7f0cfc1a9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.410+0000 7f0d00bb8640 1 -- 172.21.15.31:0/68334485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1af060 con 0x7f0cfc111ca0 2024-09-21T06:22:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.411+0000 7f0d01bba640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc11cba0 0x7f0cfc1ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:22:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.412+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc1a96d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b470 tx=0x7f0cf800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.413+0000 7f0cee7fc640 1 -- 172.21.15.31:0/68334485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8015020 con 0x7f0cfc111ca0 2024-09-21T06:22:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.413+0000 7f0cee7fc640 1 -- 172.21.15.31:0/68334485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf8011d40 con 0x7f0cfc111ca0 2024-09-21T06:22:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.413+0000 7f0cee7fc640 1 -- 172.21.15.31:0/68334485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800f340 con 0x7f0cfc111ca0 2024-09-21T06:22:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.413+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc1af350 con 0x7f0cfc111ca0 2024-09-21T06:22:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.413+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1b5ae0 con 0x7f0cfc111ca0 2024-09-21T06:22:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.418+0000 7f0cee7fc640 1 -- 172.21.15.31:0/68334485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0cf8018020 con 0x7f0cfc111ca0 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.418+0000 7f0d0a664740 1 Processor -- start 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 -- start start 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df24eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df24f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df24f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c0df1c36f0 con 0x55c0dee884d0 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0df1b2b20 con 0x55c0deea91b0 2024-09-21T06:22:49.421 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d0a664740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c0df1b6930 con 0x55c0dee802f0 2024-09-21T06:22:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df24f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d01bba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df24eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df24f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54090/0 (socket says 172.21.15.31:54090) 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d00bb8640 1 -- 172.21.15.31:0/3239967199 learned_addr learned my addr 172.21.15.31:0/3239967199 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d01bba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df24eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34352/0 (socket says 172.21.15.31:34352) 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.419+0000 7f0d013b9640 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df24f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0cee7fc640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ce80b67b0 0x7f0ce80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 -- 172.21.15.31:0/3239967199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 msgr2=0x55c0df24f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df24f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 -- 172.21.15.31:0/3239967199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 msgr2=0x55c0df24f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df24f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 -- 172.21.15.31:0/3239967199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0df178530 con 0x55c0deea91b0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0cee7fc640 1 -- 172.21.15.31:0/68334485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f0cf80e7190 con 0x7f0cfc111ca0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d013b9640 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df24f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ce80b67b0 0x7f0ce80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0d01bba640 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df24eba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400ba60 tx=0x7f0cf400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd78ce5cd688b18c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.420+0000 7f0ce77fe640 1 -- 172.21.15.31:0/3239967199 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf401d070 con 0x55c0deea91b0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.421+0000 7f0ce77fe640 1 -- 172.21.15.31:0/3239967199 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf4018c40 con 0x55c0deea91b0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.421+0000 7f0d01bba640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ce80b67b0 0x7f0ce80b8cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0cf401a000 tx=0x7f0cf400ff90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.421+0000 7f0ce77fe640 1 -- 172.21.15.31:0/3239967199 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400e3b0 con 0x55c0deea91b0 2024-09-21T06:22:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 -- 172.21.15.31:0/3239967199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 msgr2=0x55c0df24eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df24eba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400ba60 tx=0x7f0cf400bf20 comp rx=0 tx=0).stop 2024-09-21T06:22:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 -- 172.21.15.31:0/3239967199 shutdown_connections 2024-09-21T06:22:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df24f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df24f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3239967199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df24eba0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 -- 172.21.15.31:0/3239967199 >> 172.21.15.31:0/3239967199 conn(0x55c0dee9d300 msgr2=0x55c0df24ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 -- 172.21.15.31:0/3239967199 shutdown_connections 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 -- 172.21.15.31:0/3239967199 wait complete. 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 -- start start 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.422+0000 7f0d0a664740 1 Processor -- start 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee802f0 0x55c0df25a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df25a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df2665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c0df0eaab0 con 0x55c0dee802f0 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0df158730 con 0x55c0deea91b0 2024-09-21T06:22:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c0df25d2c0 con 0x55c0dee884d0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d013b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee802f0 0x55c0df25a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df25a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d0a664740 10 client.0 _ll_register_callbacks cb 0x55c0df1ac6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df25a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33784/0 (socket says 172.21.15.31:33784) 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d00bb8640 1 -- 172.21.15.31:0/3700672311 learned_addr learned my addr 172.21.15.31:0/3700672311 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.423+0000 7f0d01bba640 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df2665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0d00bb8640 1 -- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 msgr2=0x55c0df2665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df2665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0d00bb8640 1 -- 172.21.15.31:0/3700672311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee802f0 msgr2=0x55c0df25a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee802f0 0x55c0df25a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0d00bb8640 1 -- 172.21.15.31:0/3700672311 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0def14790 con 0x55c0dee884d0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df25a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80acf20 tx=0x7f0cf8010fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701435053cfa9da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0ce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/3700672311 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80acd60 con 0x55c0dee884d0 2024-09-21T06:22:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.424+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/3700672311 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f0cf80e90d0 con 0x55c0dee884d0 2024-09-21T06:22:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.425+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf4018c40 con 0x55c0dee884d0 2024-09-21T06:22:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.425+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/3700672311 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80ac700 con 0x55c0dee884d0 2024-09-21T06:22:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.426+0000 7f0d0a664740 10 client.27559 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.426+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c0def8d620 con 0x55c0dee884d0 2024-09-21T06:22:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.426+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/3700672311 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf4018c40 con 0x55c0dee884d0 2024-09-21T06:22:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/3700672311 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0cf80940d0 con 0x55c0dee884d0 2024-09-21T06:22:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d0a664740 10 client.27559 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d0a664740 10 client.27559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d0a664740 20 client.27559 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee67d80 0x55c0df26c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d0a664740 4 client.27559 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0df1c5640 con 0x55c0dee67d80 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.427+0000 7f0d013b9640 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee67d80 0x55c0df26c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.428+0000 7f0ce4ff9640 10 client.27559 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.428+0000 7f0d013b9640 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee67d80 0x55c0df26c330 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8e0e80487c60f9 server_cookie=a3b4a542c2cb81d5 in_seq=0 out_seq=0 2024-09-21T06:22:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.429+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/3700672311 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0df1c5640 con 0x55c0dee67d80 2024-09-21T06:22:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.429+0000 7f0ce4ff9640 10 client.27559 handle_command_reply: tid=0 2024-09-21T06:22:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.429+0000 7f0d0a664740 1 client.27559 shutdown 2024-09-21T06:22:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.429+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 msgr2=0x55c0df25a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.429+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df25a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80acf20 tx=0x7f0cf8010fc0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 shutdown_connections 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee67d80 0x55c0df26c330 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df2665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0ce4ff9640 0 client.27559 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df25a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 --2- 172.21.15.31:0/3700672311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee802f0 0x55c0df25a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 >> 172.21.15.31:0/3700672311 conn(0x55c0dee9d300 msgr2=0x55c0df24fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 shutdown_connections 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 1 -- 172.21.15.31:0/3700672311 wait complete. 2024-09-21T06:22:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.430+0000 7f0d0a664740 20 client.27559 trim_cache size 0 max 16384 2024-09-21T06:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.455+0000 7f0d0a664740 1 Processor -- start 2024-09-21T06:22:49.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.455+0000 7f0d0a664740 1 -- start start 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df25a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df25be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df257430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d0a664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1bc480 con 0x55c0dee884d0 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d013b9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df25a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d01bba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df257430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df25be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d013b9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df25a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33790/0 (socket says 172.21.15.31:33790) 2024-09-21T06:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d013b9640 1 -- 172.21.15.31:0/1196821356 learned_addr learned my addr 172.21.15.31:0/1196821356 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1bece0 con 0x55c0deea91b0 2024-09-21T06:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.456+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1bcad0 con 0x55c0dee802f0 2024-09-21T06:22:49.459 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0d01bba640 1 -- 172.21.15.31:0/1196821356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 msgr2=0x55c0df25a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0d01bba640 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df25a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0d01bba640 1 -- 172.21.15.31:0/1196821356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 msgr2=0x55c0df25be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0d01bba640 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df25be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0d01bba640 1 -- 172.21.15.31:0/1196821356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0df26c6a0 con 0x55c0deea91b0 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0d01bba640 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df257430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400fa10 tx=0x7f0cf4024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df9f16a70cba576 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0ce6ffd640 1 -- 172.21.15.31:0/1196821356 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf402a040 con 0x55c0deea91b0 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0ce6ffd640 1 -- 172.21.15.31:0/1196821356 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf400f800 con 0x55c0deea91b0 2024-09-21T06:22:49.460 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.457+0000 7f0ce6ffd640 1 -- 172.21.15.31:0/1196821356 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf401d040 con 0x55c0deea91b0 2024-09-21T06:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.458+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 msgr2=0x55c0df257430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.458+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df257430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400fa10 tx=0x7f0cf4024000 comp rx=0 tx=0).stop 2024-09-21T06:22:49.461 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 shutdown_connections 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0deea91b0 0x55c0df257430 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0dee884d0 0x55c0df25be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1196821356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee802f0 0x55c0df25a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 >> 172.21.15.31:0/1196821356 conn(0x55c0dee9d300 msgr2=0x55c0df25b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 shutdown_connections 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 -- 172.21.15.31:0/1196821356 wait complete. 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 -- start start 2024-09-21T06:22:49.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.459+0000 7f0d0a664740 1 Processor -- start 2024-09-21T06:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0dee802f0 0x55c0df25fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df2600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0deea91b0 0x55c0df2d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d013b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0dee802f0 0x55c0df25fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d00bb8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df2600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1bd1a0 con 0x55c0deea91b0 2024-09-21T06:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1be610 con 0x55c0dee802f0 2024-09-21T06:22:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1bd870 con 0x55c0dee884d0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d013b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0dee802f0 0x55c0df25fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34384/0 (socket says 172.21.15.31:34384) 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d013b9640 1 -- 172.21.15.31:0/1715981000 learned_addr learned my addr 172.21.15.31:0/1715981000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d01bba640 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0deea91b0 0x55c0df2d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.461+0000 7f0d0a664740 10 client.0 _ll_register_callbacks cb 0x7f0cf80092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d00bb8640 1 -- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0dee802f0 msgr2=0x55c0df25fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0dee802f0 0x55c0df25fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d00bb8640 1 -- 172.21.15.31:0/1715981000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0deea91b0 msgr2=0x55c0df2d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0deea91b0 0x55c0df2d4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d00bb8640 1 -- 172.21.15.31:0/1715981000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0df2d51e0 con 0x55c0dee884d0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d00bb8640 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df2600c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80a9560 tx=0x7f0cf80e9300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49c75d1c5874b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0ce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/1715981000 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80d4040 con 0x55c0dee884d0 2024-09-21T06:22:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/1715981000 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f0cf800eb90 con 0x55c0dee884d0 2024-09-21T06:22:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.462+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf400f800 con 0x55c0dee884d0 2024-09-21T06:22:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.463+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/1715981000 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80ac8e0 con 0x55c0dee884d0 2024-09-21T06:22:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.464+0000 7f0d0a664740 10 client.27565 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.464+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c0df2d6370 con 0x55c0dee884d0 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/1715981000 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf400f800 con 0x55c0dee884d0 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/1715981000 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0cf80d41e0 con 0x55c0dee884d0 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d0a664740 10 client.27565 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d0a664740 10 client.27565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d0a664740 20 client.27565 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee3d1c0 0x55c0df2d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:49.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d0a664740 4 client.27565 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:22:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0df1c10f0 con 0x55c0dee3d1c0 2024-09-21T06:22:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.465+0000 7f0d013b9640 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee3d1c0 0x55c0df2d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.466+0000 7f0ce4ff9640 10 client.27565 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.466+0000 7f0d013b9640 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee3d1c0 0x55c0df2d8af0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e0ac412d19d276 server_cookie=f4435c05939336c7 in_seq=0 out_seq=0 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0ce4ff9640 1 -- 172.21.15.31:0/1715981000 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0df1c10f0 con 0x55c0dee3d1c0 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0ce4ff9640 10 client.27565 handle_command_reply: tid=0 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 client.27565 shutdown 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 msgr2=0x55c0df2600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df2600c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80a9560 tx=0x7f0cf80e9300 comp rx=0 tx=0).stop 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 shutdown_connections 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0dee3d1c0 0x55c0df2d8af0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0ce4ff9640 0 client.27565 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0deea91b0 0x55c0df2d4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0dee884d0 0x55c0df2600c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 --2- 172.21.15.31:0/1715981000 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0dee802f0 0x55c0df25fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.514+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 >> 172.21.15.31:0/1715981000 conn(0x55c0dee9d300 msgr2=0x55c0df25bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.515+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 shutdown_connections 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.515+0000 7f0d0a664740 1 -- 172.21.15.31:0/1715981000 wait complete. 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.515+0000 7f0d0a664740 20 client.27565 trim_cache size 0 max 16384 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:22:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:22:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ce80b67b0 msgr2=0x7f0ce80b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ce80b67b0 0x7f0ce80b8cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0cf401a000 tx=0x7f0cf400ff90 comp rx=0 tx=0).stop 2024-09-21T06:22:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 msgr2=0x7f0cfc1a96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc1a96d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b470 tx=0x7f0cf800b930 comp rx=0 tx=0).stop 2024-09-21T06:22:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 shutdown_connections 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cfc11cba0 0x7f0cfc1ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0ce80b67b0 0x7f0ce80b8cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cfc111ca0 0x7f0cfc1a96d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 --2- 172.21.15.31:0/68334485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cfc10a900 0x7f0cfc1a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 >> 172.21.15.31:0/68334485 conn(0x7f0cfc070c00 msgr2=0x7f0cfc11ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 shutdown_connections 2024-09-21T06:22:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:22:49.516+0000 7f0d023bb640 1 -- 172.21.15.31:0/68334485 wait complete. 2024-09-21T06:22:49.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:49 smithi031 ceph-mon[31237]: pgmap v667: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 606 op/s 2024-09-21T06:22:49.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:49 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:22:49.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:49 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:49.607 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:49 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:22:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:49 smithi110 ceph-mon[34550]: pgmap v667: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 606 op/s 2024-09-21T06:22:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:49 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:22:49.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:49 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:49.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:49 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:22:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:49 smithi107 ceph-mon[34380]: pgmap v667: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 606 op/s 2024-09-21T06:22:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:49 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:22:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:49 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:49 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:22:50.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:22:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 670.0 s, 1556.6 tps, lat 63.965 ms stddev 68.082 2024-09-21T06:22:50.453 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:22:50.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:50 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:50.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:50 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:50 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:22:51.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.038+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2293131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c104840 msgr2=0x7f639c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.038+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2293131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c104840 0x7f639c104c50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f638c009900 tx=0x7f638c02a410 comp rx=0 tx=0).stop 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2293131404 shutdown_connections 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2293131404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f639c10ddf0 0x7f639c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2293131404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c105270 0x7f639c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2293131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c104840 0x7f639c104c50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2293131404 >> 172.21.15.31:0/2293131404 conn(0x7f639c079730 msgr2=0x7f639c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2293131404 shutdown_connections 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.039+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2293131404 wait complete. 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 Processor -- start 2024-09-21T06:22:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 -- start start 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c104840 0x7f639c104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f639c105270 0x7f639c102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 0x7f639c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f639c113a70 con 0x7f639c10ddf0 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1138f0 con 0x7f639c105270 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f63a1f8e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f639c113bf0 con 0x7f639c104840 2024-09-21T06:22:51.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.040+0000 7f639bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 0x7f639c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 0x7f639c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54116/0 (socket says 172.21.15.31:54116) 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639bfff640 1 -- 172.21.15.31:0/2192342592 learned_addr learned my addr 172.21.15.31:0/2192342592 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c104840 0x7f639c104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639affd640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f639c105270 0x7f639c102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 -- 172.21.15.31:0/2192342592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f639c105270 msgr2=0x7f639c102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f639c105270 0x7f639c102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 -- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 msgr2=0x7f639c102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 0x7f639c102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 -- 172.21.15.31:0/2192342592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c1034b0 con 0x7f639c104840 2024-09-21T06:22:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639bfff640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 0x7f639c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:22:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.041+0000 7f639b7fe640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c104840 0x7f639c104160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f638c02a930 tx=0x7f638c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.042+0000 7f6398ff9640 1 -- 172.21.15.31:0/2192342592 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c03f070 con 0x7f639c104840 2024-09-21T06:22:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.042+0000 7f6398ff9640 1 -- 172.21.15.31:0/2192342592 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f638c004490 con 0x7f639c104840 2024-09-21T06:22:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.042+0000 7f6398ff9640 1 -- 172.21.15.31:0/2192342592 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c035070 con 0x7f639c104840 2024-09-21T06:22:51.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.043+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c1acec0 con 0x7f639c104840 2024-09-21T06:22:51.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.043+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1ad2c0 con 0x7f639c104840 2024-09-21T06:22:51.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.046+0000 7f6398ff9640 1 -- 172.21.15.31:0/2192342592 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f638c03b460 con 0x7f639c104840 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.046+0000 7f63a8502740 1 Processor -- start 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.046+0000 7f63a8502740 1 -- start start 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f63a8502740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df495650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f63a8502740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0c51b0 0x5623df495b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f63a8502740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0cd390 0x5623df4960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f63a8502740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5623df40be50 con 0x5623df0c51b0 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f63a8502740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623df3fb280 con 0x5623df0ee070 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f63a8502740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5623df3ff090 con 0x5623df0cd390 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f6398ff9640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63840b6850 0x7f63840b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f639bfff640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63840b6850 0x7f63840b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f6398ff9640 1 -- 172.21.15.31:0/2192342592 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f638c107aa0 con 0x7f639c104840 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f639bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df495650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f639bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df495650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:34410/0 (socket says 172.21.15.31:34410) 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f639bfff640 1 -- 172.21.15.31:0/442295682 learned_addr learned my addr 172.21.15.31:0/442295682 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f639b7fe640 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0c51b0 0x5623df495b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.047+0000 7f639affd640 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0cd390 0x5623df4960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 -- 172.21.15.31:0/442295682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0cd390 msgr2=0x5623df4960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0cd390 0x5623df4960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 -- 172.21.15.31:0/442295682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0c51b0 msgr2=0x5623df495b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0c51b0 0x5623df495b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 -- 172.21.15.31:0/442295682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623df159650 con 0x5623df0ee070 2024-09-21T06:22:51.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63840b6850 0x7f63840b8d60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6394011d20 tx=0x7f63940114d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.048+0000 7f639bfff640 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df495650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6394013cc0 tx=0x7f639401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860a573c89a38206 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.049+0000 7f6381ffb640 1 -- 172.21.15.31:0/442295682 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394023070 con 0x5623df0ee070 2024-09-21T06:22:51.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.049+0000 7f6381ffb640 1 -- 172.21.15.31:0/442295682 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6394010db0 con 0x5623df0ee070 2024-09-21T06:22:51.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.049+0000 7f6381ffb640 1 -- 172.21.15.31:0/442295682 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639401e7a0 con 0x5623df0ee070 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 -- 172.21.15.31:0/442295682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 msgr2=0x5623df495650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df495650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6394013cc0 tx=0x7f639401b000 comp rx=0 tx=0).stop 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 -- 172.21.15.31:0/442295682 shutdown_connections 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0cd390 0x5623df4960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0c51b0 0x5623df495b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 --2- 172.21.15.31:0/442295682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df495650 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 -- 172.21.15.31:0/442295682 >> 172.21.15.31:0/442295682 conn(0x5623df0e2460 msgr2=0x5623df493540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.051+0000 7f63a8502740 1 -- 172.21.15.31:0/442295682 shutdown_connections 2024-09-21T06:22:51.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.052+0000 7f63a8502740 1 -- 172.21.15.31:0/442295682 wait complete. 2024-09-21T06:22:51.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.052+0000 7f63a8502740 1 -- start start 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.052+0000 7f63a8502740 1 Processor -- start 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f63a8502740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f63a8502740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df4a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f63a8502740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f63a8502740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5623df25c700 con 0x5623df0ee070 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f639b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f63a8502740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623df396110 con 0x5623df0cd390 2024-09-21T06:22:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f639b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33840/0 (socket says 172.21.15.31:33840) 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f639b7fe640 1 -- 172.21.15.31:0/3178732727 learned_addr learned my addr 172.21.15.31:0/3178732727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.053+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5623df4a3dd0 con 0x5623df0c51b0 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639bfff640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639affd640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df4a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f63a8502740 10 client.0 _ll_register_callbacks cb 0x5623df3af990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639b7fe640 1 -- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 msgr2=0x5623df4a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639b7fe640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df4a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639b7fe640 1 -- 172.21.15.31:0/3178732727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 msgr2=0x5623df4ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639b7fe640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639b7fe640 1 -- 172.21.15.31:0/3178732727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623df3c3c40 con 0x5623df0c51b0 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639affd640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df4a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639bfff640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6394010db0 con 0x5623df0c51b0 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f639b7fe640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a0d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f638c031440 tx=0x7f638c0c97f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa18a3d4e6fd060 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f637affd640 10 client.27577 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f637affd640 1 -- 172.21.15.31:0/3178732727 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c048040 con 0x5623df0c51b0 2024-09-21T06:22:51.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.054+0000 7f637affd640 1 -- 172.21.15.31:0/3178732727 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f638c0ef610 con 0x5623df0c51b0 2024-09-21T06:22:51.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f637affd640 1 -- 172.21.15.31:0/3178732727 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f638c0cd0c0 con 0x5623df0c51b0 2024-09-21T06:22:51.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 10 client.27577 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:51.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=21}) -- 0x5623df4afa10 con 0x5623df0c51b0 2024-09-21T06:22:51.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f637affd640 1 -- 172.21.15.31:0/3178732727 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f638c04b060 con 0x5623df0c51b0 2024-09-21T06:22:51.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 10 client.27577 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:51.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:51.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 20 client.27577 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:51.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df0ac9f0 0x5623df4b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 4 client.27577 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:22:51.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.055+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623df40dda0 con 0x5623df0ac9f0 2024-09-21T06:22:51.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.056+0000 7f639affd640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df0ac9f0 0x5623df4b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.056+0000 7f637affd640 10 client.27577 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.056+0000 7f639affd640 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df0ac9f0 0x5623df4b2ef0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e835367cda4735cf server_cookie=f9886d08f13fb6b in_seq=0 out_seq=0 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.057+0000 7f637affd640 1 -- 172.21.15.31:0/3178732727 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5623df40dda0 con 0x5623df0ac9f0 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.057+0000 7f637affd640 10 client.27577 handle_command_reply: tid=0 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.057+0000 7f63a8502740 1 client.27577 shutdown 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 msgr2=0x5623df4a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a0d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f638c031440 tx=0x7f638c0c97f0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 shutdown_connections 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df0ac9f0 0x5623df4b2ef0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df4a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 --2- 172.21.15.31:0/3178732727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a0d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f637affd640 0 client.27577 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 >> 172.21.15.31:0/3178732727 conn(0x5623df0e2460 msgr2=0x5623df498830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 shutdown_connections 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 1 -- 172.21.15.31:0/3178732727 wait complete. 2024-09-21T06:22:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.058+0000 7f63a8502740 20 client.27577 trim_cache size 0 max 16384 2024-09-21T06:22:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.082+0000 7f63a8502740 1 Processor -- start 2024-09-21T06:22:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.082+0000 7f63a8502740 1 -- start start 2024-09-21T06:22:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f63a8502740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f63a8502740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df496d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f63a8502740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f63a8502740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b3c10 con 0x5623df0ee070 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f63a8502740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b6470 con 0x5623df0cd390 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f63a8502740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b4260 con 0x5623df0c51b0 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54162/0 (socket says 172.21.15.31:54162) 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33856/0 (socket says 172.21.15.31:33856) 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639bfff640 1 -- 172.21.15.31:0/3866927368 learned_addr learned my addr 172.21.15.31:0/3866927368 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639affd640 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df496d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 -- 172.21.15.31:0/3866927368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 msgr2=0x5623df496d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df496d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 -- 172.21.15.31:0/3866927368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 msgr2=0x5623df4acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.083+0000 7f639b7fe640 1 -- 172.21.15.31:0/3866927368 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623df4984b0 con 0x5623df0c51b0 2024-09-21T06:22:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.084+0000 7f639b7fe640 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a2c70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f638c005d00 tx=0x7f638c044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=124dd4501e24a38d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.084+0000 7f63817fa640 1 -- 172.21.15.31:0/3866927368 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c048040 con 0x5623df0c51b0 2024-09-21T06:22:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.084+0000 7f63817fa640 1 -- 172.21.15.31:0/3866927368 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f638c0ec070 con 0x5623df0c51b0 2024-09-21T06:22:51.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.084+0000 7f63817fa640 1 -- 172.21.15.31:0/3866927368 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0c90c0 con 0x5623df0c51b0 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 -- 172.21.15.31:0/3866927368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 msgr2=0x5623df4a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a2c70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f638c005d00 tx=0x7f638c044000 comp rx=0 tx=0).stop 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 -- 172.21.15.31:0/3866927368 shutdown_connections 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0ee070 0x5623df4acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0cd390 0x5623df496d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 --2- 172.21.15.31:0/3866927368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a2c70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 -- 172.21.15.31:0/3866927368 >> 172.21.15.31:0/3866927368 conn(0x5623df0e2460 msgr2=0x5623df496680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 -- 172.21.15.31:0/3866927368 shutdown_connections 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.085+0000 7f63a8502740 1 -- 172.21.15.31:0/3866927368 wait complete. 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.086+0000 7f63a8502740 1 -- start start 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.086+0000 7f63a8502740 1 Processor -- start 2024-09-21T06:22:51.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f63a8502740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f63a8502740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0cd390 0x5623df4a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f63a8502740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df4a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f639b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f63a8502740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b4930 con 0x5623df0cd390 2024-09-21T06:22:51.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f63a8502740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b5da0 con 0x5623df0ee070 2024-09-21T06:22:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f639b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33864/0 (socket says 172.21.15.31:33864) 2024-09-21T06:22:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f639b7fe640 1 -- 172.21.15.31:0/1989143742 learned_addr learned my addr 172.21.15.31:0/1989143742 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:22:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f639affd640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0cd390 0x5623df4a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.087+0000 7f63a8502740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1b5000 con 0x5623df0c51b0 2024-09-21T06:22:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.089+0000 7f639bfff640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df4a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.089+0000 7f639b7fe640 1 -- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 msgr2=0x5623df4a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.089+0000 7f639b7fe640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df4a8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.089+0000 7f639b7fe640 1 -- 172.21.15.31:0/1989143742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0cd390 msgr2=0x5623df4a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:22:51.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.089+0000 7f639b7fe640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0cd390 0x5623df4a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.089+0000 7f63a8502740 10 client.0 _ll_register_callbacks cb 0x7f638c003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f639b7fe640 1 -- 172.21.15.31:0/1989143742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623df51d8a0 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638c0ec070 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f639b7fe640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a9a10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f638c0c9a00 tx=0x7f638c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91f07a9c3e7b4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f637affd640 10 client.27583 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f637affd640 1 -- 172.21.15.31:0/1989143742 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c048070 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f637affd640 1 -- 172.21.15.31:0/1989143742 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f638c0d6e30 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f637affd640 1 -- 172.21.15.31:0/1989143742 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f638c0cd720 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f63a8502740 10 client.27583 fetch_fsmap learned FSMap version 21 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.090+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=21}) -- 0x5623df51d8a0 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f637affd640 1 -- 172.21.15.31:0/1989143742 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f638c0ccbb0 con 0x5623df0c51b0 2024-09-21T06:22:51.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f63a8502740 10 client.27583 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f63a8502740 10 client.27583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f63a8502740 20 client.27583 populate_metadata read hostname 'smithi031' 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f63a8502740 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df081da0 0x5623df520b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f63a8502740 4 client.27583 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f638c0ec070 con 0x5623df081da0 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f639affd640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df081da0 0x5623df520b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f637affd640 10 client.27583 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:51.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.091+0000 7f639affd640 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df081da0 0x5623df520b40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df0ceb04161256b server_cookie=3e0e7964584e4781 in_seq=0 out_seq=0 2024-09-21T06:22:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f637affd640 1 -- 172.21.15.31:0/1989143742 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7954 (crc 0 0 0) 0x7f638c0ec070 con 0x5623df081da0 2024-09-21T06:22:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f637affd640 10 client.27583 handle_command_reply: tid=0 2024-09-21T06:22:51.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 client.27583 shutdown 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 msgr2=0x5623df4a9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a9a10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f638c0c9a00 tx=0x7f638c02a410 comp rx=0 tx=0).stop 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 shutdown_connections 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5623df081da0 0x5623df520b40 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623df0ee070 0x5623df4a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f637affd640 0 client.27583 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623df0cd390 0x5623df4a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 --2- 172.21.15.31:0/1989143742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5623df0c51b0 0x5623df4a9a10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 >> 172.21.15.31:0/1989143742 conn(0x5623df0e2460 msgr2=0x5623df4a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 shutdown_connections 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 1 -- 172.21.15.31:0/1989143742 wait complete. 2024-09-21T06:22:51.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.092+0000 7f63a8502740 20 client.27583 trim_cache size 0 max 16384 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571", 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.098 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.099 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:22:51.100 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:22:51.102 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.103 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.105 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.106 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:22:51.107 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.108 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.109 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:22:51.110 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.111 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.112 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:22:51.113 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:22:51.115 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63840b6850 msgr2=0x7f63840b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63840b6850 0x7f63840b8d60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6394011d20 tx=0x7f63940114d0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c104840 msgr2=0x7f639c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c104840 0x7f639c104160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f638c02a930 tx=0x7f638c031a20 comp rx=0 tx=0).stop 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 shutdown_connections 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63840b6850 0x7f63840b8d60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f639c10ddf0 0x7f639c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f639c105270 0x7f639c102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:22:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 --2- 172.21.15.31:0/2192342592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f639c104840 0x7f639c104160 secure :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f638c02a930 tx=0x7f638c031a20 comp rx=0 tx=0).stop 2024-09-21T06:22:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 >> 172.21.15.31:0/2192342592 conn(0x7f639c079730 msgr2=0x7f639c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:22:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 shutdown_connections 2024-09-21T06:22:51.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:22:51.093+0000 7f63a1f8e640 1 -- 172.21.15.31:0/2192342592 wait complete. 2024-09-21T06:22:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:51 smithi031 ceph-mon[31237]: pgmap v668: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 602 op/s 2024-09-21T06:22:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:51 smithi110 ceph-mon[34550]: pgmap v668: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 602 op/s 2024-09-21T06:22:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:51 smithi107 ceph-mon[34380]: pgmap v668: 97 pgs: 97 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 602 op/s 2024-09-21T06:22:52.023 INFO:tasks.cephfs.filesystem:scrub status for tag:8d24bb6c-bdb0-46a8-8b56-d1650bd8f571 - {'path': '/', 'tag': '8d24bb6c-bdb0-46a8-8b56-d1650bd8f571', 'options': 'recursive,force'} 2024-09-21T06:22:52.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:52 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:22:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:52 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:22:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:52 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:22:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:53 smithi031 ceph-mon[31237]: pgmap v669: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 7.9 MiB/s wr, 824 op/s 2024-09-21T06:22:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:53 smithi110 ceph-mon[34550]: pgmap v669: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 7.9 MiB/s wr, 824 op/s 2024-09-21T06:22:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:53 smithi107 ceph-mon[34380]: pgmap v669: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 7.9 MiB/s wr, 824 op/s 2024-09-21T06:22:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 675.0 s, 2166.4 tps, lat 46.791 ms stddev 58.711 2024-09-21T06:22:55.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: pgmap v670: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 6.4 MiB/s wr, 648 op/s 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:55 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: pgmap v670: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 6.4 MiB/s wr, 648 op/s 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:55 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: pgmap v670: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 6.4 MiB/s wr, 648 op/s 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:55.790 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:55 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:56.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:22:56.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:22:56.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:22:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:22:56.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:22:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:22:56.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:22:57.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:57 smithi031 ceph-mon[31237]: pgmap v671: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 8.6 MiB/s wr, 856 op/s 2024-09-21T06:22:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:57 smithi110 ceph-mon[34550]: pgmap v671: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 8.6 MiB/s wr, 856 op/s 2024-09-21T06:22:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:57 smithi107 ceph-mon[34380]: pgmap v671: 97 pgs: 97 active+clean; 4.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 8.6 MiB/s wr, 856 op/s 2024-09-21T06:22:59.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:22:59 smithi031 ceph-mon[31237]: pgmap v672: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-09-21T06:22:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:22:59 smithi110 ceph-mon[34550]: pgmap v672: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-09-21T06:22:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:22:59 smithi107 ceph-mon[34380]: pgmap v672: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 6.9 MiB/s wr, 757 op/s 2024-09-21T06:23:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 680.0 s, 1717.8 tps, lat 58.206 ms stddev 67.329 2024-09-21T06:23:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:01 smithi110 ceph-mon[34550]: pgmap v673: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-09-21T06:23:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:01 smithi107 ceph-mon[34380]: pgmap v673: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-09-21T06:23:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:01 smithi031 ceph-mon[31237]: pgmap v673: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-09-21T06:23:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 -- 172.21.15.31:0/3923506333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 msgr2=0x7f3538109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 --2- 172.21.15.31:0/3923506333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f3538109840 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3528009900 tx=0x7f352802a410 comp rx=0 tx=0).stop 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 -- 172.21.15.31:0/3923506333 shutdown_connections 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 --2- 172.21.15.31:0/3923506333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3538109d80 0x7f35381105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 --2- 172.21.15.31:0/3923506333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f3538109840 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 --2- 172.21.15.31:0/3923506333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 0x7f3538103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 -- 172.21.15.31:0/3923506333 >> 172.21.15.31:0/3923506333 conn(0x7f35380ff500 msgr2=0x7f3538101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 -- 172.21.15.31:0/3923506333 shutdown_connections 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.630+0000 7f353f758640 1 -- 172.21.15.31:0/3923506333 wait complete. 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 Processor -- start 2024-09-21T06:23:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 -- start start 2024-09-21T06:23:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 0x7f353819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f353819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3538109d80 0x7f353819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3538113a40 con 0x7f3538109430 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35381138c0 con 0x7f3538103b00 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353f758640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3538113bc0 con 0x7f3538109d80 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f353819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353d4f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 0x7f353819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f353819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36388/0 (socket says 172.21.15.31:36388) 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353ccf3640 1 -- 172.21.15.31:0/2796547317 learned_addr learned my addr 172.21.15.31:0/2796547317 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.631+0000 7f353dcf5640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3538109d80 0x7f353819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353d4f4640 1 -- 172.21.15.31:0/2796547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3538109d80 msgr2=0x7f353819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353d4f4640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3538109d80 0x7f353819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353d4f4640 1 -- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 msgr2=0x7f353819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353d4f4640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f353819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353d4f4640 1 -- 172.21.15.31:0/2796547317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35381a03b0 con 0x7f3538103b00 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353ccf3640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f353819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353d4f4640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 0x7f353819a480 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f353400e9f0 tx=0x7f353400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f352e7fc640 1 -- 172.21.15.31:0/2796547317 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353400ccc0 con 0x7f3538103b00 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f352e7fc640 1 -- 172.21.15.31:0/2796547317 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35340108e0 con 0x7f3538103b00 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35381a06a0 con 0x7f3538103b00 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f352e7fc640 1 -- 172.21.15.31:0/2796547317 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534018880 con 0x7f3538103b00 2024-09-21T06:23:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.632+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35381a6e30 con 0x7f3538103b00 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 Processor -- start 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 -- start start 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f352e7fc640 1 -- 172.21.15.31:0/2796547317 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3534010430 con 0x7f3538103b00 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a663a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a663aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a663b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c6a65af6f0 con 0x55c6a62951b0 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6a659eb20 con 0x55c6a62744d0 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f35466bd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6a65a2930 con 0x55c6a626c2f0 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a663a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.642 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a663a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36390/0 (socket says 172.21.15.31:36390) 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 -- 172.21.15.31:0/3359378604 learned_addr learned my addr 172.21.15.31:0/3359378604 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353d4f4640 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a663aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353ccf3640 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a663b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 -- 172.21.15.31:0/3359378604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 msgr2=0x55c6a663aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a663aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 -- 172.21.15.31:0/3359378604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 msgr2=0x55c6a663b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a663b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.635+0000 7f353dcf5640 1 -- 172.21.15.31:0/3359378604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a6564530 con 0x55c6a62951b0 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f353dcf5640 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a663a9b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3530009fd0 tx=0x7f35300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2832ed81af93d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f35237fe640 1 -- 172.21.15.31:0/3359378604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530004d20 con 0x55c6a62951b0 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f352e7fc640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f35240b6680 0x7f35240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f35237fe640 1 -- 172.21.15.31:0/3359378604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3530004050 con 0x55c6a62951b0 2024-09-21T06:23:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f35237fe640 1 -- 172.21.15.31:0/3359378604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530005280 con 0x55c6a62951b0 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f353d4f4640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f35240b6680 0x7f35240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.636+0000 7f352e7fc640 1 -- 172.21.15.31:0/2796547317 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f35340e7a40 con 0x7f3538103b00 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 -- 172.21.15.31:0/3359378604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 msgr2=0x55c6a663a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a663a9b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3530009fd0 tx=0x7f35300118d0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f353d4f4640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f35240b6680 0x7f35240b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f35340e5250 tx=0x7f35340eb590 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 -- 172.21.15.31:0/3359378604 shutdown_connections 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a663b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a663aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 --2- 172.21.15.31:0/3359378604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a663a9b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 -- 172.21.15.31:0/3359378604 >> 172.21.15.31:0/3359378604 conn(0x55c6a6289300 msgr2=0x55c6a66388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 -- 172.21.15.31:0/3359378604 shutdown_connections 2024-09-21T06:23:02.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 -- 172.21.15.31:0/3359378604 wait complete. 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.637+0000 7f35466bd740 1 -- start start 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 Processor -- start 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a664a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a664a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a664aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c6a64d6ab0 con 0x55c6a62951b0 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c6a6544730 con 0x55c6a62744d0 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c6a664d510 con 0x55c6a626c2f0 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f35466bd740 10 client.0 _ll_register_callbacks cb 0x55c6a65986b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a664a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a664a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a664a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56630/0 (socket says 172.21.15.31:56630) 2024-09-21T06:23:02.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a664a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33586/0 (socket says 172.21.15.31:33586) 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f353ccf3640 1 -- 172.21.15.31:0/3018671308 learned_addr learned my addr 172.21.15.31:0/3018671308 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.638+0000 7f353dcf5640 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a664aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f353ccf3640 1 -- 172.21.15.31:0/3018671308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 msgr2=0x55c6a664a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f353ccf3640 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a664a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f353ccf3640 1 -- 172.21.15.31:0/3018671308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 msgr2=0x55c6a664aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f353ccf3640 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a664aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f353ccf3640 1 -- 172.21.15.31:0/3018671308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a6300790 con 0x55c6a62744d0 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f353ccf3640 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a664a930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3528006010 tx=0x7f3528031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f603ded29827eed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f3520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f3520ff9640 1 -- 172.21.15.31:0/3018671308 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528035040 con 0x55c6a62744d0 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f3520ff9640 1 -- 172.21.15.31:0/3018671308 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f3528004450 con 0x55c6a62744d0 2024-09-21T06:23:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3530004050 con 0x55c6a62744d0 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f3520ff9640 1 -- 172.21.15.31:0/3018671308 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3528005020 con 0x55c6a62744d0 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f35466bd740 10 client.27533 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c6a6379620 con 0x55c6a62744d0 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.639+0000 7f3520ff9640 1 -- 172.21.15.31:0/3018671308 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3530004050 con 0x55c6a62744d0 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f3520ff9640 1 -- 172.21.15.31:0/3018671308 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f35280035f0 con 0x55c6a62744d0 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f35466bd740 10 client.27533 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f35466bd740 10 client.27533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f35466bd740 20 client.27533 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f35466bd740 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a6253d80 0x55c6a665c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f35466bd740 4 client.27533 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a65b1640 con 0x55c6a6253d80 2024-09-21T06:23:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f353dcf5640 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a6253d80 0x55c6a665c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f353dcf5640 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a6253d80 0x55c6a665c590 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa1ecfccf17cb5b server_cookie=5a4780912f7701c3 in_seq=0 out_seq=0 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.640+0000 7f3520ff9640 10 client.27533 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f3520ff9640 1 -- 172.21.15.31:0/3018671308 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6a65b1640 con 0x55c6a6253d80 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f3520ff9640 10 client.27533 handle_command_reply: tid=0 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 client.27533 shutdown 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 msgr2=0x55c6a664a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a664a930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3528006010 tx=0x7f3528031740 comp rx=0 tx=0).stop 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 shutdown_connections 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a6253d80 0x55c6a665c590 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a62951b0 0x55c6a664aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f3520ff9640 0 client.27533 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a62744d0 0x55c6a664a930 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 --2- 172.21.15.31:0/3018671308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a664a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 >> 172.21.15.31:0/3018671308 conn(0x55c6a6289300 msgr2=0x55c6a663c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 shutdown_connections 2024-09-21T06:23:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 1 -- 172.21.15.31:0/3018671308 wait complete. 2024-09-21T06:23:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.641+0000 7f35466bd740 20 client.27533 trim_cache size 0 max 16384 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 Processor -- start 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 -- start start 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a6656740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 0x55c6a66475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a6647b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ab560 con 0x55c6a6253d80 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35381addc0 con 0x55c6a6289300 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.656+0000 7f35466bd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35381abbb0 con 0x55c6a626c2f0 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 0x55c6a66475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a6656740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 0x55c6a66475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36406/0 (socket says 172.21.15.31:36406) 2024-09-21T06:23:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a6656740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56636/0 (socket says 172.21.15.31:56636) 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 -- 172.21.15.31:0/2036163580 learned_addr learned my addr 172.21.15.31:0/2036163580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353dcf5640 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a6647b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 -- 172.21.15.31:0/2036163580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 msgr2=0x55c6a6647b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a6647b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 -- 172.21.15.31:0/2036163580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 msgr2=0x55c6a6656740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a6656740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.657+0000 7f353ccf3640 1 -- 172.21.15.31:0/2036163580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a663d810 con 0x55c6a6253d80 2024-09-21T06:23:02.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.659+0000 7f353ccf3640 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 0x55c6a66475c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35280314e0 tx=0x7f352802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e782250a87ec43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.659+0000 7f3522ffd640 1 -- 172.21.15.31:0/2036163580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528035030 con 0x55c6a6253d80 2024-09-21T06:23:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.659+0000 7f3522ffd640 1 -- 172.21.15.31:0/2036163580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528003c00 con 0x55c6a6253d80 2024-09-21T06:23:02.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.659+0000 7f3522ffd640 1 -- 172.21.15.31:0/2036163580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528054040 con 0x55c6a6253d80 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.660+0000 7f35466bd740 1 -- 172.21.15.31:0/2036163580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 msgr2=0x55c6a66475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.660+0000 7f35466bd740 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 0x55c6a66475c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f35280314e0 tx=0x7f352802a410 comp rx=0 tx=0).stop 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.660+0000 7f35466bd740 1 -- 172.21.15.31:0/2036163580 shutdown_connections 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.660+0000 7f35466bd740 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a626c2f0 0x55c6a6647b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.660+0000 7f35466bd740 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a6253d80 0x55c6a66475c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.660+0000 7f35466bd740 1 --2- 172.21.15.31:0/2036163580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a6656740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.661+0000 7f35466bd740 1 -- 172.21.15.31:0/2036163580 >> 172.21.15.31:0/2036163580 conn(0x55c6a665c900 msgr2=0x55c6a6648c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.661+0000 7f35466bd740 1 -- 172.21.15.31:0/2036163580 shutdown_connections 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.661+0000 7f35466bd740 1 -- 172.21.15.31:0/2036163580 wait complete. 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.661+0000 7f35466bd740 1 -- start start 2024-09-21T06:23:02.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.661+0000 7f35466bd740 1 Processor -- start 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a6253d80 0x55c6a664e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 0x55c6a664ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a664f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ac280 con 0x55c6a626c2f0 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ad6f0 con 0x55c6a6289300 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35381ac950 con 0x55c6a6253d80 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f35466bd740 10 client.0 _ll_register_callbacks cb 0x7f3528009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 0x55c6a664ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353dcf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a664f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 0x55c6a664ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36412/0 (socket says 172.21.15.31:36412) 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 -- 172.21.15.31:0/230305640 learned_addr learned my addr 172.21.15.31:0/230305640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353dcf5640 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a6253d80 0x55c6a664e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 -- 172.21.15.31:0/230305640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a6253d80 msgr2=0x55c6a664e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a6253d80 0x55c6a664e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 -- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 msgr2=0x55c6a664f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a664f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 -- 172.21.15.31:0/230305640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a66c5f70 con 0x55c6a626c2f0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f353ccf3640 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 0x55c6a664ebd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3528036040 tx=0x7f3528031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaedb86d0e1f775 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.662+0000 7f3520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f3520ff9640 1 -- 172.21.15.31:0/230305640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004450 con 0x55c6a626c2f0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f3520ff9640 1 -- 172.21.15.31:0/230305640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8499+0+0 (secure 0 0 0) 0x7f35280051c0 con 0x55c6a626c2f0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528003c00 con 0x55c6a626c2f0 2024-09-21T06:23:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f3520ff9640 1 -- 172.21.15.31:0/230305640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f352806d050 con 0x55c6a626c2f0 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 10 client.17757 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c6a66c7120 con 0x55c6a626c2f0 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f3520ff9640 1 -- 172.21.15.31:0/230305640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528003c00 con 0x55c6a626c2f0 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f3520ff9640 1 -- 172.21.15.31:0/230305640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f3528068030 con 0x55c6a626c2f0 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 10 client.17757 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 20 client.17757 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a62744d0 0x55c6a66c9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 4 client.17757 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.663+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6a65ad0f0 con 0x55c6a62744d0 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.664+0000 7f353dcf5640 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a62744d0 0x55c6a66c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.664+0000 7f3520ff9640 10 client.17757 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.664+0000 7f353dcf5640 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a62744d0 0x55c6a66c9890 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fe7e500434bc64 server_cookie=cea6c393124c25f4 in_seq=0 out_seq=0 2024-09-21T06:23:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.684+0000 7f3520ff9640 1 -- 172.21.15.31:0/230305640 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6a65ad0f0 con 0x55c6a62744d0 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.684+0000 7f3520ff9640 10 client.17757 handle_command_reply: tid=0 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.684+0000 7f35466bd740 1 client.17757 shutdown 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.684+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 msgr2=0x55c6a664ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.684+0000 7f35466bd740 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 0x55c6a664ebd0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3528036040 tx=0x7f3528031720 comp rx=0 tx=0).stop 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 shutdown_connections 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c6a62744d0 0x55c6a66c9890 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c6a6289300 0x55c6a664f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6a626c2f0 0x55c6a664ebd0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 --2- 172.21.15.31:0/230305640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6a6253d80 0x55c6a664e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f3520ff9640 0 client.17757 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 >> 172.21.15.31:0/230305640 conn(0x55c6a665c900 msgr2=0x55c6a663b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 shutdown_connections 2024-09-21T06:23:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 1 -- 172.21.15.31:0/230305640 wait complete. 2024-09-21T06:23:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f35466bd740 20 client.17757 trim_cache size 0 max 16384 2024-09-21T06:23:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:23:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:23:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:23:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f35240b6680 msgr2=0x7f35240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f353f758640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f35240b6680 0x7f35240b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f35340e5250 tx=0x7f35340eb590 comp rx=0 tx=0).stop 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 msgr2=0x7f353819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.685+0000 7f353f758640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 0x7f353819a480 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f353400e9f0 tx=0x7f353400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 shutdown_connections 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3538109d80 0x7f353819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f35240b6680 0x7f35240b8b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3538109430 0x7f353819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 --2- 172.21.15.31:0/2796547317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3538103b00 0x7f353819a480 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 >> 172.21.15.31:0/2796547317 conn(0x7f35380ff500 msgr2=0x7f3538108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 shutdown_connections 2024-09-21T06:23:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:02.686+0000 7f353f758640 1 -- 172.21.15.31:0/2796547317 wait complete. 2024-09-21T06:23:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:03 smithi110 ceph-mon[34550]: pgmap v674: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 7.8 MiB/s wr, 843 op/s 2024-09-21T06:23:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:03 smithi107 ceph-mon[34380]: pgmap v674: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 7.8 MiB/s wr, 843 op/s 2024-09-21T06:23:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:03 smithi031 ceph-mon[31237]: pgmap v674: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 7.8 MiB/s wr, 843 op/s 2024-09-21T06:23:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 685.0 s, 409.4 tps, lat 227.881 ms stddev 318.733 2024-09-21T06:23:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:05 smithi110 ceph-mon[34550]: pgmap v675: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-09-21T06:23:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:05 smithi107 ceph-mon[34380]: pgmap v675: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-09-21T06:23:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:05 smithi031 ceph-mon[31237]: pgmap v675: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-09-21T06:23:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:07 smithi110 ceph-mon[34550]: pgmap v676: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.1 MiB/s wr, 638 op/s 2024-09-21T06:23:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:07 smithi107 ceph-mon[34380]: pgmap v676: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.1 MiB/s wr, 638 op/s 2024-09-21T06:23:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:07 smithi031 ceph-mon[31237]: pgmap v676: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.1 MiB/s wr, 638 op/s 2024-09-21T06:23:08.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:08 smithi110 ceph-mon[34550]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:23:08.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:08 smithi107 ceph-mon[34380]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:23:08.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:08 smithi031 ceph-mon[31237]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:23:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:09 smithi110 ceph-mon[34550]: pgmap v678: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2024-09-21T06:23:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:09 smithi107 ceph-mon[34380]: pgmap v678: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2024-09-21T06:23:09.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:09 smithi031 ceph-mon[31237]: pgmap v678: 97 pgs: 97 active+clean; 4.8 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.1 MiB/s wr, 472 op/s 2024-09-21T06:23:10.206 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 690.0 s, 80.4 tps, lat 552.906 ms stddev 341.806 2024-09-21T06:23:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:11 smithi110 ceph-mon[34550]: pgmap v679: 97 pgs: 8 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 84 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-09-21T06:23:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:11 smithi107 ceph-mon[34380]: pgmap v679: 97 pgs: 8 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 84 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-09-21T06:23:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:11 smithi031 ceph-mon[31237]: pgmap v679: 97 pgs: 8 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 84 active+clean; 4.7 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-09-21T06:23:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:13 smithi110 ceph-mon[34550]: pgmap v680: 97 pgs: 22 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 61 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-21T06:23:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:13 smithi107 ceph-mon[34380]: pgmap v680: 97 pgs: 22 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 61 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-21T06:23:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:13 smithi031 ceph-mon[31237]: pgmap v680: 97 pgs: 22 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 61 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-21T06:23:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 695.0 s, 109.4 tps, lat 1408.467 ms stddev 1898.838 2024-09-21T06:23:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:15 smithi110 ceph-mon[34550]: pgmap v681: 97 pgs: 22 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 61 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-21T06:23:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:15 smithi107 ceph-mon[34380]: pgmap v681: 97 pgs: 22 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 61 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 -- 172.21.15.31:0/3601240921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 msgr2=0x7f69b0104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 --2- 172.21.15.31:0/3601240921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b0104d70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009860 tx=0x7f69a002a340 comp rx=0 tx=0).stop 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 -- 172.21.15.31:0/3601240921 shutdown_connections 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 --2- 172.21.15.31:0/3601240921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69b010dd80 0x7f69b0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 --2- 172.21.15.31:0/3601240921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69b01052b0 0x7f69b010d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 --2- 172.21.15.31:0/3601240921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b0104d70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 -- 172.21.15.31:0/3601240921 >> 172.21.15.31:0/3601240921 conn(0x7f69b007a740 msgr2=0x7f69b007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:15.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 -- 172.21.15.31:0/3601240921 shutdown_connections 2024-09-21T06:23:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.793+0000 7f69b54de640 1 -- 172.21.15.31:0/3601240921 wait complete. 2024-09-21T06:23:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 Processor -- start 2024-09-21T06:23:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 -- start start 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69b01052b0 0x7f69b019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69b010dd80 0x7f69b019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01139d0 con 0x7f69b01052b0 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69b0113850 con 0x7f69b010dd80 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69b54de640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69b0113b50 con 0x7f69b0104960 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69b01052b0 0x7f69b019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34416/0 (socket says 172.21.15.31:34416) 2024-09-21T06:23:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69b010dd80 0x7f69b019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.794+0000 7f69aeffd640 1 -- 172.21.15.31:0/2319418427 learned_addr learned my addr 172.21.15.31:0/2319418427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69aeffd640 1 -- 172.21.15.31:0/2319418427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69b010dd80 msgr2=0x7f69b019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69aeffd640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69b010dd80 0x7f69b019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69aeffd640 1 -- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69b01052b0 msgr2=0x7f69b019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69aeffd640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69b01052b0 0x7f69b019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69aeffd640 1 -- 172.21.15.31:0/2319418427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b01a0340 con 0x7f69b0104960 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69aeffd640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b019a410 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f69a00303f0 tx=0x7f69a0032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f6997fff640 1 -- 172.21.15.31:0/2319418427 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0040020 con 0x7f69b0104960 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f6997fff640 1 -- 172.21.15.31:0/2319418427 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a0030dd0 con 0x7f69b0104960 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b01a05d0 con 0x7f69b0104960 2024-09-21T06:23:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f6997fff640 1 -- 172.21.15.31:0/2319418427 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0037040 con 0x7f69b0104960 2024-09-21T06:23:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.795+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b01a6dc0 con 0x7f69b0104960 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 Processor -- start 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 -- start start 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f6997fff640 1 -- 172.21.15.31:0/2319418427 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f69a0043020 con 0x7f69b0104960 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd20e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd20f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5631dd1836f0 con 0x5631dce484d0 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631dd172b20 con 0x5631dce691b0 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69bc44d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5631dd176930 con 0x5631dce402f0 2024-09-21T06:23:15.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd20f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34430/0 (socket says 172.21.15.31:34430) 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69aeffd640 1 -- 172.21.15.31:0/4110801454 learned_addr learned my addr 172.21.15.31:0/4110801454 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.798+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd20f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36970/0 (socket says 172.21.15.31:36970) 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69af7fe640 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd20e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69aeffd640 1 -- 172.21.15.31:0/4110801454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 msgr2=0x5631dd20e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69aeffd640 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd20e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69aeffd640 1 -- 172.21.15.31:0/4110801454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 msgr2=0x5631dd20f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69aeffd640 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd20f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69aeffd640 1 -- 172.21.15.31:0/4110801454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dd138530 con 0x5631dce402f0 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f6997fff640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69980b6680 0x7f69980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69aeffd640 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20eef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f69a0104cb0 tx=0x7f69a010db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20bddced9ef00a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69af7fe640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69980b6680 0x7f69980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69957fa640 1 -- 172.21.15.31:0/4110801454 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a00c9ac0 con 0x5631dce402f0 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f6997fff640 1 -- 172.21.15.31:0/2319418427 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8543+0+0 (secure 0 0 0) 0x7f69a0107630 con 0x7f69b0104960 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69957fa640 1 -- 172.21.15.31:0/4110801454 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a00ccf80 con 0x5631dce402f0 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.799+0000 7f69957fa640 1 -- 172.21.15.31:0/4110801454 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a004de30 con 0x5631dce402f0 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 -- 172.21.15.31:0/4110801454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 msgr2=0x5631dd20eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69af7fe640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69980b6680 0x7f69980b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005fd0 tx=0x7f69a8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20eef0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f69a0104cb0 tx=0x7f69a010db40 comp rx=0 tx=0).stop 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 -- 172.21.15.31:0/4110801454 shutdown_connections 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd20f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20eef0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4110801454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd20e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 -- 172.21.15.31:0/4110801454 >> 172.21.15.31:0/4110801454 conn(0x5631dce5d300 msgr2=0x5631dd20c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 -- 172.21.15.31:0/4110801454 shutdown_connections 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 -- 172.21.15.31:0/4110801454 wait complete. 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 -- start start 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.801+0000 7f69bc44d740 1 Processor -- start 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd21a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd21a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd226150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5631dd0aaab0 con 0x5631dce484d0 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5631dd118730 con 0x5631dce691b0 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5631dd21cef0 con 0x5631dce402f0 2024-09-21T06:23:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69bc44d740 10 client.0 _ll_register_callbacks cb 0x5631dd16c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd21a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd21a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36978/0 (socket says 172.21.15.31:36978) 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 -- 172.21.15.31:0/1708111145 learned_addr learned my addr 172.21.15.31:0/1708111145 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69aeffd640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd21a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69af7fe640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd226150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 -- 172.21.15.31:0/1708111145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 msgr2=0x5631dd21a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd21a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 -- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 msgr2=0x5631dd226150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd226150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69ae7fc640 1 -- 172.21.15.31:0/1708111145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dced4790 con 0x5631dce484d0 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.802+0000 7f69af7fe640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd226150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd21a5c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f69a40116a0 tx=0x7f69a4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26aa246bbdd23488 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f6992ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f6992ff5640 1 -- 172.21.15.31:0/1708111145 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a401e060 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f6992ff5640 1 -- 172.21.15.31:0/1708111145 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8543+0+0 (secure 0 0 0) 0x7f69a400fbf0 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a00ccf80 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f6992ff5640 1 -- 172.21.15.31:0/1708111145 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69a400fe30 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f69bc44d740 10 client.17769 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5631dcf4d620 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.803+0000 7f6992ff5640 1 -- 172.21.15.31:0/1708111145 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a00ccf80 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f6992ff5640 1 -- 172.21.15.31:0/1708111145 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f69a4019070 con 0x5631dce484d0 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69bc44d740 10 client.17769 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69bc44d740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69bc44d740 20 client.17769 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69bc44d740 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dce27d80 0x5631dd22bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69bc44d740 4 client.17769 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631dd185640 con 0x5631dce27d80 2024-09-21T06:23:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f69aeffd640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dce27d80 0x5631dd22bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.804+0000 7f6992ff5640 10 client.17769 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69aeffd640 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dce27d80 0x5631dd22bf70 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d39fc827efb1b09 server_cookie=e49387fe11400605 in_seq=0 out_seq=0 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f6992ff5640 1 -- 172.21.15.31:0/1708111145 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631dd185640 con 0x5631dce27d80 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f6992ff5640 10 client.17769 handle_command_reply: tid=0 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 client.17769 shutdown 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 msgr2=0x5631dd21a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd21a5c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f69a40116a0 tx=0x7f69a4011b60 comp rx=0 tx=0).stop 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 shutdown_connections 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dce27d80 0x5631dd22bf70 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce691b0 0x5631dd226150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce484d0 0x5631dd21a5c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 --2- 172.21.15.31:0/1708111145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd21a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 >> 172.21.15.31:0/1708111145 conn(0x5631dce5d300 msgr2=0x5631dd20f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.805+0000 7f6992ff5640 0 client.17769 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.806+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 shutdown_connections 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.806+0000 7f69bc44d740 1 -- 172.21.15.31:0/1708111145 wait complete. 2024-09-21T06:23:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.806+0000 7f69bc44d740 20 client.17769 trim_cache size 0 max 16384 2024-09-21T06:23:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 Processor -- start 2024-09-21T06:23:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 -- start start 2024-09-21T06:23:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd217250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dcdfd1c0 0x5631dd217790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ad720 con 0x5631dce691b0 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01aff80 con 0x5631dcdfd1c0 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.824+0000 7f69bc44d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01add70 con 0x5631dce402f0 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd217250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd217250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36982/0 (socket says 172.21.15.31:36982) 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dcdfd1c0 0x5631dd217790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 -- 172.21.15.31:0/4270514162 learned_addr learned my addr 172.21.15.31:0/4270514162 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69aeffd640 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 -- 172.21.15.31:0/4270514162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 msgr2=0x5631dd20c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 -- 172.21.15.31:0/4270514162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dcdfd1c0 msgr2=0x5631dd217790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dcdfd1c0 0x5631dd217790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 -- 172.21.15.31:0/4270514162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dd21b340 con 0x5631dce691b0 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69aeffd640 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f69ae7fc640 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd217250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f69a4013040 tx=0x7f69a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f323c6a19994f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f6994ff9640 1 -- 172.21.15.31:0/4270514162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4028080 con 0x5631dce691b0 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f6994ff9640 1 -- 172.21.15.31:0/4270514162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a401a510 con 0x5631dce691b0 2024-09-21T06:23:15.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.825+0000 7f6994ff9640 1 -- 172.21.15.31:0/4270514162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a40035f0 con 0x5631dce691b0 2024-09-21T06:23:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 -- 172.21.15.31:0/4270514162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 msgr2=0x5631dd217250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd217250 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f69a4013040 tx=0x7f69a4009da0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 -- 172.21.15.31:0/4270514162 shutdown_connections 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dcdfd1c0 0x5631dd217790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd217250 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 --2- 172.21.15.31:0/4270514162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dce402f0 0x5631dd20c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 -- 172.21.15.31:0/4270514162 >> 172.21.15.31:0/4270514162 conn(0x5631dce5d300 msgr2=0x5631dd20c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 -- 172.21.15.31:0/4270514162 shutdown_connections 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.826+0000 7f69bc44d740 1 -- 172.21.15.31:0/4270514162 wait complete. 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 -- start start 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 Processor -- start 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dcdfd1c0 0x5631dd21d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce402f0 0x5631dd21d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd2948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01ae440 con 0x5631dce691b0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01af8b0 con 0x5631dce402f0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01aeb10 con 0x5631dcdfd1c0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69bc44d740 10 client.0 _ll_register_callbacks cb 0x7f69a4009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd2948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dcdfd1c0 0x5631dd21d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd2948f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36986/0 (socket says 172.21.15.31:36986) 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69ae7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce402f0 0x5631dd21d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 -- 172.21.15.31:0/10316038 learned_addr learned my addr 172.21.15.31:0/10316038 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69aeffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dcdfd1c0 0x5631dd21d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34462/0 (socket says 172.21.15.31:34462) 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 -- 172.21.15.31:0/10316038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dcdfd1c0 msgr2=0x5631dd21d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dcdfd1c0 0x5631dd21d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 -- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce402f0 msgr2=0x5631dd21d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.827+0000 7f69af7fe640 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce402f0 0x5631dd21d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.828+0000 7f69af7fe640 1 -- 172.21.15.31:0/10316038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dd294e30 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.828+0000 7f69af7fe640 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd2948f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f69a8004610 tx=0x7f69a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcab28341f0f4b65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.828+0000 7f6992ff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.828+0000 7f6992ff5640 1 -- 172.21.15.31:0/10316038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8004850 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.829+0000 7f6992ff5640 1 -- 172.21.15.31:0/10316038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8543+0+0 (secure 0 0 0) 0x7f69a8003bb0 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.829+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a401a510 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.829+0000 7f6992ff5640 1 -- 172.21.15.31:0/10316038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f69a8003df0 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.829+0000 7f69bc44d740 10 client.17775 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.829+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5631dd295f10 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.829+0000 7f6992ff5640 1 -- 172.21.15.31:0/10316038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a401a510 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f6992ff5640 1 -- 172.21.15.31:0/10316038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f69a8028070 con 0x5631dce691b0 2024-09-21T06:23:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69bc44d740 10 client.17775 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69bc44d740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69bc44d740 20 client.17775 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69bc44d740 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dd2966f0 0x5631dd298b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69bc44d740 4 client.17775 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631dd1810f0 con 0x5631dd2966f0 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f69aeffd640 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dd2966f0 0x5631dd298b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.830+0000 7f6992ff5640 10 client.17775 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.831+0000 7f69aeffd640 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dd2966f0 0x5631dd298b10 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35dd6f5d61ffc414 server_cookie=97a7ca8cb3291add in_seq=0 out_seq=0 2024-09-21T06:23:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f6992ff5640 1 -- 172.21.15.31:0/10316038 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631dd1810f0 con 0x5631dd2966f0 2024-09-21T06:23:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f6992ff5640 10 client.17775 handle_command_reply: tid=0 2024-09-21T06:23:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 client.17775 shutdown 2024-09-21T06:23:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 msgr2=0x5631dd2948f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd2948f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f69a8004610 tx=0x7f69a8004ab0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 shutdown_connections 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5631dd2966f0 0x5631dd298b10 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5631dce691b0 0x5631dd2948f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f6992ff5640 0 client.17775 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5631dce402f0 0x5631dd21d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 --2- 172.21.15.31:0/10316038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5631dcdfd1c0 0x5631dd21d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 >> 172.21.15.31:0/10316038 conn(0x5631dce5d300 msgr2=0x5631dd20c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 shutdown_connections 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 1 -- 172.21.15.31:0/10316038 wait complete. 2024-09-21T06:23:15.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.850+0000 7f69bc44d740 20 client.17775 trim_cache size 0 max 16384 2024-09-21T06:23:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:23:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:23:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:23:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69980b6680 msgr2=0x7f69980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69980b6680 0x7f69980b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005fd0 tx=0x7f69a8009500 comp rx=0 tx=0).stop 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 msgr2=0x7f69b019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b019a410 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f69a00303f0 tx=0x7f69a0032da0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 shutdown_connections 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69b010dd80 0x7f69b019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69980b6680 0x7f69980b8b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69b01052b0 0x7f69b019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 --2- 172.21.15.31:0/2319418427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0104960 0x7f69b019a410 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 >> 172.21.15.31:0/2319418427 conn(0x7f69b007a740 msgr2=0x7f69b01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 shutdown_connections 2024-09-21T06:23:15.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:15.851+0000 7f69b54de640 1 -- 172.21.15.31:0/2319418427 wait complete. 2024-09-21T06:23:15.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:15 smithi031 ceph-mon[31237]: pgmap v681: 97 pgs: 22 active+clean+snaptrim_wait, 14 active+clean+snaptrim, 61 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.6 MiB/s wr, 242 op/s 2024-09-21T06:23:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:17 smithi110 ceph-mon[34550]: pgmap v682: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 4.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 883 KiB/s wr, 160 op/s 2024-09-21T06:23:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:17 smithi107 ceph-mon[34380]: pgmap v682: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 4.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 883 KiB/s wr, 160 op/s 2024-09-21T06:23:17.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:17 smithi031 ceph-mon[31237]: pgmap v682: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 4.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 883 KiB/s wr, 160 op/s 2024-09-21T06:23:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:19 smithi110 ceph-mon[34550]: pgmap v683: 97 pgs: 97 active+clean; 4.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-09-21T06:23:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:19 smithi107 ceph-mon[34380]: pgmap v683: 97 pgs: 97 active+clean; 4.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-09-21T06:23:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:19 smithi031 ceph-mon[31237]: pgmap v683: 97 pgs: 97 active+clean; 4.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 302 op/s 2024-09-21T06:23:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 700.0 s, 287.6 tps, lat 375.803 ms stddev 823.846 2024-09-21T06:23:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:21 smithi110 ceph-mon[34550]: pgmap v684: 97 pgs: 97 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 454 op/s 2024-09-21T06:23:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:21 smithi107 ceph-mon[34380]: pgmap v684: 97 pgs: 97 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 454 op/s 2024-09-21T06:23:21.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:21 smithi031 ceph-mon[31237]: pgmap v684: 97 pgs: 97 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.3 MiB/s wr, 454 op/s 2024-09-21T06:23:22.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:23:22.416 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.113+0000 7f55567fc640 1 -- 172.21.15.31:0/1677961516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500307d0 con 0x7f555810ddf0 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.113+0000 7f555dd99640 1 -- 172.21.15.31:0/1677961516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f555810ddf0 msgr2=0x7f5558110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.113+0000 7f555dd99640 1 --2- 172.21.15.31:0/1677961516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f555810ddf0 0x7f5558110300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5550009ca0 tx=0x7f555002a8d0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.113+0000 7f555dd99640 1 -- 172.21.15.31:0/1677961516 shutdown_connections 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.113+0000 7f555dd99640 1 --2- 172.21.15.31:0/1677961516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f555810ddf0 0x7f5558110300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.114+0000 7f555dd99640 1 --2- 172.21.15.31:0/1677961516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5558105270 0x7f555810d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.114+0000 7f555dd99640 1 --2- 172.21.15.31:0/1677961516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5558104840 0x7f5558104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.114+0000 7f555dd99640 1 -- 172.21.15.31:0/1677961516 >> 172.21.15.31:0/1677961516 conn(0x7f5558079730 msgr2=0x7f5558079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:23.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.115+0000 7f555dd99640 1 -- 172.21.15.31:0/1677961516 shutdown_connections 2024-09-21T06:23:23.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.115+0000 7f555dd99640 1 -- 172.21.15.31:0/1677961516 wait complete. 2024-09-21T06:23:23.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.116+0000 7f555dd99640 1 Processor -- start 2024-09-21T06:23:23.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.117+0000 7f555dd99640 1 -- start start 2024-09-21T06:23:23.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f555dd99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5558104840 0x7f555819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f555dd99640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5558105270 0x7f555819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f555dd99640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 0x7f555819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f555dd99640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5558113b60 con 0x7f5558104840 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f5557fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 0x7f555819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f5557fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 0x7f555819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50744/0 (socket says 172.21.15.31:50744) 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f5557fff640 1 -- 172.21.15.31:0/2505366007 learned_addr learned my addr 172.21.15.31:0/2505366007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.118+0000 7f5556ffd640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5558105270 0x7f555819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.119+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55581139e0 con 0x7f5558105270 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.119+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5558113ce0 con 0x7f555810ddf0 2024-09-21T06:23:23.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f55577fe640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5558104840 0x7f555819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f5557fff640 1 -- 172.21.15.31:0/2505366007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5558105270 msgr2=0x7f555819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f5557fff640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5558105270 0x7f555819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f5557fff640 1 -- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5558104840 msgr2=0x7f555819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f5557fff640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5558104840 0x7f555819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f5557fff640 1 -- 172.21.15.31:0/2505366007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55581a0440 con 0x7f555810ddf0 2024-09-21T06:23:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.120+0000 7f5557fff640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 0x7f555819fd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55500097b0 tx=0x7f5550003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:23.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.121+0000 7f5554ff9640 1 -- 172.21.15.31:0/2505366007 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500307d0 con 0x7f555810ddf0 2024-09-21T06:23:23.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.121+0000 7f5554ff9640 1 -- 172.21.15.31:0/2505366007 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5550003e50 con 0x7f555810ddf0 2024-09-21T06:23:23.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.121+0000 7f5554ff9640 1 -- 172.21.15.31:0/2505366007 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500047d0 con 0x7f555810ddf0 2024-09-21T06:23:23.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.121+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55581a06d0 con 0x7f555810ddf0 2024-09-21T06:23:23.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.121+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5558102630 con 0x7f555810ddf0 2024-09-21T06:23:23.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.124+0000 7f556430d740 1 Processor -- start 2024-09-21T06:23:23.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.125+0000 7f5554ff9640 1 -- 172.21.15.31:0/2505366007 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5550030970 con 0x7f555810ddf0 2024-09-21T06:23:23.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.126+0000 7f556430d740 1 -- start start 2024-09-21T06:23:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.126+0000 7f556430d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3185f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.126+0000 7f556430d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3185fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f556430d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f556430d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559a317d5e50 con 0x559a314b8070 2024-09-21T06:23:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f556430d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a317c5280 con 0x559a31497390 2024-09-21T06:23:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f556430d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559a317c9090 con 0x559a3148f1b0 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3185fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3185fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50752/0 (socket says 172.21.15.31:50752) 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 -- 172.21.15.31:0/3043521156 learned_addr learned my addr 172.21.15.31:0/3043521156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f5556ffd640 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3185f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 -- 172.21.15.31:0/3043521156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 msgr2=0x559a318600d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318600d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 -- 172.21.15.31:0/3043521156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 msgr2=0x559a3185f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3185f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 -- 172.21.15.31:0/3043521156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a31523650 con 0x559a3148f1b0 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f5554ff9640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55400b67d0 0x7f55400b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f5556ffd640 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3185f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f55577fe640 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3185fb90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f554800ddc0 tx=0x7f554800d4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40f00b759b7337e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.127+0000 7f5556ffd640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55400b67d0 0x7f55400b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.128+0000 7f5554ff9640 1 -- 172.21.15.31:0/2505366007 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8543+0+0 (secure 0 0 0) 0x7f5550108ab0 con 0x7f555810ddf0 2024-09-21T06:23:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.128+0000 7f553dffb640 1 -- 172.21.15.31:0/3043521156 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554801b070 con 0x559a3148f1b0 2024-09-21T06:23:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.128+0000 7f553dffb640 1 -- 172.21.15.31:0/3043521156 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5548016420 con 0x559a3148f1b0 2024-09-21T06:23:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.128+0000 7f553dffb640 1 -- 172.21.15.31:0/3043521156 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548014580 con 0x559a3148f1b0 2024-09-21T06:23:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 -- 172.21.15.31:0/3043521156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 msgr2=0x559a3185fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f5556ffd640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55400b67d0 0x7f55400b8ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f555810fbf0 tx=0x7f554c005f30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3185fb90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f554800ddc0 tx=0x7f554800d4f0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 -- 172.21.15.31:0/3043521156 shutdown_connections 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3185fb90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 --2- 172.21.15.31:0/3043521156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3185f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 -- 172.21.15.31:0/3043521156 >> 172.21.15.31:0/3043521156 conn(0x559a314ac460 msgr2=0x559a3185d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 -- 172.21.15.31:0/3043521156 shutdown_connections 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.129+0000 7f556430d740 1 -- 172.21.15.31:0/3043521156 wait complete. 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f556430d740 1 -- start start 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f556430d740 1 Processor -- start 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f556430d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3186ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f556430d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a3186b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f556430d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3186b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f556430d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559a31626700 con 0x559a314b8070 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.130+0000 7f55577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3186ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3186ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50768/0 (socket says 172.21.15.31:50768) 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 -- 172.21.15.31:0/232584249 learned_addr learned my addr 172.21.15.31:0/232584249 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a31760110 con 0x559a31497390 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559a3186ded0 con 0x559a3148f1b0 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f556430d740 10 client.0 _ll_register_callbacks cb 0x559a31779990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f5557fff640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3186b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f5556ffd640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a3186b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 -- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 msgr2=0x559a3186b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a3186b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 -- 172.21.15.31:0/232584249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 msgr2=0x559a3186b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3186b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 -- 172.21.15.31:0/232584249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3178dc40 con 0x559a3148f1b0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f5556ffd640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a3186b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f55577fe640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3186ac60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5548016d70 tx=0x7f554800bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31ae72d8213270b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f5536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f5536ffd640 1 -- 172.21.15.31:0/232584249 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548023020 con 0x559a3148f1b0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.131+0000 7f5536ffd640 1 -- 172.21.15.31:0/232584249 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8543+0+0 (secure 0 0 0) 0x7f55480167d0 con 0x559a3148f1b0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.132+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5548016420 con 0x559a3148f1b0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.132+0000 7f5536ffd640 1 -- 172.21.15.31:0/232584249 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f554802e050 con 0x559a3148f1b0 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.133+0000 7f556430d740 10 client.27628 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:23.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.133+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559a31879ac0 con 0x559a3148f1b0 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.133+0000 7f5536ffd640 1 -- 172.21.15.31:0/232584249 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5548016420 con 0x559a3148f1b0 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.133+0000 7f5536ffd640 1 -- 172.21.15.31:0/232584249 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f554801e3a0 con 0x559a3148f1b0 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f556430d740 10 client.27628 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f556430d740 10 client.27628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f556430d740 20 client.27628 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f556430d740 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a314769f0 0x559a3187ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f556430d740 4 client.27628 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:23.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a317d7da0 con 0x559a314769f0 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f5556ffd640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a314769f0 0x559a3187ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f5536ffd640 10 client.27628 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.134+0000 7f5556ffd640 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a314769f0 0x559a3187ce80 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a81f4b249c8805a server_cookie=26e404356c114328 in_seq=0 out_seq=0 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f5536ffd640 1 -- 172.21.15.31:0/232584249 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a317d7da0 con 0x559a314769f0 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f5536ffd640 10 client.27628 handle_command_reply: tid=0 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f556430d740 1 client.27628 shutdown 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 msgr2=0x559a3186ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f556430d740 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3186ac60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5548016d70 tx=0x7f554800bfc0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 shutdown_connections 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f556430d740 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a314769f0 0x559a3187ce80 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.135+0000 7f5536ffd640 0 client.27628 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:23.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a314b8070 0x559a3186b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a3186b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 1 --2- 172.21.15.31:0/232584249 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a3186ac60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 >> 172.21.15.31:0/232584249 conn(0x559a314ac460 msgr2=0x559a31860680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 shutdown_connections 2024-09-21T06:23:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 1 -- 172.21.15.31:0/232584249 wait complete. 2024-09-21T06:23:23.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.136+0000 7f556430d740 20 client.27628 trim_cache size 0 max 16384 2024-09-21T06:23:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 Processor -- start 2024-09-21T06:23:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 -- start start 2024-09-21T06:23:23.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 0x559a3186c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a314b8070 0x559a31867e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b3db0 con 0x559a3148f1b0 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b6610 con 0x559a31497390 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f556430d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b4400 con 0x559a314b8070 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f55577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 0x559a3186c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f5557fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a314b8070 0x559a31867e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f55577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 0x559a3186c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41732/0 (socket says 172.21.15.31:41732) 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f55577fe640 1 -- 172.21.15.31:0/3918428221 learned_addr learned my addr 172.21.15.31:0/3918428221 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f5556ffd640 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.158+0000 7f5557fff640 1 -- 172.21.15.31:0/3918428221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 msgr2=0x559a318770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f5557fff640 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f5557fff640 1 -- 172.21.15.31:0/3918428221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 msgr2=0x559a3186c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f5557fff640 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 0x559a3186c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f5557fff640 1 -- 172.21.15.31:0/3918428221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a318624b0 con 0x559a314b8070 2024-09-21T06:23:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f55577fe640 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 0x559a3186c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f5557fff640 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a314b8070 0x559a31867e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f55501061f0 tx=0x7f555002a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5626d37ac93a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f553d7fa640 1 -- 172.21.15.31:0/3918428221 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500ca060 con 0x559a314b8070 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f553d7fa640 1 -- 172.21.15.31:0/3918428221 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5550036040 con 0x559a314b8070 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.159+0000 7f553d7fa640 1 -- 172.21.15.31:0/3918428221 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500cab60 con 0x559a314b8070 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.160+0000 7f556430d740 1 -- 172.21.15.31:0/3918428221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a314b8070 msgr2=0x559a31867e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.160+0000 7f556430d740 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a314b8070 0x559a31867e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f55501061f0 tx=0x7f555002a8d0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 -- 172.21.15.31:0/3918428221 shutdown_connections 2024-09-21T06:23:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a314b8070 0x559a31867e50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a31497390 0x559a318770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 --2- 172.21.15.31:0/3918428221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a3148f1b0 0x559a3186c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 -- 172.21.15.31:0/3918428221 >> 172.21.15.31:0/3918428221 conn(0x559a314ac460 msgr2=0x559a31861040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 -- 172.21.15.31:0/3918428221 shutdown_connections 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 -- 172.21.15.31:0/3918428221 wait complete. 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 -- start start 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.161+0000 7f556430d740 1 Processor -- start 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a31873ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 0x559a3186efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a314b8070 0x559a3186f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b4ad0 con 0x559a31497390 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b5f40 con 0x559a314b8070 2024-09-21T06:23:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55581b51a0 con 0x559a3148f1b0 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f556430d740 10 client.0 _ll_register_callbacks cb 0x7f5548019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 0x559a3186efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 0x559a3186efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41746/0 (socket says 172.21.15.31:41746) 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f55577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a31873ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 -- 172.21.15.31:0/1806698522 learned_addr learned my addr 172.21.15.31:0/1806698522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5557fff640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a314b8070 0x559a3186f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 -- 172.21.15.31:0/1806698522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 msgr2=0x559a31873ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a31873ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 -- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a314b8070 msgr2=0x559a3186f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a314b8070 0x559a3186f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 -- 172.21.15.31:0/1806698522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a318e6780 con 0x559a31497390 2024-09-21T06:23:23.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5557fff640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a314b8070 0x559a3186f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.162+0000 7f5556ffd640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 0x559a3186efb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f554c00bfd0 tx=0x7f554c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15487788bc1e0016 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f5536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f5536ffd640 1 -- 172.21.15.31:0/1806698522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c01f830 con 0x559a31497390 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5550036040 con 0x559a31497390 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f5536ffd640 1 -- 172.21.15.31:0/1806698522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8543+0+0 (secure 0 0 0) 0x7f554c01f9d0 con 0x559a31497390 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f5536ffd640 1 -- 172.21.15.31:0/1806698522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f554c012e50 con 0x559a31497390 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 10 client.17790 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559a318e7910 con 0x559a31497390 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f5536ffd640 1 -- 172.21.15.31:0/1806698522 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5550036040 con 0x559a31497390 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f5536ffd640 1 -- 172.21.15.31:0/1806698522 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f554c00f040 con 0x559a31497390 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 10 client.17790 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 20 client.17790 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a3144bda0 0x559a318ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 4 client.17790 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.163+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a317d3850 con 0x559a3144bda0 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f55577fe640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a3144bda0 0x559a318ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f5536ffd640 10 client.17790 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f55577fe640 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a3144bda0 0x559a318ea060 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3787e74498d9495 server_cookie=a89ea977ee981ed7 in_seq=0 out_seq=0 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f5536ffd640 1 -- 172.21.15.31:0/1806698522 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7758 (crc 0 0 0) 0x559a317d3850 con 0x559a3144bda0 2024-09-21T06:23:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f5536ffd640 10 client.17790 handle_command_reply: tid=0 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f556430d740 1 client.17790 shutdown 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 msgr2=0x559a3186efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f556430d740 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 0x559a3186efb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f554c00bfd0 tx=0x7f554c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 shutdown_connections 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.164+0000 7f556430d740 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559a3144bda0 0x559a318ea060 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a314b8070 0x559a3186f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a31497390 0x559a3186efb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:23:23.170 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:23:23.171 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.173 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.174 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.175 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.176 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.180 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.181 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.182 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.183 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.184 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:23:23.185 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:23.186 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 1 --2- 172.21.15.31:0/1806698522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a3148f1b0 0x559a31873ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 >> 172.21.15.31:0/1806698522 conn(0x559a314ac460 msgr2=0x559a3185c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f5536ffd640 0 client.17790 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 shutdown_connections 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 1 -- 172.21.15.31:0/1806698522 wait complete. 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f556430d740 20 client.17790 trim_cache size 0 max 16384 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55400b67d0 msgr2=0x7f55400b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55400b67d0 0x7f55400b8ce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f555810fbf0 tx=0x7f554c005f30 comp rx=0 tx=0).stop 2024-09-21T06:23:23.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 msgr2=0x7f555819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 0x7f555819fd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55500097b0 tx=0x7f5550003ba0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 shutdown_connections 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f555810ddf0 0x7f555819fd10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5558105270 0x7f555819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.165+0000 7f555dd99640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f55400b67d0 0x7f55400b8ce0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.166+0000 7f555dd99640 1 --2- 172.21.15.31:0/2505366007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5558104840 0x7f555819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.166+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 >> 172.21.15.31:0/2505366007 conn(0x7f5558079730 msgr2=0x7f5558101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.166+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 shutdown_connections 2024-09-21T06:23:23.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:23.166+0000 7f555dd99640 1 -- 172.21.15.31:0/2505366007 wait complete. 2024-09-21T06:23:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:23 smithi110 ceph-mon[34550]: pgmap v685: 97 pgs: 97 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 920 op/s 2024-09-21T06:23:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:23 smithi110 ceph-mon[34550]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:23:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:23 smithi107 ceph-mon[34380]: pgmap v685: 97 pgs: 97 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 920 op/s 2024-09-21T06:23:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:23 smithi107 ceph-mon[34380]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:23:23.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:23 smithi031 ceph-mon[31237]: pgmap v685: 97 pgs: 97 active+clean; 4.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 920 op/s 2024-09-21T06:23:23.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:23 smithi031 ceph-mon[31237]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:23:24.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:23:24.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:23:24.649 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 705.0 s, 1893.6 tps, lat 53.274 ms stddev 68.323 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 -- 172.21.15.31:0/1900483597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 msgr2=0x7f55a010a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 --2- 172.21.15.31:0/1900483597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a010a5c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f559c009ca0 tx=0x7f559c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 -- 172.21.15.31:0/1900483597 shutdown_connections 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 --2- 172.21.15.31:0/1900483597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a0110440 0x7f55a0116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 --2- 172.21.15.31:0/1900483597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a010faf0 0x7f55a010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 --2- 172.21.15.31:0/1900483597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a010a5c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.204+0000 7f55a871c640 1 -- 172.21.15.31:0/1900483597 >> 172.21.15.31:0/1900483597 conn(0x7f55a00780e0 msgr2=0x7f55a01081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.205+0000 7f55a871c640 1 -- 172.21.15.31:0/1900483597 shutdown_connections 2024-09-21T06:23:25.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.205+0000 7f55a871c640 1 -- 172.21.15.31:0/1900483597 wait complete. 2024-09-21T06:23:25.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.205+0000 7f55a871c640 1 Processor -- start 2024-09-21T06:23:25.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 -- start start 2024-09-21T06:23:25.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a010faf0 0x7f55a01a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a0110440 0x7f55a01a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f55a011a140 con 0x7f55a010a1b0 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0119fc0 con 0x7f55a010faf0 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a871c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55a011a2c0 con 0x7f55a0110440 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a64b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a64b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a01a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41762/0 (socket says 172.21.15.31:41762) 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a64b8640 1 -- 172.21.15.31:0/3713694533 learned_addr learned my addr 172.21.15.31:0/3713694533 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a5cb7640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a010faf0 0x7f55a01a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.206+0000 7f55a6cb9640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a0110440 0x7f55a01a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.207+0000 7f55a64b8640 1 -- 172.21.15.31:0/3713694533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a0110440 msgr2=0x7f55a01a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.207+0000 7f55a64b8640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a0110440 0x7f55a01a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.207+0000 7f55a64b8640 1 -- 172.21.15.31:0/3713694533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a010faf0 msgr2=0x7f55a01a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.207+0000 7f55a64b8640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a010faf0 0x7f55a01a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.207+0000 7f55a64b8640 1 -- 172.21.15.31:0/3713694533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a01a6b00 con 0x7f55a010a1b0 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.207+0000 7f55a64b8640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a01a0b90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f559c005e30 tx=0x7f559c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.208+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c003b70 con 0x7f55a010a1b0 2024-09-21T06:23:25.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.208+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c003d10 con 0x7f55a010a1b0 2024-09-21T06:23:25.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.209+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a01a6d90 con 0x7f55a010a1b0 2024-09-21T06:23:25.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.209+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a007dc20 con 0x7f55a010a1b0 2024-09-21T06:23:25.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.209+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c02fa30 con 0x7f55a010a1b0 2024-09-21T06:23:25.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.211+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a010a5c0 con 0x7f55a010a1b0 2024-09-21T06:23:25.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.213+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f559c02fbd0 con 0x7f55a010a1b0 2024-09-21T06:23:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.214+0000 7f55977fe640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f558c0b67d0 0x7f558c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:25.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.215+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f559c108940 con 0x7f55a010a1b0 2024-09-21T06:23:25.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.215+0000 7f55a5cb7640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f558c0b67d0 0x7f558c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:25.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.216+0000 7f55a5cb7640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f558c0b67d0 0x7f558c0b8ce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f55a006b0a0 tx=0x7f5590005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:25.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.221+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f559c0ceea0 con 0x7f55a010a1b0 2024-09-21T06:23:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.646+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55a01a1cc0 con 0x7f55a010a1b0 2024-09-21T06:23:25.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.649+0000 7f55977fe640 1 -- 172.21.15.31:0/3713694533 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f559c0d3280 con 0x7f55a010a1b0 2024-09-21T06:23:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:23:25.654 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:23:25.654 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:23:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f558c0b67d0 msgr2=0x7f558c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f558c0b67d0 0x7f558c0b8ce0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f55a006b0a0 tx=0x7f5590005f90 comp rx=0 tx=0).stop 2024-09-21T06:23:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 msgr2=0x7f55a01a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a01a0b90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f559c005e30 tx=0x7f559c031840 comp rx=0 tx=0).stop 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 shutdown_connections 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a0110440 0x7f55a01a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55a010faf0 0x7f55a01a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f558c0b67d0 0x7f558c0b8ce0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 --2- 172.21.15.31:0/3713694533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55a010a1b0 0x7f55a01a0b90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.657+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 >> 172.21.15.31:0/3713694533 conn(0x7f55a00780e0 msgr2=0x7f55a0078900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.658+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 shutdown_connections 2024-09-21T06:23:25.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:25.658+0000 7f55a871c640 1 -- 172.21.15.31:0/3713694533 wait complete. 2024-09-21T06:23:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:25 smithi110 ceph-mon[34550]: pgmap v687: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 1.54k op/s 2024-09-21T06:23:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:25 smithi107 ceph-mon[34380]: pgmap v687: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 1.54k op/s 2024-09-21T06:23:25.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:25 smithi031 ceph-mon[31237]: pgmap v687: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 1.54k op/s 2024-09-21T06:23:26.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:23:26.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:26 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3713694533' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:26.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:26 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3713694533' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:26.879 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:26.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:26 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3713694533' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:23:27.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/3586554367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f4810de10 msgr2=0x7f6f48110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/3586554367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f4810de10 0x7f6f48110300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ca0 tx=0x7f6f4002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/3586554367 shutdown_connections 2024-09-21T06:23:27.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/3586554367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f4810de10 0x7f6f48110300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/3586554367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f48105300 0x7f6f4810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/3586554367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f481049b0 0x7f6f48104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/3586554367 >> 172.21.15.31:0/3586554367 conn(0x7f6f48079730 msgr2=0x7f6f48079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/3586554367 shutdown_connections 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.416+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/3586554367 wait complete. 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 Processor -- start 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 -- start start 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f481049b0 0x7f6f4819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 0x7f6f4819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f4810de10 0x7f6f4819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48113ac0 con 0x7f6f48105300 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48113940 con 0x7f6f4810de10 2024-09-21T06:23:27.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.417+0000 7f6f4cfd4640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f48113c40 con 0x7f6f481049b0 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 0x7f6f4819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 0x7f6f4819aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41782/0 (socket says 172.21.15.31:41782) 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 -- 172.21.15.31:0/1579216316 learned_addr learned my addr 172.21.15.31:0/1579216316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f4810de10 0x7f6f4819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f4659c640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f481049b0 0x7f6f4819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 -- 172.21.15.31:0/1579216316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f481049b0 msgr2=0x7f6f4819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f481049b0 0x7f6f4819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 -- 172.21.15.31:0/1579216316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f4810de10 msgr2=0x7f6f4819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f4810de10 0x7f6f4819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 -- 172.21.15.31:0/1579216316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f481a03e0 con 0x7f6f48105300 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.418+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 0x7f6f4819aa10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00d740 tx=0x7f6f3c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.419+0000 7f6f377fe640 1 -- 172.21.15.31:0/1579216316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c017070 con 0x7f6f48105300 2024-09-21T06:23:27.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.419+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f481a06d0 con 0x7f6f48105300 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.419+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f481a6e30 con 0x7f6f48105300 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.419+0000 7f6f377fe640 1 -- 172.21.15.31:0/1579216316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c00de50 con 0x7f6f48105300 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.419+0000 7f6f377fe640 1 -- 172.21.15.31:0/1579216316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c021060 con 0x7f6f48105300 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.422+0000 7f6f377fe640 1 -- 172.21.15.31:0/1579216316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6f3c01a070 con 0x7f6f48105300 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.422+0000 7f6f53548740 1 Processor -- start 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 -- start start 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672072e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x55672072ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x55672072f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5567206a4ea0 con 0x556720366390 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5567206942d0 con 0x55672035e1b0 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f53548740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5567206980e0 con 0x556720387070 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f377fe640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f300b6700 0x7f6f300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f46d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672072e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f46d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672072e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50820/0 (socket says 172.21.15.31:50820) 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x55672072f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f46d9d640 1 -- 172.21.15.31:0/2712020280 learned_addr learned my addr 172.21.15.31:0/2712020280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f300b6700 0x7f6f300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.423+0000 7f6f377fe640 1 -- 172.21.15.31:0/1579216316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f6f3c0e7010 con 0x7f6f48105300 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f4659c640 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x55672072ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 -- 172.21.15.31:0/2712020280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 msgr2=0x55672072ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x55672072ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 -- 172.21.15.31:0/2712020280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 msgr2=0x55672072f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x55672072f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 -- 172.21.15.31:0/2712020280 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567203f2650 con 0x556720387070 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672072e6a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6f4002ae10 tx=0x7f6f40032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5e00feafaed46cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f300b6700 0x7f6f300b8c10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ca0 tx=0x7f6f4003b000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f34ff9640 1 -- 172.21.15.31:0/2712020280 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40004240 con 0x556720387070 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f34ff9640 1 -- 172.21.15.31:0/2712020280 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f40036040 con 0x556720387070 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.424+0000 7f6f34ff9640 1 -- 172.21.15.31:0/2712020280 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400309d0 con 0x556720387070 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 -- 172.21.15.31:0/2712020280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 msgr2=0x55672072e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672072e6a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6f4002ae10 tx=0x7f6f40032cb0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 -- 172.21.15.31:0/2712020280 shutdown_connections 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x55672072f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x55672072ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 --2- 172.21.15.31:0/2712020280 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672072e6a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.425+0000 7f6f53548740 1 -- 172.21.15.31:0/2712020280 >> 172.21.15.31:0/2712020280 conn(0x55672037b460 msgr2=0x55672072c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.427+0000 7f6f53548740 1 -- 172.21.15.31:0/2712020280 shutdown_connections 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.427+0000 7f6f53548740 1 -- 172.21.15.31:0/2712020280 wait complete. 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.428+0000 7f6f53548740 1 -- start start 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.428+0000 7f6f53548740 1 Processor -- start 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55672035e1b0 0x556720739d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x556720746140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5567204f5700 con 0x55672035e1b0 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55672062f160 con 0x556720366390 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55672073ce20 con 0x556720387070 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f53548740 10 client.0 _ll_register_callbacks cb 0x5567206489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:27.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f46d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x556720746140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f46d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x556720746140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50826/0 (socket says 172.21.15.31:50826) 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f45d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f46d9d640 1 -- 172.21.15.31:0/433895346 learned_addr learned my addr 172.21.15.31:0/433895346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f4659c640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55672035e1b0 0x556720739d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f45d9b640 1 -- 172.21.15.31:0/433895346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 msgr2=0x556720746140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x556720746140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f45d9b640 1 -- 172.21.15.31:0/433895346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55672035e1b0 msgr2=0x556720739d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55672035e1b0 0x556720739d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f45d9b640 1 -- 172.21.15.31:0/433895346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55672065cc90 con 0x556720366390 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.429+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x556720746140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073a280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0e4770 tx=0x7f6f3c0217f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=653e0ac78374e443 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f2dffb640 1 -- 172.21.15.31:0/433895346 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0ac870 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f40036040 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f2dffb640 1 -- 172.21.15.31:0/433895346 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f6f3c0940f0 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f2dffb640 1 -- 172.21.15.31:0/433895346 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f3c0ac690 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f53548740 10 client.27584 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.430+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556720748b40 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f2dffb640 1 -- 172.21.15.31:0/433895346 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f40036040 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f2dffb640 1 -- 172.21.15.31:0/433895346 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6f3c059140 con 0x556720366390 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f53548740 10 client.27584 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f53548740 10 client.27584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f53548740 20 client.27584 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f53548740 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5567203459f0 0x55672074c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f53548740 4 client.27584 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567206a6df0 con 0x5567203459f0 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.431+0000 7f6f4659c640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5567203459f0 0x55672074c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.432+0000 7f6f2dffb640 10 client.27584 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.432+0000 7f6f4659c640 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5567203459f0 0x55672074c390 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dffe305fd23e9200 server_cookie=464b7d7af3098542 in_seq=0 out_seq=0 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f2dffb640 1 -- 172.21.15.31:0/433895346 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567206a6df0 con 0x5567203459f0 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f2dffb640 10 client.27584 handle_command_reply: tid=0 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 client.27584 shutdown 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 msgr2=0x55672073a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073a280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0e4770 tx=0x7f6f3c0217f0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 shutdown_connections 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5567203459f0 0x55672074c390 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x556720746140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073a280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 --2- 172.21.15.31:0/433895346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55672035e1b0 0x556720739d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 >> 172.21.15.31:0/433895346 conn(0x55672037b460 msgr2=0x556720731830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f2dffb640 0 client.27584 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 shutdown_connections 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.433+0000 7f6f53548740 1 -- 172.21.15.31:0/433895346 wait complete. 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.434+0000 7f6f53548740 20 client.27584 trim_cache size 0 max 16384 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.455+0000 7f6f53548740 1 Processor -- start 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.455+0000 7f6f53548740 1 -- start start 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.456+0000 7f6f53548740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x556720746040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.456+0000 7f6f53548740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x556720739af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.456+0000 7f6f53548740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672073a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.456+0000 7f6f53548740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481ad790 con 0x556720366390 2024-09-21T06:23:27.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.456+0000 7f6f53548740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481afff0 con 0x55672035e1b0 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.456+0000 7f6f53548740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481adde0 con 0x556720387070 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.459+0000 7f6f45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x556720739af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.459+0000 7f6f45d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x556720739af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41798/0 (socket says 172.21.15.31:41798) 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.459+0000 7f6f45d9b640 1 -- 172.21.15.31:0/337589158 learned_addr learned my addr 172.21.15.31:0/337589158 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.460+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672073a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.463+0000 7f6f4659c640 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x556720746040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.463+0000 7f6f4659c640 1 -- 172.21.15.31:0/337589158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 msgr2=0x55672073a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.463+0000 7f6f4659c640 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672073a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.463+0000 7f6f4659c640 1 -- 172.21.15.31:0/337589158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 msgr2=0x556720739af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.463+0000 7f6f4659c640 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x556720739af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.464+0000 7f6f4659c640 1 -- 172.21.15.31:0/337589158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556720730990 con 0x55672035e1b0 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.464+0000 7f6f4659c640 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x556720746040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55672074cb00 tx=0x7f6f3800bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13356a42e2d155b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.466+0000 7f6f2ffff640 1 -- 172.21.15.31:0/337589158 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38003b20 con 0x55672035e1b0 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.466+0000 7f6f2ffff640 1 -- 172.21.15.31:0/337589158 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f380054e0 con 0x55672035e1b0 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.468+0000 7f6f53548740 1 -- 172.21.15.31:0/337589158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 msgr2=0x556720746040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.468+0000 7f6f53548740 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x556720746040 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55672074cb00 tx=0x7f6f3800bbe0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.470+0000 7f6f53548740 1 -- 172.21.15.31:0/337589158 shutdown_connections 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.470+0000 7f6f53548740 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556720387070 0x55672073a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.470+0000 7f6f53548740 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720366390 0x556720739af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.470+0000 7f6f53548740 1 --2- 172.21.15.31:0/337589158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55672035e1b0 0x556720746040 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.470+0000 7f6f53548740 1 -- 172.21.15.31:0/337589158 >> 172.21.15.31:0/337589158 conn(0x55672037b460 msgr2=0x5567207385c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.477+0000 7f6f53548740 1 -- 172.21.15.31:0/337589158 shutdown_connections 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.479+0000 7f6f53548740 1 -- 172.21.15.31:0/337589158 wait complete. 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.479+0000 7f6f53548740 1 -- start start 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.479+0000 7f6f53548740 1 Processor -- start 2024-09-21T06:23:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55672035e1b0 0x55672073d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 0x5567207bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481ae4b0 con 0x556720387070 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481af920 con 0x556720366390 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6f481aeb80 con 0x55672035e1b0 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.480+0000 7f6f53548740 10 client.0 _ll_register_callbacks cb 0x7f6f3c0e7310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.482+0000 7f6f46d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 0x5567207bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.482+0000 7f6f46d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 0x5567207bfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41804/0 (socket says 172.21.15.31:41804) 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.482+0000 7f6f46d9d640 1 -- 172.21.15.31:0/3698210302 learned_addr learned my addr 172.21.15.31:0/3698210302 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.483+0000 7f6f4659c640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55672035e1b0 0x55672073d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.485+0000 7f6f45d9b640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.487+0000 7f6f46d9d640 1 -- 172.21.15.31:0/3698210302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55672035e1b0 msgr2=0x55672073d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.487+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55672035e1b0 0x55672073d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.487+0000 7f6f46d9d640 1 -- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 msgr2=0x55672073def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.487+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.487+0000 7f6f46d9d640 1 -- 172.21.15.31:0/3698210302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567207c03d0 con 0x556720387070 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.491+0000 7f6f4659c640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55672035e1b0 0x55672073d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.494+0000 7f6f46d9d640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 0x5567207bfd60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6f4002ae10 tx=0x7f6f400305f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce78a8c06d3ead6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.496+0000 7f6f2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.496+0000 7f6f2dffb640 1 -- 172.21.15.31:0/3698210302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f40061060 con 0x556720387070 2024-09-21T06:23:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.496+0000 7f6f2dffb640 1 -- 172.21.15.31:0/3698210302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f6f40009890 con 0x556720387070 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.496+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f380054e0 con 0x556720387070 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.497+0000 7f6f2dffb640 1 -- 172.21.15.31:0/3698210302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f40030eb0 con 0x556720387070 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.502+0000 7f6f53548740 10 client.17814 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.502+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5567207c7700 con 0x556720387070 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.506+0000 7f6f2dffb640 1 -- 172.21.15.31:0/3698210302 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f380054e0 con 0x556720387070 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.506+0000 7f6f2dffb640 1 -- 172.21.15.31:0/3698210302 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6f4005b070 con 0x556720387070 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.507+0000 7f6f53548740 10 client.17814 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.507+0000 7f6f53548740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.507+0000 7f6f53548740 20 client.17814 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.507+0000 7f6f53548740 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55672031ada0 0x5567207c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.507+0000 7f6f53548740 4 client.17814 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.507+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567206a28a0 con 0x55672031ada0 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.510+0000 7f6f4659c640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55672031ada0 0x5567207c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.514+0000 7f6f4659c640 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55672031ada0 0x5567207c9ad0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3d8d31c70e37cf server_cookie=17c5d7ec9caec9e6 in_seq=0 out_seq=0 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.515+0000 7f6f2dffb640 10 client.17814 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.515+0000 7f6f2dffb640 1 -- 172.21.15.31:0/3698210302 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567206a28a0 con 0x55672031ada0 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.515+0000 7f6f2dffb640 10 client.17814 handle_command_reply: tid=0 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.515+0000 7f6f53548740 1 client.17814 shutdown 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 msgr2=0x5567207bfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 0x5567207bfd60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6f4002ae10 tx=0x7f6f400305f0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 shutdown_connections 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55672031ada0 0x5567207c9ad0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556720387070 0x5567207bfd60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556720366390 0x55672073def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 --2- 172.21.15.31:0/3698210302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55672035e1b0 0x55672073d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f2dffb640 0 client.17814 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.517+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 >> 172.21.15.31:0/3698210302 conn(0x55672037b460 msgr2=0x556720739af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.518+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 shutdown_connections 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.518+0000 7f6f53548740 1 -- 172.21.15.31:0/3698210302 wait complete. 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.518+0000 7f6f53548740 20 client.17814 trim_cache size 0 max 16384 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.519+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f300b6700 msgr2=0x7f6f300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.519+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f300b6700 0x7f6f300b8c10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009ca0 tx=0x7f6f4003b000 comp rx=0 tx=0).stop 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.519+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 msgr2=0x7f6f4819aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.519+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 0x7f6f4819aa10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00d740 tx=0x7f6f3c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.519+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 shutdown_connections 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.520+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f4810de10 0x7f6f4819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.520+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f6f300b6700 0x7f6f300b8c10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.520+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f48105300 0x7f6f4819aa10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.520+0000 7f6f4cfd4640 1 --2- 172.21.15.31:0/1579216316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f481049b0 0x7f6f4819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.520+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 >> 172.21.15.31:0/1579216316 conn(0x7f6f48079730 msgr2=0x7f6f480ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.522+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 shutdown_connections 2024-09-21T06:23:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:27.522+0000 7f6f4cfd4640 1 -- 172.21.15.31:0/1579216316 wait complete. 2024-09-21T06:23:27.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:27 smithi031 ceph-mon[31237]: pgmap v688: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2024-09-21T06:23:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:27 smithi110 ceph-mon[34550]: pgmap v688: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2024-09-21T06:23:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:27 smithi107 ceph-mon[34380]: pgmap v688: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 35 MiB/s wr, 1.80k op/s 2024-09-21T06:23:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.000+0000 7fa748627640 1 -- 172.21.15.31:0/2010749871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa740110350 msgr2=0x7fa740110b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.000+0000 7fa748627640 1 --2- 172.21.15.31:0/2010749871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa740110350 0x7fa740110b70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009860 tx=0x7fa73c02a380 comp rx=0 tx=0).stop 2024-09-21T06:23:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.003+0000 7fa748627640 1 -- 172.21.15.31:0/2010749871 shutdown_connections 2024-09-21T06:23:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.003+0000 7fa748627640 1 --2- 172.21.15.31:0/2010749871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa740110350 0x7fa740110b70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.003+0000 7fa748627640 1 --2- 172.21.15.31:0/2010749871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa74010fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.003+0000 7fa748627640 1 --2- 172.21.15.31:0/2010749871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa74010a660 0x7fa74010aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.003+0000 7fa748627640 1 -- 172.21.15.31:0/2010749871 >> 172.21.15.31:0/2010749871 conn(0x7fa740070dd0 msgr2=0x7fa7400711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.003+0000 7fa748627640 1 -- 172.21.15.31:0/2010749871 shutdown_connections 2024-09-21T06:23:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.004+0000 7fa748627640 1 -- 172.21.15.31:0/2010749871 wait complete. 2024-09-21T06:23:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.005+0000 7fa748627640 1 Processor -- start 2024-09-21T06:23:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.006+0000 7fa748627640 1 -- start start 2024-09-21T06:23:29.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.006+0000 7fa748627640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa74010a660 0x7fa7401a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.006+0000 7fa748627640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa7401a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa748627640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa740110350 0x7fa7401aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa748627640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa740122850 con 0x7fa74010a660 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa748627640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401226d0 con 0x7fa740110350 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa748627640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401229d0 con 0x7fa74010fa00 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa7401a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa7401a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50850/0 (socket says 172.21.15.31:50850) 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 -- 172.21.15.31:0/1185629996 learned_addr learned my addr 172.21.15.31:0/1185629996 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa740110350 0x7fa7401aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa7463c3640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa74010a660 0x7fa7401a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 -- 172.21.15.31:0/1185629996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa740110350 msgr2=0x7fa7401aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa740110350 0x7fa7401aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 -- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa74010a660 msgr2=0x7fa7401a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa74010a660 0x7fa7401a9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.007+0000 7fa745bc2640 1 -- 172.21.15.31:0/1185629996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7401af1c0 con 0x7fa74010fa00 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.008+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa7401a9830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b990 tx=0x7fa73800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.008+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa740110350 0x7fa7401aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.008+0000 7fa7377fe640 1 -- 172.21.15.31:0/1185629996 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738019070 con 0x7fa74010fa00 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.008+0000 7fa7377fe640 1 -- 172.21.15.31:0/1185629996 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa73800cba0 con 0x7fa74010fa00 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.008+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7401af4b0 con 0x7fa74010fa00 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.009+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7401b5c40 con 0x7fa74010fa00 2024-09-21T06:23:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.009+0000 7fa7377fe640 1 -- 172.21.15.31:0/1185629996 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7380125a0 con 0x7fa74010fa00 2024-09-21T06:23:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.013+0000 7fa7377fe640 1 -- 172.21.15.31:0/1185629996 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa73801c070 con 0x7fa74010fa00 2024-09-21T06:23:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 Processor -- start 2024-09-21T06:23:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 -- start start 2024-09-21T06:23:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa7377fe640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa72c0b6880 0x7fa72c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x5566035979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x556603597ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x556603598430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55660350c6f0 con 0x5566031f21b0 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5566034fbb20 con 0x5566031d14d0 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa74f66f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5566034ff930 con 0x5566031c92f0 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.015+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa72c0b6880 0x7fa72c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x556603597ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa746bc4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x556603598430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x556603597ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50864/0 (socket says 172.21.15.31:50864) 2024-09-21T06:23:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 -- 172.21.15.31:0/1400353601 learned_addr learned my addr 172.21.15.31:0/1400353601 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 -- 172.21.15.31:0/1400353601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 msgr2=0x556603598430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x556603598430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 -- 172.21.15.31:0/1400353601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 msgr2=0x5566035979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x5566035979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 -- 172.21.15.31:0/1400353601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566034c1530 con 0x5566031c92f0 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.016+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x556603597ef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ad870 tx=0x7fa738009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7961753df52c62ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.017+0000 7fa734bfe640 1 -- 172.21.15.31:0/1400353601 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7380833c0 con 0x5566031c92f0 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.017+0000 7fa734bfe640 1 -- 172.21.15.31:0/1400353601 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7380ad3f0 con 0x5566031c92f0 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.017+0000 7fa734bfe640 1 -- 172.21.15.31:0/1400353601 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738081340 con 0x5566031c92f0 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.017+0000 7fa7377fe640 1 -- 172.21.15.31:0/1185629996 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7fa7380e7ae0 con 0x7fa74010fa00 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 -- 172.21.15.31:0/1400353601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 msgr2=0x556603597ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa72c0b6880 0x7fa72c0b8d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa73c02a8f0 tx=0x7fa73c02a880 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x556603597ef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ad870 tx=0x7fa738009950 comp rx=0 tx=0).stop 2024-09-21T06:23:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 -- 172.21.15.31:0/1400353601 shutdown_connections 2024-09-21T06:23:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x556603598430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x556603597ef0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1400353601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x5566035979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 -- 172.21.15.31:0/1400353601 >> 172.21.15.31:0/1400353601 conn(0x5566031e6300 msgr2=0x5566035958a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 -- 172.21.15.31:0/1400353601 shutdown_connections 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.018+0000 7fa74f66f740 1 -- 172.21.15.31:0/1400353601 wait complete. 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 -- start start 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 Processor -- start 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556603433ab0 con 0x5566031c92f0 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5566034a1730 con 0x5566031d14d0 2024-09-21T06:23:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5566035a6130 con 0x5566031f21b0 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.019+0000 7fa74f66f740 10 client.0 _ll_register_callbacks cb 0x5566034f56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41846/0 (socket says 172.21.15.31:41846) 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 -- 172.21.15.31:0/771739832 learned_addr learned my addr 172.21.15.31:0/771739832 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa745bc2640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa746bc4640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 -- 172.21.15.31:0/771739832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 msgr2=0x5566035af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 -- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 msgr2=0x5566035a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 -- 172.21.15.31:0/771739832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660325d790 con 0x5566031c92f0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa7463c3640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035a3050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa73000de10 tx=0x7fa73000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3ba76d8ff1a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.020+0000 7fa729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.021+0000 7fa729ffb640 1 -- 172.21.15.31:0/771739832 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730011850 con 0x5566031c92f0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.021+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7380ad3f0 con 0x5566031c92f0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.021+0000 7fa729ffb640 1 -- 172.21.15.31:0/771739832 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7fa7300119f0 con 0x5566031c92f0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.021+0000 7fa729ffb640 1 -- 172.21.15.31:0/771739832 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa730023050 con 0x5566031c92f0 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.021+0000 7fa74f66f740 10 client.17823 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.021+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5566032d6620 con 0x5566031c92f0 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa729ffb640 1 -- 172.21.15.31:0/771739832 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7380ad3f0 con 0x5566031c92f0 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa729ffb640 1 -- 172.21.15.31:0/771739832 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa730010b70 con 0x5566031c92f0 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa74f66f740 10 client.17823 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa74f66f740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa74f66f740 20 client.17823 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa74f66f740 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031b0d80 0x5566035b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa74f66f740 4 client.17823 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.022+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55660350e640 con 0x5566031b0d80 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.023+0000 7fa745bc2640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031b0d80 0x5566035b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.023+0000 7fa745bc2640 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031b0d80 0x5566035b52a0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4bff6d29ace6aa7 server_cookie=2028c2caa7769252 in_seq=0 out_seq=0 2024-09-21T06:23:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.023+0000 7fa729ffb640 10 client.17823 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa729ffb640 1 -- 172.21.15.31:0/771739832 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55660350e640 con 0x5566031b0d80 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa729ffb640 10 client.17823 handle_command_reply: tid=0 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 client.17823 shutdown 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 msgr2=0x5566035a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035a3050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa73000de10 tx=0x7fa73000f730 comp rx=0 tx=0).stop 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 shutdown_connections 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031b0d80 0x5566035b52a0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 --2- 172.21.15.31:0/771739832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035a3050 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 >> 172.21.15.31:0/771739832 conn(0x5566031e6300 msgr2=0x5566035989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa729ffb640 0 client.17823 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 shutdown_connections 2024-09-21T06:23:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 1 -- 172.21.15.31:0/771739832 wait complete. 2024-09-21T06:23:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.024+0000 7fa74f66f740 20 client.17823 trim_cache size 0 max 16384 2024-09-21T06:23:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 Processor -- start 2024-09-21T06:23:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 -- start start 2024-09-21T06:23:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401bc550 con 0x5566031c92f0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401bedb0 con 0x5566031d14d0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.043+0000 7fa74f66f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401bcba0 con 0x5566031f21b0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41854/0 (socket says 172.21.15.31:41854) 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 -- 172.21.15.31:0/3454555835 learned_addr learned my addr 172.21.15.31:0/3454555835 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa746bc4640 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa745bc2640 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 -- 172.21.15.31:0/3454555835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 msgr2=0x5566035af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 -- 172.21.15.31:0/3454555835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 msgr2=0x5566035af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 -- 172.21.15.31:0/3454555835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556603599cb0 con 0x5566031c92f0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa7463c3640 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035990f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa730010f40 tx=0x7fa73000d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e29d0e3ddfbd0f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa72bfff640 1 -- 172.21.15.31:0/3454555835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73002c070 con 0x5566031c92f0 2024-09-21T06:23:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa72bfff640 1 -- 172.21.15.31:0/3454555835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa730017070 con 0x5566031c92f0 2024-09-21T06:23:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.044+0000 7fa72bfff640 1 -- 172.21.15.31:0/3454555835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730003850 con 0x5566031c92f0 2024-09-21T06:23:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.046+0000 7fa74f66f740 1 -- 172.21.15.31:0/3454555835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 msgr2=0x5566035990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.046+0000 7fa74f66f740 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035990f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa730010f40 tx=0x7fa73000d5a0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 -- 172.21.15.31:0/3454555835 shutdown_connections 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031f21b0 0x5566035af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 --2- 172.21.15.31:0/3454555835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031c92f0 0x5566035990f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 -- 172.21.15.31:0/3454555835 >> 172.21.15.31:0/3454555835 conn(0x5566031e6300 msgr2=0x5566035989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 -- 172.21.15.31:0/3454555835 shutdown_connections 2024-09-21T06:23:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.047+0000 7fa74f66f740 1 -- 172.21.15.31:0/3454555835 wait complete. 2024-09-21T06:23:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 -- start start 2024-09-21T06:23:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 Processor -- start 2024-09-21T06:23:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x5566035a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.052 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x55660361dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401bd270 con 0x5566031f21b0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401be6e0 con 0x5566031d14d0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401bd940 con 0x5566031c92f0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa74f66f740 10 client.0 _ll_register_callbacks cb 0x7fa73000d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa7463c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x5566035a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa746bc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x55660361dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.048+0000 7fa7463c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x5566035a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50884/0 (socket says 172.21.15.31:50884) 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa7463c3640 1 -- 172.21.15.31:0/1505524478 learned_addr learned my addr 172.21.15.31:0/1505524478 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa745bc2640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa746bc4640 1 -- 172.21.15.31:0/1505524478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 msgr2=0x5566035a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x5566035a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa746bc4640 1 -- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 msgr2=0x5566035a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:29.053 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa746bc4640 1 -- 172.21.15.31:0/1505524478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55660361e1a0 con 0x5566031f21b0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa7463c3640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x5566035a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa746bc4640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x55660361dc60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa73c02a970 tx=0x7fa73c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4746b635c9af9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa729ffb640 1 -- 172.21.15.31:0/1505524478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c052070 con 0x5566031f21b0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa730017070 con 0x5566031f21b0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa729ffb640 1 -- 172.21.15.31:0/1505524478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7fa73c0035f0 con 0x5566031f21b0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.049+0000 7fa729ffb640 1 -- 172.21.15.31:0/1505524478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa73c05e050 con 0x5566031f21b0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 10 client.17829 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55660361f300 con 0x5566031f21b0 2024-09-21T06:23:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa729ffb640 1 -- 172.21.15.31:0/1505524478 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa730017070 con 0x5566031f21b0 2024-09-21T06:23:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa729ffb640 1 -- 172.21.15.31:0/1505524478 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa73c0328b0 con 0x5566031f21b0 2024-09-21T06:23:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 10 client.17829 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 20 client.17829 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031861c0 0x556603621ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 4 client.17829 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55660350a0f0 con 0x5566031861c0 2024-09-21T06:23:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.050+0000 7fa7463c3640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031861c0 0x556603621ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.051+0000 7fa7463c3640 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031861c0 0x556603621ac0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae1a4353c434261 server_cookie=4d3310497219a788 in_seq=0 out_seq=0 2024-09-21T06:23:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.051+0000 7fa729ffb640 10 client.17829 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa729ffb640 1 -- 172.21.15.31:0/1505524478 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55660350a0f0 con 0x5566031861c0 2024-09-21T06:23:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa729ffb640 10 client.17829 handle_command_reply: tid=0 2024-09-21T06:23:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 client.17829 shutdown 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 msgr2=0x55660361dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x55660361dc60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa73c02a970 tx=0x7fa73c005f90 comp rx=0 tx=0).stop 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 shutdown_connections 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5566031861c0 0x556603621ac0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5566031f21b0 0x55660361dc60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5566031d14d0 0x5566035a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 --2- 172.21.15.31:0/1505524478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5566031c92f0 0x5566035a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa729ffb640 0 client.17829 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 >> 172.21.15.31:0/1505524478 conn(0x5566031e6300 msgr2=0x5566035989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.088 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.084+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 shutdown_connections 2024-09-21T06:23:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.086+0000 7fa74f66f740 1 -- 172.21.15.31:0/1505524478 wait complete. 2024-09-21T06:23:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.086+0000 7fa74f66f740 20 client.17829 trim_cache size 0 max 16384 2024-09-21T06:23:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:23:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:23:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:23:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:23:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.086+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa72c0b6880 msgr2=0x7fa72c0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.086+0000 7fa748627640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa72c0b6880 0x7fa72c0b8d90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa73c02a8f0 tx=0x7fa73c02a880 comp rx=0 tx=0).stop 2024-09-21T06:23:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.086+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 msgr2=0x7fa7401a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.086+0000 7fa748627640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa7401a9830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b990 tx=0x7fa73800be50 comp rx=0 tx=0).stop 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 shutdown_connections 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa740110350 0x7fa7401aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa74010fa00 0x7fa7401a9830 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa72c0b6880 0x7fa72c0b8d90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 --2- 172.21.15.31:0/1185629996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa74010a660 0x7fa7401a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 >> 172.21.15.31:0/1185629996 conn(0x7fa740070dd0 msgr2=0x7fa740073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 shutdown_connections 2024-09-21T06:23:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:29.087+0000 7fa748627640 1 -- 172.21.15.31:0/1185629996 wait complete. 2024-09-21T06:23:29.385 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:23:29.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:23:29.745 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:29 smithi110 ceph-mon[34550]: pgmap v689: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 1.72k op/s 2024-09-21T06:23:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:29 smithi107 ceph-mon[34380]: pgmap v689: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 1.72k op/s 2024-09-21T06:23:29.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:29 smithi031 ceph-mon[31237]: pgmap v689: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 MiB/s wr, 1.72k op/s 2024-09-21T06:23:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 710.0 s, 1608.4 tps, lat 61.215 ms stddev 77.231 2024-09-21T06:23:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.419+0000 7f698bf1e640 1 -- 172.21.15.31:0/3087936633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410a9e0 msgr2=0x7f698410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.419+0000 7f698bf1e640 1 --2- 172.21.15.31:0/3087936633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410a9e0 0x7f698410adf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f697401a910 tx=0x7f697403b470 comp rx=0 tx=0).stop 2024-09-21T06:23:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.421+0000 7f698bf1e640 1 -- 172.21.15.31:0/3087936633 shutdown_connections 2024-09-21T06:23:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.421+0000 7f698bf1e640 1 --2- 172.21.15.31:0/3087936633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698411cc30 0x7f698411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.421+0000 7f698bf1e640 1 --2- 172.21.15.31:0/3087936633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f698410a9e0 0x7f698410adf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.421+0000 7f698bf1e640 1 --2- 172.21.15.31:0/3087936633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6984111f40 0x7f698410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.421+0000 7f698bf1e640 1 -- 172.21.15.31:0/3087936633 >> 172.21.15.31:0/3087936633 conn(0x7f6984070ec0 msgr2=0x7f69840712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.431+0000 7f698bf1e640 1 -- 172.21.15.31:0/3087936633 shutdown_connections 2024-09-21T06:23:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.431+0000 7f698bf1e640 1 -- 172.21.15.31:0/3087936633 wait complete. 2024-09-21T06:23:30.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 Processor -- start 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 -- start start 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 0x7f698411c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984111f40 0x7f69841186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698411cc30 0x7f6984118c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6984114bc0 con 0x7f698411cc30 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6984114a40 con 0x7f698410a9e0 2024-09-21T06:23:30.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.432+0000 7f698bf1e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6984114d40 con 0x7f6984111f40 2024-09-21T06:23:30.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.435+0000 7f6989cba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 0x7f698411c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.435+0000 7f6989cba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 0x7f698411c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33318/0 (socket says 172.21.15.31:33318) 2024-09-21T06:23:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.435+0000 7f698a4bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698411cc30 0x7f6984118c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.435+0000 7f698a4bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698411cc30 0x7f6984118c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52364/0 (socket says 172.21.15.31:52364) 2024-09-21T06:23:30.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.435+0000 7f698a4bb640 1 -- 172.21.15.31:0/395464555 learned_addr learned my addr 172.21.15.31:0/395464555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f69894b9640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984111f40 0x7f69841186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f6989cba640 1 -- 172.21.15.31:0/395464555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984111f40 msgr2=0x7f69841186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f6989cba640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984111f40 0x7f69841186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f6989cba640 1 -- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698411cc30 msgr2=0x7f6984118c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f6989cba640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698411cc30 0x7f6984118c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f6989cba640 1 -- 172.21.15.31:0/395464555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6984119430 con 0x7f698410a9e0 2024-09-21T06:23:30.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.441+0000 7f69894b9640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984111f40 0x7f69841186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:30.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.445+0000 7f6989cba640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 0x7f698411c3f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f698000c970 tx=0x7f698000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.451+0000 7f697affd640 1 -- 172.21.15.31:0/395464555 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980007bb0 con 0x7f698410a9e0 2024-09-21T06:23:30.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.451+0000 7f697affd640 1 -- 172.21.15.31:0/395464555 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6980007d50 con 0x7f698410a9e0 2024-09-21T06:23:30.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.451+0000 7f697affd640 1 -- 172.21.15.31:0/395464555 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980004df0 con 0x7f698410a9e0 2024-09-21T06:23:30.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.451+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841ace30 con 0x7f698410a9e0 2024-09-21T06:23:30.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.451+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841ad290 con 0x7f698410a9e0 2024-09-21T06:23:30.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.456+0000 7f6992492740 1 Processor -- start 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.459+0000 7f6992492740 1 -- start start 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f6992492740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e922430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f6992492740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e922970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f6992492740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e922eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f6992492740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56164e898e60 con 0x56164e57b350 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f6992492740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56164e888290 con 0x56164e55a670 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f6992492740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56164e88c0a0 con 0x56164e552490 2024-09-21T06:23:30.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.460+0000 7f697affd640 1 -- 172.21.15.31:0/395464555 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f69800046c0 con 0x7f698410a9e0 2024-09-21T06:23:30.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.462+0000 7f698a4bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e922430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.462+0000 7f698a4bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e922430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52366/0 (socket says 172.21.15.31:52366) 2024-09-21T06:23:30.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.463+0000 7f698a4bb640 1 -- 172.21.15.31:0/1136312187 learned_addr learned my addr 172.21.15.31:0/1136312187 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.463+0000 7f69894b9640 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e922eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.464+0000 7f697affd640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69700b6880 0x7f69700b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.468+0000 7f6989cba640 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e922970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.469+0000 7f697affd640 1 -- 172.21.15.31:0/395464555 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f69800e7090 con 0x7f698410a9e0 2024-09-21T06:23:30.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.470+0000 7f6989cba640 1 -- 172.21.15.31:0/1136312187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 msgr2=0x56164e922eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.474+0000 7f698a4bb640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69700b6880 0x7f69700b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.477+0000 7f6989cba640 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e922eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.477+0000 7f6989cba640 1 -- 172.21.15.31:0/1136312187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 msgr2=0x56164e922430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.477+0000 7f6989cba640 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e922430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.477+0000 7f6989cba640 1 -- 172.21.15.31:0/1136312187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164e5e6930 con 0x56164e552490 2024-09-21T06:23:30.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.477+0000 7f698a4bb640 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e922430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.479+0000 7f6989cba640 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e922970 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f69800e4790 tx=0x7f69800acd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eabc7fb74fdb52d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.479+0000 7f698a4bb640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69700b6880 0x7f69700b8d90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f697c009fd0 tx=0x7f697c015040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.480+0000 7f696ffff640 1 -- 172.21.15.31:0/1136312187 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698001d050 con 0x56164e552490 2024-09-21T06:23:30.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.480+0000 7f696ffff640 1 -- 172.21.15.31:0/1136312187 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69800ac9a0 con 0x56164e552490 2024-09-21T06:23:30.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.480+0000 7f696ffff640 1 -- 172.21.15.31:0/1136312187 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800abb60 con 0x56164e552490 2024-09-21T06:23:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 -- 172.21.15.31:0/1136312187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 msgr2=0x56164e922970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e922970 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f69800e4790 tx=0x7f69800acd20 comp rx=0 tx=0).stop 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 -- 172.21.15.31:0/1136312187 shutdown_connections 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e922eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e922970 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 --2- 172.21.15.31:0/1136312187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e922430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 -- 172.21.15.31:0/1136312187 >> 172.21.15.31:0/1136312187 conn(0x56164e56f740 msgr2=0x56164e920320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.482+0000 7f6992492740 1 -- 172.21.15.31:0/1136312187 shutdown_connections 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.483+0000 7f6992492740 1 -- 172.21.15.31:0/1136312187 wait complete. 2024-09-21T06:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.483+0000 7f6992492740 1 -- start start 2024-09-21T06:23:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.483+0000 7f6992492740 1 Processor -- start 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 0x56164e92dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e92e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e57b350 0x56164e939f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56164e6e9890 con 0x56164e552490 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56164e822d40 con 0x56164e55a670 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56164e930bb0 con 0x56164e57b350 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.484+0000 7f6992492740 10 client.0 _ll_register_callbacks cb 0x56164e886c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.485+0000 7f6989cba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 0x56164e92dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.485+0000 7f6989cba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 0x56164e92dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52368/0 (socket says 172.21.15.31:52368) 2024-09-21T06:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.485+0000 7f6989cba640 1 -- 172.21.15.31:0/750349386 learned_addr learned my addr 172.21.15.31:0/750349386 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.485+0000 7f69894b9640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e92e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.485+0000 7f698a4bb640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e57b350 0x56164e939f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.486+0000 7f6989cba640 1 -- 172.21.15.31:0/750349386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e57b350 msgr2=0x56164e939f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.486+0000 7f6989cba640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e57b350 0x56164e939f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.486+0000 7f6989cba640 1 -- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 msgr2=0x56164e92e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.486+0000 7f6989cba640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e92e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.486+0000 7f6989cba640 1 -- 172.21.15.31:0/750349386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164e93a440 con 0x56164e552490 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.486+0000 7f6989cba640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 0x56164e92dcf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6980007590 tx=0x7f69800a9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7d8ae7046024bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.487+0000 7f696d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.487+0000 7f696d7fa640 1 -- 172.21.15.31:0/750349386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980098020 con 0x56164e552490 2024-09-21T06:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.487+0000 7f696d7fa640 1 -- 172.21.15.31:0/750349386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f69800d41b0 con 0x56164e552490 2024-09-21T06:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.487+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69800ac9a0 con 0x56164e552490 2024-09-21T06:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.488+0000 7f696d7fa640 1 -- 172.21.15.31:0/750349386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f698001d030 con 0x56164e552490 2024-09-21T06:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.488+0000 7f6992492740 10 client.17838 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.488+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56164e93cca0 con 0x56164e552490 2024-09-21T06:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f696d7fa640 1 -- 172.21.15.31:0/750349386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69800ac9a0 con 0x56164e552490 2024-09-21T06:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f696d7fa640 1 -- 172.21.15.31:0/750349386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f69800abb60 con 0x56164e552490 2024-09-21T06:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f6992492740 10 client.17838 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f6992492740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f6992492740 20 client.17838 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f6992492740 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e539cd0 0x56164e940300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f6992492740 4 client.17838 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:30.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.490+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164e89adb0 con 0x56164e539cd0 2024-09-21T06:23:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.491+0000 7f69894b9640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e539cd0 0x56164e940300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.492+0000 7f696d7fa640 10 client.17838 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:30.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.492+0000 7f69894b9640 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e539cd0 0x56164e940300 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902d2424468d7f7c server_cookie=af891d73e7b7a0c7 in_seq=0 out_seq=0 2024-09-21T06:23:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.493+0000 7f696d7fa640 1 -- 172.21.15.31:0/750349386 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56164e89adb0 con 0x56164e539cd0 2024-09-21T06:23:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.493+0000 7f696d7fa640 10 client.17838 handle_command_reply: tid=0 2024-09-21T06:23:30.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.493+0000 7f6992492740 1 client.17838 shutdown 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.493+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 msgr2=0x56164e92dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.493+0000 7f6992492740 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 0x56164e92dcf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6980007590 tx=0x7f69800a9330 comp rx=0 tx=0).stop 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 shutdown_connections 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e539cd0 0x56164e940300 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e57b350 0x56164e939f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e92e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 --2- 172.21.15.31:0/750349386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e552490 0x56164e92dcf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 >> 172.21.15.31:0/750349386 conn(0x56164e56f740 msgr2=0x56164e9255c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f696d7fa640 0 client.17838 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 shutdown_connections 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.495+0000 7f6992492740 1 -- 172.21.15.31:0/750349386 wait complete. 2024-09-21T06:23:30.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.496+0000 7f6992492740 20 client.17838 trim_cache size 0 max 16384 2024-09-21T06:23:30.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 Processor -- start 2024-09-21T06:23:30.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 -- start start 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e939e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 0x56164e923b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e57b350 0x56164e92aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f696800c770 con 0x56164e55a670 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f696800efd0 con 0x56164e57b350 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.525+0000 7f6992492740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f696800cdc0 con 0x56164e552490 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f69894b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 0x56164e923b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f69894b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 0x56164e923b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52380/0 (socket says 172.21.15.31:52380) 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f69894b9640 1 -- 172.21.15.31:0/2219947095 learned_addr learned my addr 172.21.15.31:0/2219947095 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:30.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e939e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f698a4bb640 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e57b350 0x56164e92aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 -- 172.21.15.31:0/2219947095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e57b350 msgr2=0x56164e92aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e57b350 0x56164e92aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 -- 172.21.15.31:0/2219947095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 msgr2=0x56164e923b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 0x56164e923b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 -- 172.21.15.31:0/2219947095 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164e92efb0 con 0x56164e552490 2024-09-21T06:23:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f69894b9640 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 0x56164e923b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.526+0000 7f6989cba640 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e939e00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6980000c00 tx=0x7f69800acd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59528c2d0f98ee3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.527+0000 7f696f7fe640 1 -- 172.21.15.31:0/2219947095 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980098040 con 0x56164e552490 2024-09-21T06:23:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.527+0000 7f696f7fe640 1 -- 172.21.15.31:0/2219947095 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69800c8020 con 0x56164e552490 2024-09-21T06:23:30.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.527+0000 7f696f7fe640 1 -- 172.21.15.31:0/2219947095 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69800981e0 con 0x56164e552490 2024-09-21T06:23:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 -- 172.21.15.31:0/2219947095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 msgr2=0x56164e939e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e939e00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6980000c00 tx=0x7f69800acd20 comp rx=0 tx=0).stop 2024-09-21T06:23:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 -- 172.21.15.31:0/2219947095 shutdown_connections 2024-09-21T06:23:30.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e57b350 0x56164e92aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e55a670 0x56164e923b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 --2- 172.21.15.31:0/2219947095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e939e00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.528+0000 7f6992492740 1 -- 172.21.15.31:0/2219947095 >> 172.21.15.31:0/2219947095 conn(0x56164e56f740 msgr2=0x56164e923460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.529+0000 7f6992492740 1 -- 172.21.15.31:0/2219947095 shutdown_connections 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.529+0000 7f6992492740 1 -- 172.21.15.31:0/2219947095 wait complete. 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.529+0000 7f6992492740 1 -- start start 2024-09-21T06:23:30.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.529+0000 7f6992492740 1 Processor -- start 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e932010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e932550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e932a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f696800d490 con 0x56164e57b350 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f696800e900 con 0x56164e55a670 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f696800db60 con 0x56164e552490 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.530+0000 7f6992492740 10 client.0 _ll_register_callbacks cb 0x7f69800b1970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f698a4bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e932a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f698a4bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e932a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52388/0 (socket says 172.21.15.31:52388) 2024-09-21T06:23:30.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f698a4bb640 1 -- 172.21.15.31:0/1517384192 learned_addr learned my addr 172.21.15.31:0/1517384192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e932550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f6989cba640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e932010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 -- 172.21.15.31:0/1517384192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 msgr2=0x56164e932010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e932010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 -- 172.21.15.31:0/1517384192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 msgr2=0x56164e932a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e932a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 -- 172.21.15.31:0/1517384192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164e9a9f00 con 0x56164e55a670 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f6989cba640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e932010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f69894b9640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e932550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6974005dc0 tx=0x7f69740042c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295c0db10f9e35e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.531+0000 7f696d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f696d7fa640 1 -- 172.21.15.31:0/1517384192 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974063060 con 0x56164e55a670 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69800c8020 con 0x56164e55a670 2024-09-21T06:23:30.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f696d7fa640 1 -- 172.21.15.31:0/1517384192 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f6974040510 con 0x56164e55a670 2024-09-21T06:23:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f696d7fa640 1 -- 172.21.15.31:0/1517384192 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6974040710 con 0x56164e55a670 2024-09-21T06:23:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f6992492740 10 client.27620 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:30.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56164e9ab250 con 0x56164e55a670 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.532+0000 7f696d7fa640 1 -- 172.21.15.31:0/1517384192 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69800c8020 con 0x56164e55a670 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f696d7fa640 1 -- 172.21.15.31:0/1517384192 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f697405d070 con 0x56164e55a670 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6992492740 10 client.27620 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6992492740 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6992492740 20 client.27620 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6992492740 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e50f080 0x56164e9ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6992492740 4 client.27620 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56164e896860 con 0x56164e50f080 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.533+0000 7f6989cba640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e50f080 0x56164e9ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:30.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.534+0000 7f696d7fa640 10 client.27620 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.534+0000 7f6989cba640 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e50f080 0x56164e9ad9c0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb654f8ae336fea server_cookie=7153e49eb06fa8 in_seq=0 out_seq=0 2024-09-21T06:23:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.535+0000 7f696d7fa640 1 -- 172.21.15.31:0/1517384192 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56164e896860 con 0x56164e50f080 2024-09-21T06:23:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.535+0000 7f696d7fa640 10 client.27620 handle_command_reply: tid=0 2024-09-21T06:23:30.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.535+0000 7f6992492740 1 client.27620 shutdown 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.535+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 msgr2=0x56164e932550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.535+0000 7f6992492740 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e932550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6974005dc0 tx=0x7f69740042c0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 shutdown_connections 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56164e50f080 0x56164e9ad9c0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56164e57b350 0x56164e932a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f696d7fa640 0 client.27620 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56164e55a670 0x56164e932550 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 --2- 172.21.15.31:0/1517384192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56164e552490 0x56164e932010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 >> 172.21.15.31:0/1517384192 conn(0x56164e56f740 msgr2=0x56164e923440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 shutdown_connections 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 1 -- 172.21.15.31:0/1517384192 wait complete. 2024-09-21T06:23:30.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.536+0000 7f6992492740 20 client.27620 trim_cache size 0 max 16384 2024-09-21T06:23:30.541 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:23:30.541 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:23:30.541 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "25079243-b4e8-4dfe-98d8-245107f7d0c4", 2024-09-21T06:23:30.541 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:23:30.541 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69700b6880 msgr2=0x7f69700b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69700b6880 0x7f69700b8d90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f697c009fd0 tx=0x7f697c015040 comp rx=0 tx=0).stop 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 msgr2=0x7f698411c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 0x7f698411c3f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f698000c970 tx=0x7f698000ce30 comp rx=0 tx=0).stop 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 shutdown_connections 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f69700b6880 0x7f69700b8d90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698411cc30 0x7f6984118c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6984111f40 0x7f69841186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 --2- 172.21.15.31:0/395464555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f698410a9e0 0x7f698411c3f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.537+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 >> 172.21.15.31:0/395464555 conn(0x7f6984070ec0 msgr2=0x7f698410ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.538+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 shutdown_connections 2024-09-21T06:23:30.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:30.538+0000 7f698bf1e640 1 -- 172.21.15.31:0/395464555 wait complete. 2024-09-21T06:23:31.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:23:31.591 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:31 smithi031 ceph-mon[31237]: pgmap v690: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2024-09-21T06:23:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:31 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:23:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:31 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:31.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:31 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:23:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:31 smithi110 ceph-mon[34550]: pgmap v690: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2024-09-21T06:23:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:31 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:23:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:31 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:31 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:23:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:31 smithi107 ceph-mon[34380]: pgmap v690: 97 pgs: 97 active+clean; 4.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 27 MiB/s wr, 1.58k op/s 2024-09-21T06:23:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:31 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:23:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:31 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:31 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:23:32.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.228+0000 7f514ca73640 1 -- 172.21.15.31:0/1512308575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5148111f40 msgr2=0x7f514810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.228+0000 7f514ca73640 1 --2- 172.21.15.31:0/1512308575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5148111f40 0x7f514810a3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5140009ca0 tx=0x7f514002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 -- 172.21.15.31:0/1512308575 shutdown_connections 2024-09-21T06:23:32.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 --2- 172.21.15.31:0/1512308575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f514811cc30 0x7f514811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 --2- 172.21.15.31:0/1512308575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514810a9e0 0x7f514810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 --2- 172.21.15.31:0/1512308575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5148111f40 0x7f514810a3c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 -- 172.21.15.31:0/1512308575 >> 172.21.15.31:0/1512308575 conn(0x7f5148070ec0 msgr2=0x7f51480712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 -- 172.21.15.31:0/1512308575 shutdown_connections 2024-09-21T06:23:32.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.229+0000 7f514ca73640 1 -- 172.21.15.31:0/1512308575 wait complete. 2024-09-21T06:23:32.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.230+0000 7f514ca73640 1 Processor -- start 2024-09-21T06:23:32.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 -- start start 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 0x7f514811bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5148111f40 0x7f514811c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514811cc30 0x7f51481163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5148112830 con 0x7f514811cc30 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f51481126b0 con 0x7f514810a9e0 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514ca73640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f51481129b0 con 0x7f5148111f40 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 0x7f514811bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 0x7f514811bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33382/0 (socket says 172.21.15.31:33382) 2024-09-21T06:23:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.231+0000 7f514659c640 1 -- 172.21.15.31:0/2825189234 learned_addr learned my addr 172.21.15.31:0/2825189234 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f5145d9b640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5148111f40 0x7f514811c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f514659c640 1 -- 172.21.15.31:0/2825189234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5148111f40 msgr2=0x7f514811c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f514659c640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5148111f40 0x7f514811c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f514659c640 1 -- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514811cc30 msgr2=0x7f51481163b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f514659c640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514811cc30 0x7f51481163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f514659c640 1 -- 172.21.15.31:0/2825189234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5148116bd0 con 0x7f514810a9e0 2024-09-21T06:23:32.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.232+0000 7f514659c640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 0x7f514811bfd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5140000f80 tx=0x7f5140030b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.238+0000 7f51377fe640 1 -- 172.21.15.31:0/2825189234 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51400319e0 con 0x7f514810a9e0 2024-09-21T06:23:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.238+0000 7f51377fe640 1 -- 172.21.15.31:0/2825189234 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f514002d070 con 0x7f514810a9e0 2024-09-21T06:23:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.238+0000 7f51377fe640 1 -- 172.21.15.31:0/2825189234 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514003b930 con 0x7f514810a9e0 2024-09-21T06:23:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.240+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5148116e60 con 0x7f514810a9e0 2024-09-21T06:23:32.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.240+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51481be370 con 0x7f514810a9e0 2024-09-21T06:23:32.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.247+0000 7f5152fe7740 1 Processor -- start 2024-09-21T06:23:32.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 -- start start 2024-09-21T06:23:32.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 0x55f9628f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f9628f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96252d390 0x55f9628f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f96286be50 con 0x55f96254e070 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f96285b280 con 0x55f9625251b0 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.251+0000 7f5152fe7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f96285f090 con 0x55f96252d390 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 0x55f9628f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 0x55f9628f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52420/0 (socket says 172.21.15.31:52420) 2024-09-21T06:23:32.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 -- 172.21.15.31:0/1410001097 learned_addr learned my addr 172.21.15.31:0/1410001097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 -- 172.21.15.31:0/1410001097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96252d390 msgr2=0x55f9628f5cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96252d390 0x55f9628f5cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 -- 172.21.15.31:0/1410001097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 msgr2=0x55f9628f57b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f9628f57b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.252+0000 7f5146d9d640 1 -- 172.21.15.31:0/1410001097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9625b9650 con 0x55f96254e070 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.253+0000 7f5146d9d640 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 0x55f9628f5270 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5138009c40 tx=0x7f513800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc74c7e43c3a5df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.253+0000 7f51377fe640 1 -- 172.21.15.31:0/2825189234 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5140045c70 con 0x7f514810a9e0 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.254+0000 7f51377fe640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51300b6800 0x7f51300b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.254+0000 7f51377fe640 1 -- 172.21.15.31:0/2825189234 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5140108100 con 0x7f514810a9e0 2024-09-21T06:23:32.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.255+0000 7f514659c640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51300b6800 0x7f51300b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.255+0000 7f5134ff9640 1 -- 172.21.15.31:0/1410001097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138005020 con 0x55f96254e070 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.256+0000 7f5134ff9640 1 -- 172.21.15.31:0/1410001097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51380047a0 con 0x55f96254e070 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.257+0000 7f5152fe7740 1 -- 172.21.15.31:0/1410001097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 msgr2=0x55f9628f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.257+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 0x55f9628f5270 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5138009c40 tx=0x7f513800e720 comp rx=0 tx=0).stop 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.258+0000 7f5152fe7740 1 -- 172.21.15.31:0/1410001097 shutdown_connections 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.258+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96252d390 0x55f9628f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.258+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f9628f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.258+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1410001097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96254e070 0x55f9628f5270 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.258+0000 7f5152fe7740 1 -- 172.21.15.31:0/1410001097 >> 172.21.15.31:0/1410001097 conn(0x55f962542460 msgr2=0x55f9628f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.258+0000 7f5152fe7740 1 -- 172.21.15.31:0/1410001097 shutdown_connections 2024-09-21T06:23:32.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.259+0000 7f5152fe7740 1 -- 172.21.15.31:0/1410001097 wait complete. 2024-09-21T06:23:32.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.259+0000 7f514659c640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51300b6800 0x7f51300b8d10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55f9628f3ef0 tx=0x7f514002a8f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.259+0000 7f5152fe7740 1 -- start start 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.259+0000 7f5152fe7740 1 Processor -- start 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f9629008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96252d390 0x55f962900e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f96290ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f9626bc700 con 0x55f9625251b0 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9627f6110 con 0x55f96252d390 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f962903aa0 con 0x55f96254e070 2024-09-21T06:23:32.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.260+0000 7f5152fe7740 10 client.0 _ll_register_callbacks cb 0x55f96280f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.264+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f96290ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.264+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f9629008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.264+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f9629008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52436/0 (socket says 172.21.15.31:52436) 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.264+0000 7f5146d9d640 1 -- 172.21.15.31:0/1943039012 learned_addr learned my addr 172.21.15.31:0/1943039012 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.265+0000 7f5145d9b640 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96252d390 0x55f962900e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.265+0000 7f5146d9d640 1 -- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96252d390 msgr2=0x55f962900e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.265+0000 7f5146d9d640 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96252d390 0x55f962900e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.265+0000 7f5146d9d640 1 -- 172.21.15.31:0/1943039012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 msgr2=0x55f9629008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.265+0000 7f5146d9d640 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f9629008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.266+0000 7f5146d9d640 1 -- 172.21.15.31:0/1943039012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f962823c40 con 0x55f96254e070 2024-09-21T06:23:32.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.266+0000 7f5146d9d640 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f96290ce10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5138009a50 tx=0x7f513800eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a324bd9ef7551c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.266+0000 7f512dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:23:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.268+0000 7f512dffb640 1 -- 172.21.15.31:0/1943039012 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513802b060 con 0x55f96254e070 2024-09-21T06:23:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.268+0000 7f512dffb640 1 -- 172.21.15.31:0/1943039012 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5138005250 con 0x55f96254e070 2024-09-21T06:23:32.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.268+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51380047a0 con 0x55f96254e070 2024-09-21T06:23:32.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.273+0000 7f512dffb640 1 -- 172.21.15.31:0/1943039012 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5138005450 con 0x55f96254e070 2024-09-21T06:23:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.273+0000 7f5152fe7740 10 client.27691 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:32.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.273+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f962931800 con 0x55f96254e070 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.279+0000 7f512dffb640 1 -- 172.21.15.31:0/1943039012 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51380047a0 con 0x55f96254e070 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f512dffb640 1 -- 172.21.15.31:0/1943039012 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5138020030 con 0x55f96254e070 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f5152fe7740 10 client.27691 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f5152fe7740 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f5152fe7740 20 client.27691 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96250c9f0 0x55f962934bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f5152fe7740 4 client.27691 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:32.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.280+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96286dda0 con 0x55f96250c9f0 2024-09-21T06:23:32.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.283+0000 7f5145d9b640 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96250c9f0 0x55f962934bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.290+0000 7f5145d9b640 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96250c9f0 0x55f962934bc0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867fed2bd75a2694 server_cookie=1d769aa6b56c8926 in_seq=0 out_seq=0 2024-09-21T06:23:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.291+0000 7f512dffb640 10 client.27691 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.292+0000 7f512dffb640 1 -- 172.21.15.31:0/1943039012 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f96286dda0 con 0x55f96250c9f0 2024-09-21T06:23:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.292+0000 7f512dffb640 10 client.27691 handle_command_reply: tid=0 2024-09-21T06:23:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 client.27691 shutdown 2024-09-21T06:23:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 msgr2=0x55f96290ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f96290ce10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5138009a50 tx=0x7f513800eed0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 shutdown_connections 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96250c9f0 0x55f962934bc0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f96290ce10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96252d390 0x55f962900e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 --2- 172.21.15.31:0/1943039012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f9629008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 >> 172.21.15.31:0/1943039012 conn(0x55f962542460 msgr2=0x55f9628f8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f512dffb640 0 client.27691 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 shutdown_connections 2024-09-21T06:23:32.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.294+0000 7f5152fe7740 1 -- 172.21.15.31:0/1943039012 wait complete. 2024-09-21T06:23:32.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.295+0000 7f5152fe7740 20 client.27691 trim_cache size 0 max 16384 2024-09-21T06:23:32.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.337+0000 7f5152fe7740 1 Processor -- start 2024-09-21T06:23:32.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.337+0000 7f5152fe7740 1 -- start start 2024-09-21T06:23:32.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.338+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f962900db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.338+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 0x55f96290cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.338+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f962900db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.338+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f962900db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33406/0 (socket says 172.21.15.31:33406) 2024-09-21T06:23:32.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.339+0000 7f5145d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 0x55f96290cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.339+0000 7f5145d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 0x55f96290cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52446/0 (socket says 172.21.15.31:52446) 2024-09-21T06:23:32.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.339+0000 7f5146d9d640 1 -- 172.21.15.31:0/2980966053 learned_addr learned my addr 172.21.15.31:0/2980966053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.339+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f9628fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.340+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5148117590 con 0x55f96252d390 2024-09-21T06:23:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.340+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f514807e4b0 con 0x55f9625251b0 2024-09-21T06:23:32.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.340+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f514807c2c0 con 0x55f96254e070 2024-09-21T06:23:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.341+0000 7f5146d9d640 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f9628fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.341+0000 7f5145d9b640 1 -- 172.21.15.31:0/2980966053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 msgr2=0x55f9628fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.341+0000 7f5145d9b640 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f9628fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.341+0000 7f5145d9b640 1 -- 172.21.15.31:0/2980966053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 msgr2=0x55f962900db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.341+0000 7f5145d9b640 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f962900db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.341+0000 7f5145d9b640 1 -- 172.21.15.31:0/2980966053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9628ff850 con 0x55f96252d390 2024-09-21T06:23:32.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.342+0000 7f5146d9d640 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f9628fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:32.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.343+0000 7f5145d9b640 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 0x55f96290cd80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55f962903aa0 tx=0x7f513c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4788277159c05d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.343+0000 7f512ffff640 1 -- 172.21.15.31:0/2980966053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c01d070 con 0x55f96252d390 2024-09-21T06:23:32.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.344+0000 7f512ffff640 1 -- 172.21.15.31:0/2980966053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513c0054d0 con 0x55f96252d390 2024-09-21T06:23:32.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 msgr2=0x55f96290cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 0x55f96290cd80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55f962903aa0 tx=0x7f513c005c90 comp rx=0 tx=0).stop 2024-09-21T06:23:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 shutdown_connections 2024-09-21T06:23:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f96254e070 0x55f9628fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f96252d390 0x55f96290cd80 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2980966053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9625251b0 0x55f962900db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.345+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 >> 172.21.15.31:0/2980966053 conn(0x55f962542460 msgr2=0x55f9629006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.349+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 shutdown_connections 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.349+0000 7f5152fe7740 1 -- 172.21.15.31:0/2980966053 wait complete. 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.349+0000 7f5152fe7740 1 -- start start 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.349+0000 7f5152fe7740 1 Processor -- start 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f962938510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96254e070 0x55f962935a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 0x55f962936120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f514807c970 con 0x55f9625251b0 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f514807dde0 con 0x55f96254e070 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f514807d040 con 0x55f9624e1da0 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.350+0000 7f5152fe7740 10 client.0 _ll_register_callbacks cb 0x7f5138009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.351+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 0x55f962936120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.351+0000 7f5145d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96254e070 0x55f962935a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.351+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 0x55f962936120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38538/0 (socket says 172.21.15.31:38538) 2024-09-21T06:23:32.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.351+0000 7f5146d9d640 1 -- 172.21.15.31:0/2744411750 learned_addr learned my addr 172.21.15.31:0/2744411750 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:32.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.351+0000 7f5145d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96254e070 0x55f962935a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33410/0 (socket says 172.21.15.31:33410) 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.352+0000 7f5146d9d640 1 -- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96254e070 msgr2=0x55f962935a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.352+0000 7f5146d9d640 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96254e070 0x55f962935a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.352+0000 7f5146d9d640 1 -- 172.21.15.31:0/2744411750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 msgr2=0x55f962938510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.352+0000 7f5146d9d640 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f962938510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.352+0000 7f5146d9d640 1 -- 172.21.15.31:0/2744411750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f962936890 con 0x55f9624e1da0 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.353+0000 7f5146d9d640 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 0x55f962936120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5138009c40 tx=0x7f5138004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e49146f6695221b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:32.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.353+0000 7f512dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:23:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.353+0000 7f512dffb640 1 -- 172.21.15.31:0/2744411750 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513802b070 con 0x55f9624e1da0 2024-09-21T06:23:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.354+0000 7f512dffb640 1 -- 172.21.15.31:0/2744411750 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5138005250 con 0x55f9624e1da0 2024-09-21T06:23:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.354+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f962869850 con 0x55f9624e1da0 2024-09-21T06:23:32.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.355+0000 7f512dffb640 1 -- 172.21.15.31:0/2744411750 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5138035080 con 0x55f9624e1da0 2024-09-21T06:23:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.355+0000 7f5152fe7740 10 client.27697 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:32.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.355+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f96299f460 con 0x55f9624e1da0 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.365+0000 7f512dffb640 1 -- 172.21.15.31:0/2744411750 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f962869850 con 0x55f9624e1da0 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.366+0000 7f512dffb640 1 -- 172.21.15.31:0/2744411750 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5138020040 con 0x55f9624e1da0 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.367+0000 7f5152fe7740 10 client.27697 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.367+0000 7f5152fe7740 10 client.27697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.367+0000 7f5152fe7740 20 client.27697 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.367+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96299fa60 0x55f9629a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.367+0000 7f5152fe7740 4 client.27697 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:23:32.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.367+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f962866c80 con 0x55f96299fa60 2024-09-21T06:23:32.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.368+0000 7f5145d9b640 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96299fa60 0x55f9629a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.370+0000 7f5145d9b640 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96299fa60 0x55f9629a1e80 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab146358e2e683e server_cookie=9d715d4f0586d2e0 in_seq=0 out_seq=0 2024-09-21T06:23:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.370+0000 7f512dffb640 10 client.27697 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:32.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.371+0000 7f512dffb640 1 -- 172.21.15.31:0/2744411750 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8042 (crc 0 0 0) 0x55f962866c80 con 0x55f96299fa60 2024-09-21T06:23:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.371+0000 7f512dffb640 10 client.27697 handle_command_reply: tid=0 2024-09-21T06:23:32.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.371+0000 7f5152fe7740 1 client.27697 shutdown 2024-09-21T06:23:32.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.375+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 msgr2=0x55f962936120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.375+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 0x55f962936120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5138009c40 tx=0x7f5138004d10 comp rx=0 tx=0).stop 2024-09-21T06:23:32.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.376+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 shutdown_connections 2024-09-21T06:23:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.376+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f96299fa60 0x55f9629a1e80 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.376+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9624e1da0 0x55f962936120 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.376+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f96254e070 0x55f962935a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.376+0000 7f5152fe7740 1 --2- 172.21.15.31:0/2744411750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9625251b0 0x55f962938510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.376+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 >> 172.21.15.31:0/2744411750 conn(0x55f962542460 msgr2=0x55f9628f6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.378+0000 7f512dffb640 0 client.27697 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.378+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 shutdown_connections 2024-09-21T06:23:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.378+0000 7f5152fe7740 1 -- 172.21.15.31:0/2744411750 wait complete. 2024-09-21T06:23:32.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.378+0000 7f5152fe7740 20 client.27697 trim_cache size 0 max 16384 2024-09-21T06:23:32.381 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:23:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:23:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:23:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:23:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:23:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:23:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.387+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51300b6800 msgr2=0x7f51300b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.387+0000 7f514ca73640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51300b6800 0x7f51300b8d10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55f9628f3ef0 tx=0x7f514002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.387+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 msgr2=0x7f514811bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.387+0000 7f514ca73640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 0x7f514811bfd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5140000f80 tx=0x7f5140030b70 comp rx=0 tx=0).stop 2024-09-21T06:23:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 shutdown_connections 2024-09-21T06:23:32.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f51300b6800 0x7f51300b8d10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514811cc30 0x7f51481163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5148111f40 0x7f514811c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 --2- 172.21.15.31:0/2825189234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f514810a9e0 0x7f514811bfd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:32.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 >> 172.21.15.31:0/2825189234 conn(0x7f5148070ec0 msgr2=0x7f5148073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:32.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 shutdown_connections 2024-09-21T06:23:32.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:32.391+0000 7f514ca73640 1 -- 172.21.15.31:0/2825189234 wait complete. 2024-09-21T06:23:32.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:32 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:32.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:32 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:23:32.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:32 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:32.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:32 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:23:32.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:32 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:32.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:32 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:23:33.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:23:33.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:23:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:33 smithi031 ceph-mon[31237]: pgmap v691: 97 pgs: 97 active+clean; 4.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-09-21T06:23:33.739 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:33 smithi110 ceph-mon[34550]: pgmap v691: 97 pgs: 97 active+clean; 4.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-09-21T06:23:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:33 smithi107 ceph-mon[34380]: pgmap v691: 97 pgs: 97 active+clean; 4.3 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 16 MiB/s wr, 1.15k op/s 2024-09-21T06:23:34.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.224+0000 7fa62a34f640 1 -- 172.21.15.31:0/571373258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 msgr2=0x7fa62406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.224+0000 7fa62a34f640 1 --2- 172.21.15.31:0/571373258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 0x7fa62406ae60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009ca0 tx=0x7fa61c02a8f0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.227+0000 7fa62a34f640 1 -- 172.21.15.31:0/571373258 shutdown_connections 2024-09-21T06:23:34.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.227+0000 7fa62a34f640 1 --2- 172.21.15.31:0/571373258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 0x7fa62406ae60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.227+0000 7fa62a34f640 1 --2- 172.21.15.31:0/571373258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 0x7fa6241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.227+0000 7fa62a34f640 1 --2- 172.21.15.31:0/571373258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 0x7fa624102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.227+0000 7fa62a34f640 1 -- 172.21.15.31:0/571373258 >> 172.21.15.31:0/571373258 conn(0x7fa624079730 msgr2=0x7fa624079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.227+0000 7fa62a34f640 1 -- 172.21.15.31:0/571373258 shutdown_connections 2024-09-21T06:23:34.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.228+0000 7fa62a34f640 1 -- 172.21.15.31:0/571373258 wait complete. 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 Processor -- start 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 -- start start 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 0x7fa62410b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 0x7fa62410bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 0x7fa6241146c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa62406cf60 con 0x7fa624108cf0 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa62406cde0 con 0x7fa624102a60 2024-09-21T06:23:34.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.229+0000 7fa62a34f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa62406d0e0 con 0x7fa6241083a0 2024-09-21T06:23:34.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.231+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 0x7fa62410bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.231+0000 7fa6237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 0x7fa62410bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38560/0 (socket says 172.21.15.31:38560) 2024-09-21T06:23:34.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.231+0000 7fa6237fe640 1 -- 172.21.15.31:0/2563032912 learned_addr learned my addr 172.21.15.31:0/2563032912 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:34.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.231+0000 7fa623fff640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 0x7fa62410b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.232+0000 7fa6288ec640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 0x7fa6241146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.236+0000 7fa623fff640 1 -- 172.21.15.31:0/2563032912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 msgr2=0x7fa62410bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.236+0000 7fa623fff640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 0x7fa62410bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.236+0000 7fa623fff640 1 -- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 msgr2=0x7fa6241146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.236+0000 7fa623fff640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 0x7fa6241146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.236+0000 7fa623fff640 1 -- 172.21.15.31:0/2563032912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624114df0 con 0x7fa624102a60 2024-09-21T06:23:34.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.242+0000 7fa623fff640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 0x7fa62410b820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c970 tx=0x7fa61400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.243+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614007bb0 con 0x7fa624102a60 2024-09-21T06:23:34.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.243+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6241150e0 con 0x7fa624102a60 2024-09-21T06:23:34.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.243+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa624115670 con 0x7fa624102a60 2024-09-21T06:23:34.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.243+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa614007d50 con 0x7fa624102a60 2024-09-21T06:23:34.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.243+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614004e40 con 0x7fa624102a60 2024-09-21T06:23:34.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.246+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa610005290 con 0x7fa624102a60 2024-09-21T06:23:34.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.250+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa6140046c0 con 0x7fa624102a60 2024-09-21T06:23:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.252+0000 7fa6217fa640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa60c0b67b0 0x7fa60c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:34.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.252+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7fa6140e7180 con 0x7fa624102a60 2024-09-21T06:23:34.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.256+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa614014030 con 0x7fa624102a60 2024-09-21T06:23:34.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.257+0000 7fa6237fe640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa60c0b67b0 0x7fa60c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:34.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.267+0000 7fa6237fe640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa60c0b67b0 0x7fa60c0b8cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa624108110 tx=0x7fa61800a300 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:34.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.611+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa610005580 con 0x7fa624102a60 2024-09-21T06:23:34.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.612+0000 7fa6217fa640 1 -- 172.21.15.31:0/2563032912 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7fa6140c6e60 con 0x7fa624102a60 2024-09-21T06:23:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:23:34.615 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:23:34.615 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:23:34.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.615+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa60c0b67b0 msgr2=0x7fa60c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.615+0000 7fa62a34f640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa60c0b67b0 0x7fa60c0b8cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa624108110 tx=0x7fa61800a300 comp rx=0 tx=0).stop 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.615+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 msgr2=0x7fa62410b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.615+0000 7fa62a34f640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 0x7fa62410b820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c970 tx=0x7fa61400ce30 comp rx=0 tx=0).stop 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 shutdown_connections 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa60c0b67b0 0x7fa60c0b8cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa624108cf0 0x7fa6241146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6241083a0 0x7fa62410bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 --2- 172.21.15.31:0/2563032912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa624102a60 0x7fa62410b820 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 >> 172.21.15.31:0/2563032912 conn(0x7fa624079730 msgr2=0x7fa624101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 shutdown_connections 2024-09-21T06:23:34.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:34.616+0000 7fa62a34f640 1 -- 172.21.15.31:0/2563032912 wait complete. 2024-09-21T06:23:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 715.0 s, 1325.6 tps, lat 75.616 ms stddev 120.283 2024-09-21T06:23:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:35 smithi110 ceph-mon[34550]: pgmap v692: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 9.5 MiB/s wr, 1.01k op/s 2024-09-21T06:23:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:35 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2563032912' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:35 smithi107 ceph-mon[34380]: pgmap v692: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 9.5 MiB/s wr, 1.01k op/s 2024-09-21T06:23:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:35 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2563032912' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:35.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:35 smithi031 ceph-mon[31237]: pgmap v692: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 9.5 MiB/s wr, 1.01k op/s 2024-09-21T06:23:35.853 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:35 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2563032912' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:23:35.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:23:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 -- 172.21.15.31:0/2275153215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc1083e0 msgr2=0x7efccc1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 --2- 172.21.15.31:0/2275153215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc1083e0 0x7efccc1087f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009860 tx=0x7efcc002a340 comp rx=0 tx=0).stop 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 -- 172.21.15.31:0/2275153215 shutdown_connections 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 --2- 172.21.15.31:0/2275153215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc108d30 0x7efccc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 --2- 172.21.15.31:0/2275153215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc1083e0 0x7efccc1087f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 --2- 172.21.15.31:0/2275153215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc102aa0 0x7efccc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 -- 172.21.15.31:0/2275153215 >> 172.21.15.31:0/2275153215 conn(0x7efccc0797e0 msgr2=0x7efccc079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.899+0000 7efcd1119640 1 -- 172.21.15.31:0/2275153215 shutdown_connections 2024-09-21T06:23:36.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.900+0000 7efcd1119640 1 -- 172.21.15.31:0/2275153215 wait complete. 2024-09-21T06:23:36.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.901+0000 7efcd1119640 1 Processor -- start 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.901+0000 7efcd1119640 1 -- start start 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.901+0000 7efcd1119640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc102aa0 0x7efccc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.901+0000 7efcd1119640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 0x7efccc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcd1119640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc108d30 0x7efccc1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcd1119640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efccc113a70 con 0x7efccc108d30 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcd1119640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1138f0 con 0x7efccc1083e0 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcd1119640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efccc113bf0 con 0x7efccc102aa0 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 0x7efccc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efccad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc102aa0 0x7efccc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 0x7efccc19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33470/0 (socket says 172.21.15.31:33470) 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 -- 172.21.15.31:0/1137111436 learned_addr learned my addr 172.21.15.31:0/1137111436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efccb59e640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc108d30 0x7efccc1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 -- 172.21.15.31:0/1137111436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc102aa0 msgr2=0x7efccc19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc102aa0 0x7efccc19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 -- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc108d30 msgr2=0x7efccc1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc108d30 0x7efccc1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 -- 172.21.15.31:0/1137111436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc1a2690 con 0x7efccc1083e0 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efccad9d640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc102aa0 0x7efccc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.902+0000 7efcca59c640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 0x7efccc19ccc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efcc0035040 tx=0x7efcc0032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.903+0000 7efcb3fff640 1 -- 172.21.15.31:0/1137111436 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0037040 con 0x7efccc1083e0 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.903+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc1a2920 con 0x7efccc1083e0 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.903+0000 7efcb3fff640 1 -- 172.21.15.31:0/1137111436 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc0041070 con 0x7efccc1083e0 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.903+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc1a90e0 con 0x7efccc1083e0 2024-09-21T06:23:36.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.903+0000 7efcb3fff640 1 -- 172.21.15.31:0/1137111436 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003850 con 0x7efccc1083e0 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.906+0000 7efcd768d740 1 Processor -- start 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.906+0000 7efcd768d740 1 -- start start 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcb3fff640 1 -- 172.21.15.31:0/1137111436 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efcc00039f0 con 0x7efccc1083e0 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcd768d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 0x55e9b5c686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcd768d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58981b0 0x55e9b5c68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcd768d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c69120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcd768d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5bdeea0 con 0x55e9b58981b0 2024-09-21T06:23:36.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcd768d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5bce2d0 con 0x55e9b58a0390 2024-09-21T06:23:36.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcd768d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5bd20e0 con 0x55e9b58c1070 2024-09-21T06:23:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efccb59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 0x55e9b5c686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efccb59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 0x55e9b5c686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38588/0 (socket says 172.21.15.31:38588) 2024-09-21T06:23:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efccb59e640 1 -- 172.21.15.31:0/3451675576 learned_addr learned my addr 172.21.15.31:0/3451675576 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.907+0000 7efcca59c640 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c69120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccad9d640 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58981b0 0x55e9b5c68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccb59e640 1 -- 172.21.15.31:0/3451675576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 msgr2=0x55e9b5c69120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccb59e640 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c69120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccb59e640 1 -- 172.21.15.31:0/3451675576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58981b0 msgr2=0x55e9b5c68be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccb59e640 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58981b0 0x55e9b5c68be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccb59e640 1 -- 172.21.15.31:0/3451675576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b592c650 con 0x55e9b58c1070 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.908+0000 7efccad9d640 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58981b0 0x55e9b5c68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.909+0000 7efccb59e640 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 0x55e9b5c686a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e480 tx=0x7efcc40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f186e765dd42ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.909+0000 7efcb17fa640 1 -- 172.21.15.31:0/3451675576 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4003970 con 0x55e9b58c1070 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.909+0000 7efcb17fa640 1 -- 172.21.15.31:0/3451675576 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc400d7d0 con 0x55e9b58c1070 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.909+0000 7efcb17fa640 1 -- 172.21.15.31:0/3451675576 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4004350 con 0x55e9b58c1070 2024-09-21T06:23:36.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcb3fff640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcb40b6680 0x7efcb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efccad9d640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcb40b6680 0x7efcb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcb3fff640 1 -- 172.21.15.31:0/1137111436 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7efcc0107120 con 0x7efccc1083e0 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 -- 172.21.15.31:0/3451675576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 msgr2=0x55e9b5c686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 0x55e9b5c686a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e480 tx=0x7efcc40095c0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 -- 172.21.15.31:0/3451675576 shutdown_connections 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c69120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58981b0 0x55e9b5c68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 --2- 172.21.15.31:0/3451675576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58c1070 0x55e9b5c686a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.910+0000 7efcd768d740 1 -- 172.21.15.31:0/3451675576 >> 172.21.15.31:0/3451675576 conn(0x55e9b58b5460 msgr2=0x55e9b5c66590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.911+0000 7efccad9d640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcb40b6680 0x7efcb40b8b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55e9b5c67320 tx=0x7efcbc00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.911+0000 7efcd768d740 1 -- 172.21.15.31:0/3451675576 shutdown_connections 2024-09-21T06:23:36.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.912+0000 7efcd768d740 1 -- 172.21.15.31:0/3451675576 wait complete. 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.912+0000 7efcd768d740 1 -- start start 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.912+0000 7efcd768d740 1 Processor -- start 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5a2f700 con 0x55e9b58c1070 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5b69160 con 0x55e9b58a0390 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e9b5c76e80 con 0x55e9b58981b0 2024-09-21T06:23:36.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcd768d740 10 client.0 _ll_register_callbacks cb 0x55e9b5b829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcca59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c73ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:38602/0 (socket says 172.21.15.31:38602) 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efcca59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c74220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33486/0 (socket says 172.21.15.31:33486) 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 -- 172.21.15.31:0/3541078204 learned_addr learned my addr 172.21.15.31:0/3541078204 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccb59e640 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 -- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 msgr2=0x55e9b5c74220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c74220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 -- 172.21.15.31:0/3541078204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 msgr2=0x55e9b5c747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.913+0000 7efccad9d640 1 -- 172.21.15.31:0/3541078204 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b5b96c90 con 0x55e9b58981b0 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.914+0000 7efccad9d640 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c73ce0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efcbc012b20 tx=0x7efcbc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f0e5f75b523668 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:36.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.914+0000 7efcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:23:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.914+0000 7efcaeff5640 1 -- 172.21.15.31:0/3541078204 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00fe20 con 0x55e9b58981b0 2024-09-21T06:23:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.914+0000 7efcaeff5640 1 -- 172.21.15.31:0/3541078204 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7efcbc004020 con 0x55e9b58981b0 2024-09-21T06:23:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.914+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc400d7d0 con 0x55e9b58981b0 2024-09-21T06:23:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.915+0000 7efcaeff5640 1 -- 172.21.15.31:0/3541078204 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efcbc004220 con 0x55e9b58981b0 2024-09-21T06:23:36.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.915+0000 7efcd768d740 10 client.27712 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.915+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e9b5c82b40 con 0x55e9b58981b0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcaeff5640 1 -- 172.21.15.31:0/3541078204 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc400d7d0 con 0x55e9b58981b0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcaeff5640 1 -- 172.21.15.31:0/3541078204 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efcbc031420 con 0x55e9b58981b0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcd768d740 10 client.27712 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcd768d740 10 client.27712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcd768d740 20 client.27712 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcd768d740 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b587f9f0 0x55e9b5c86110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcd768d740 4 client.27712 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b5be0df0 con 0x55e9b587f9f0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcca59c640 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b587f9f0 0x55e9b5c86110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.916+0000 7efcaeff5640 10 client.27712 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.917+0000 7efcca59c640 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b587f9f0 0x55e9b5c86110 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f7fd21d4379a00e server_cookie=fcbbfb9440178bce in_seq=0 out_seq=0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.917+0000 7efcaeff5640 1 -- 172.21.15.31:0/3541078204 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9b5be0df0 con 0x55e9b587f9f0 2024-09-21T06:23:36.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.917+0000 7efcaeff5640 10 client.27712 handle_command_reply: tid=0 2024-09-21T06:23:36.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.917+0000 7efcd768d740 1 client.27712 shutdown 2024-09-21T06:23:36.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.918+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 msgr2=0x55e9b5c73ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.918+0000 7efcd768d740 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c73ce0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efcbc012b20 tx=0x7efcbc0046c0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.918+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 shutdown_connections 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.918+0000 7efcd768d740 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b587f9f0 0x55e9b5c86110 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.918+0000 7efcd768d740 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.918+0000 7efcd768d740 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58a0390 0x55e9b5c74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.919+0000 7efcaeff5640 0 client.27712 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.919+0000 7efcd768d740 1 --2- 172.21.15.31:0/3541078204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c73ce0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.919+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 >> 172.21.15.31:0/3541078204 conn(0x55e9b58b5460 msgr2=0x55e9b5c6b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.919+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 shutdown_connections 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.919+0000 7efcd768d740 1 -- 172.21.15.31:0/3541078204 wait complete. 2024-09-21T06:23:36.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.919+0000 7efcd768d740 20 client.27712 trim_cache size 0 max 16384 2024-09-21T06:23:36.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.937+0000 7efcd768d740 1 Processor -- start 2024-09-21T06:23:36.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.937+0000 7efcd768d740 1 -- start start 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcd768d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58981b0 0x55e9b5c6a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcd768d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58a0390 0x55e9b5c75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcd768d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c65bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcd768d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1afa70 con 0x55e9b58c1070 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcd768d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1b22d0 con 0x55e9b58981b0 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcd768d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1b00c0 con 0x55e9b58a0390 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efccad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58981b0 0x55e9b5c6a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcca59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58a0390 0x55e9b5c75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efccad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58981b0 0x55e9b5c6a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:33500/0 (socket says 172.21.15.31:33500) 2024-09-21T06:23:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efccad9d640 1 -- 172.21.15.31:0/4213346475 learned_addr learned my addr 172.21.15.31:0/4213346475 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efccb59e640 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c65bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcca59c640 1 -- 172.21.15.31:0/4213346475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58981b0 msgr2=0x55e9b5c6a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcca59c640 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58981b0 0x55e9b5c6a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.938+0000 7efcca59c640 1 -- 172.21.15.31:0/4213346475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 msgr2=0x55e9b5c65bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.939+0000 7efcca59c640 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c65bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.939+0000 7efcca59c640 1 -- 172.21.15.31:0/4213346475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b5c80070 con 0x55e9b58a0390 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.939+0000 7efccb59e640 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.939+0000 7efcca59c640 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58a0390 0x55e9b5c75910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efcc0104880 tx=0x7efcc00eef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c178a1b2a8835f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.939+0000 7efcb0ff9640 1 -- 172.21.15.31:0/4213346475 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00ccd50 con 0x55e9b58a0390 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.940+0000 7efcb0ff9640 1 -- 172.21.15.31:0/4213346475 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc00328f0 con 0x55e9b58a0390 2024-09-21T06:23:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.940+0000 7efcb0ff9640 1 -- 172.21.15.31:0/4213346475 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00cc6f0 con 0x55e9b58a0390 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 -- 172.21.15.31:0/4213346475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58a0390 msgr2=0x55e9b5c75910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58a0390 0x55e9b5c75910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efcc0104880 tx=0x7efcc00eef90 comp rx=0 tx=0).stop 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 -- 172.21.15.31:0/4213346475 shutdown_connections 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58c1070 0x55e9b5c65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58a0390 0x55e9b5c75910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 --2- 172.21.15.31:0/4213346475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58981b0 0x55e9b5c6a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 -- 172.21.15.31:0/4213346475 >> 172.21.15.31:0/4213346475 conn(0x55e9b58b5460 msgr2=0x55e9b5c75200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 -- 172.21.15.31:0/4213346475 shutdown_connections 2024-09-21T06:23:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.941+0000 7efcd768d740 1 -- 172.21.15.31:0/4213346475 wait complete. 2024-09-21T06:23:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.942+0000 7efcd768d740 1 -- start start 2024-09-21T06:23:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.942+0000 7efcd768d740 1 Processor -- start 2024-09-21T06:23:36.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcd768d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c78270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcd768d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 0x55e9b5c787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcd768d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58c1070 0x55e9b5c78cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 0x55e9b5c787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcd768d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1b0790 con 0x55e9b58a0390 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcd768d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1b1c00 con 0x55e9b58c1070 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcd768d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efccc1b0e60 con 0x55e9b58981b0 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efcca59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 0x55e9b5c787b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52512/0 (socket says 172.21.15.31:52512) 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 -- 172.21.15.31:0/3735035035 learned_addr learned my addr 172.21.15.31:0/3735035035 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcd768d740 10 client.0 _ll_register_callbacks cb 0x7efcbc005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.943+0000 7efccb59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58c1070 0x55e9b5c78cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efccad9d640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c78270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 -- 172.21.15.31:0/3735035035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 msgr2=0x55e9b5c78270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c78270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 -- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58c1070 msgr2=0x55e9b5c78cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58c1070 0x55e9b5c78cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 -- 172.21.15.31:0/3735035035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b5cefcd0 con 0x55e9b58a0390 2024-09-21T06:23:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efccb59e640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58c1070 0x55e9b5c78cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcca59c640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 0x55e9b5c787b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efcc0030a80 tx=0x7efcc002af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94587780a6bbb05a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcaeff5640 1 -- 172.21.15.31:0/3735035035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00cc8d0 con 0x55e9b58a0390 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.944+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc00328f0 con 0x55e9b58a0390 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcaeff5640 1 -- 172.21.15.31:0/3735035035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7efcc00cc1e0 con 0x55e9b58a0390 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcaeff5640 1 -- 172.21.15.31:0/3735035035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efcc00ccd50 con 0x55e9b58a0390 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 10 client.17877 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e9b5cf0f10 con 0x55e9b58a0390 2024-09-21T06:23:36.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcaeff5640 1 -- 172.21.15.31:0/3735035035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc00328f0 con 0x55e9b58a0390 2024-09-21T06:23:36.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcaeff5640 1 -- 172.21.15.31:0/3735035035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efcc01090e0 con 0x55e9b58a0390 2024-09-21T06:23:36.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 10 client.17877 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:36.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:36.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 20 client.17877 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b5854da0 0x55e9b5cf3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 4 client.17877 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.945+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9b5bdc8a0 con 0x55e9b5854da0 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.946+0000 7efccad9d640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b5854da0 0x55e9b5cf3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.946+0000 7efccad9d640 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b5854da0 0x55e9b5cf3580 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab4f34ab0d0d373 server_cookie=627164467b18bd1a in_seq=0 out_seq=0 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.946+0000 7efcaeff5640 10 client.17877 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.947+0000 7efcaeff5640 1 -- 172.21.15.31:0/3735035035 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9b5bdc8a0 con 0x55e9b5854da0 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.947+0000 7efcaeff5640 10 client.17877 handle_command_reply: tid=0 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.947+0000 7efcd768d740 1 client.17877 shutdown 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 msgr2=0x55e9b5c787b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 0x55e9b5c787b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efcc0030a80 tx=0x7efcc002af80 comp rx=0 tx=0).stop 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 shutdown_connections 2024-09-21T06:23:36.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e9b5854da0 0x55e9b5cf3580 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b58c1070 0x55e9b5c78cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcaeff5640 0 client.17877 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e9b58a0390 0x55e9b5c787b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 --2- 172.21.15.31:0/3735035035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e9b58981b0 0x55e9b5c78270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 >> 172.21.15.31:0/3735035035 conn(0x55e9b58b5460 msgr2=0x55e9b5c69e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 shutdown_connections 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.948+0000 7efcd768d740 1 -- 172.21.15.31:0/3735035035 wait complete. 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd768d740 20 client.17877 trim_cache size 0 max 16384 2024-09-21T06:23:36.952 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:23:36.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcb40b6680 msgr2=0x7efcb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcb40b6680 0x7efcb40b8b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55e9b5c67320 tx=0x7efcbc00b410 comp rx=0 tx=0).stop 2024-09-21T06:23:36.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 msgr2=0x7efccc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 0x7efccc19ccc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efcc0035040 tx=0x7efcc0032dc0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 shutdown_connections 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efcb40b6680 0x7efcb40b8b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc108d30 0x7efccc1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efccc1083e0 0x7efccc19ccc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.949+0000 7efcd1119640 1 --2- 172.21.15.31:0/1137111436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efccc102aa0 0x7efccc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.950+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 >> 172.21.15.31:0/1137111436 conn(0x7efccc0797e0 msgr2=0x7efccc110200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.950+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 shutdown_connections 2024-09-21T06:23:36.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:36.950+0000 7efcd1119640 1 -- 172.21.15.31:0/1137111436 wait complete. 2024-09-21T06:23:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:37 smithi110 ceph-mon[34550]: pgmap v693: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-21T06:23:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:37 smithi107 ceph-mon[34380]: pgmap v693: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-21T06:23:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:37 smithi031 ceph-mon[31237]: pgmap v693: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 14 MiB/s wr, 1.49k op/s 2024-09-21T06:23:38.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:23:38.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:23:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:39 smithi110 ceph-mon[34550]: pgmap v694: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-09-21T06:23:39.874 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:39 smithi031 ceph-mon[31237]: pgmap v694: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-09-21T06:23:39.874 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:39 smithi107 ceph-mon[34380]: pgmap v694: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 14 MiB/s wr, 1.48k op/s 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 -- 172.21.15.31:0/2231265790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc108b40 msgr2=0x7f90dc10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 --2- 172.21.15.31:0/2231265790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc108b40 0x7f90dc10f370 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f90d400aca0 tx=0x7f90d402a9f0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 -- 172.21.15.31:0/2231265790 shutdown_connections 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 --2- 172.21.15.31:0/2231265790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc108b40 0x7f90dc10f370 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 --2- 172.21.15.31:0/2231265790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90dc1081f0 0x7f90dc108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 --2- 172.21.15.31:0/2231265790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90dc1028b0 0x7f90dc102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 -- 172.21.15.31:0/2231265790 >> 172.21.15.31:0/2231265790 conn(0x7f90dc0fe320 msgr2=0x7f90dc100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.883+0000 7f90e23a6640 1 -- 172.21.15.31:0/2231265790 shutdown_connections 2024-09-21T06:23:39.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.884+0000 7f90e23a6640 1 -- 172.21.15.31:0/2231265790 wait complete. 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.885+0000 7f90e23a6640 1 Processor -- start 2024-09-21T06:23:39.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.885+0000 7f90e23a6640 1 -- start start 2024-09-21T06:23:39.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.885+0000 7f90e23a6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90dc1028b0 0x7f90dc198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90e23a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 0x7f90dc199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90e23a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90dc108b40 0x7f90dc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90e23a6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc112930 con 0x7f90dc1081f0 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90e23a6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1127b0 con 0x7f90dc108b40 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90e23a6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc112ab0 con 0x7f90dc1028b0 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 0x7f90dc199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 0x7f90dc199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34758/0 (socket says 172.21.15.31:34758) 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 -- 172.21.15.31:0/21779141 learned_addr learned my addr 172.21.15.31:0/21779141 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90e0943640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90dc108b40 0x7f90dc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 -- 172.21.15.31:0/21779141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90dc1028b0 msgr2=0x7f90dc198f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90dc1028b0 0x7f90dc198f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 -- 172.21.15.31:0/21779141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90dc108b40 msgr2=0x7f90dc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.886+0000 7f90db7fe640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90dc108b40 0x7f90dc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.887+0000 7f90db7fe640 1 -- 172.21.15.31:0/21779141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc19ef00 con 0x7f90dc1081f0 2024-09-21T06:23:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.888+0000 7f90db7fe640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 0x7f90dc199530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c970 tx=0x7f90c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.888+0000 7f90d97fa640 1 -- 172.21.15.31:0/21779141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8007bb0 con 0x7f90dc1081f0 2024-09-21T06:23:39.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.888+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc19f1f0 con 0x7f90dc1081f0 2024-09-21T06:23:39.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.888+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc06b820 con 0x7f90dc1081f0 2024-09-21T06:23:39.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.888+0000 7f90d97fa640 1 -- 172.21.15.31:0/21779141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8007d50 con 0x7f90dc1081f0 2024-09-21T06:23:39.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.888+0000 7f90d97fa640 1 -- 172.21.15.31:0/21779141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c801d060 con 0x7f90dc1081f0 2024-09-21T06:23:39.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.892+0000 7f90e891a740 1 Processor -- start 2024-09-21T06:23:39.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.893+0000 7f90e891a740 1 -- start start 2024-09-21T06:23:39.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.893+0000 7f90e891a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2ceaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.894+0000 7f90e0943640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2ceaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.894+0000 7f90e0943640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2ceaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46718/0 (socket says 172.21.15.31:46718) 2024-09-21T06:23:39.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.894+0000 7f90e891a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ceaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.894+0000 7f90e0943640 1 -- 172.21.15.31:0/1120188036 learned_addr learned my addr 172.21.15.31:0/1120188036 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:39.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e891a740 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2ceb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562bc2c5ce60 con 0x562bc291e670 2024-09-21T06:23:39.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562bc2c4c290 con 0x562bc293f350 2024-09-21T06:23:39.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562bc2c500a0 con 0x562bc2916490 2024-09-21T06:23:39.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90d97fa640 1 -- 172.21.15.31:0/21779141 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f90c800f450 con 0x7f90dc1081f0 2024-09-21T06:23:39.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90db7fe640 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2ceb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e0943640 1 -- 172.21.15.31:0/1120188036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 msgr2=0x562bc2ceaf60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:23:39.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e0943640 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ceaf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e0943640 1 -- 172.21.15.31:0/1120188036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 msgr2=0x562bc2ceb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e0943640 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2ceb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.895+0000 7f90e0943640 1 -- 172.21.15.31:0/1120188036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc29aa930 con 0x562bc293f350 2024-09-21T06:23:39.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.896+0000 7f90e0943640 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2ceaa20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000d30 tx=0x7f90d400aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f99e84b6464a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.896+0000 7f90c67fc640 1 -- 172.21.15.31:0/1120188036 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4033030 con 0x562bc293f350 2024-09-21T06:23:39.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.896+0000 7f90c67fc640 1 -- 172.21.15.31:0/1120188036 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d402e870 con 0x562bc293f350 2024-09-21T06:23:39.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.896+0000 7f90c67fc640 1 -- 172.21.15.31:0/1120188036 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4039430 con 0x562bc293f350 2024-09-21T06:23:39.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90d97fa640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90cc0b6780 0x7f90cc0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 msgr2=0x562bc2ceaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2ceaa20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000d30 tx=0x7f90d400aa10 comp rx=0 tx=0).stop 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90d97fa640 1 -- 172.21.15.31:0/21779141 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f90c80e7330 con 0x7f90dc1081f0 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 shutdown_connections 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2ceb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ceaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 --2- 172.21.15.31:0/1120188036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2ceaa20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.897+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 >> 172.21.15.31:0/1120188036 conn(0x562bc2933740 msgr2=0x562bc2ce8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.898+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 shutdown_connections 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.898+0000 7f90e891a740 1 -- 172.21.15.31:0/1120188036 wait complete. 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.898+0000 7f90dbfff640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90cc0b6780 0x7f90cc0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.899+0000 7f90e891a740 1 -- start start 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.899+0000 7f90dbfff640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90cc0b6780 0x7f90cc0b8c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005d00 tx=0x7f90d0005c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.899+0000 7f90e891a740 1 Processor -- start 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.900+0000 7f90e891a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 0x562bc2cf65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.900+0000 7f90e891a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc291e670 0x562bc2cf6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.900+0000 7f90dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 0x562bc2cf65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.900+0000 7f90dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 0x562bc2cf65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34766/0 (socket says 172.21.15.31:34766) 2024-09-21T06:23:39.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.900+0000 7f90dbfff640 1 -- 172.21.15.31:0/4282535929 learned_addr learned my addr 172.21.15.31:0/4282535929 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.900+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d02520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90db7fe640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc291e670 0x562bc2cf6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562bc2aad890 con 0x562bc2916490 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562bc2be6d40 con 0x562bc293f350 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562bc2cf91a0 con 0x562bc291e670 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d02520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e891a740 10 client.0 _ll_register_callbacks cb 0x562bc2c4ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 -- 172.21.15.31:0/4282535929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc291e670 msgr2=0x562bc2cf6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc291e670 0x562bc2cf6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 -- 172.21.15.31:0/4282535929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 msgr2=0x562bc2cf65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 0x562bc2cf65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc2d02a90 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90dbfff640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 0x562bc2cf65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90e0943640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d02520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000b60 tx=0x7f90d402f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95732a90a214a41c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90bb7fe640 1 -- 172.21.15.31:0/4282535929 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4051060 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.901+0000 7f90bb7fe640 1 -- 172.21.15.31:0/4282535929 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f90d4040040 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d402e870 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90bb7fe640 1 -- 172.21.15.31:0/4282535929 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90d4033050 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 10 client.27665 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562bc2d053c0 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90bb7fe640 1 -- 172.21.15.31:0/4282535929 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d402e870 con 0x562bc293f350 2024-09-21T06:23:39.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90bb7fe640 1 -- 172.21.15.31:0/4282535929 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f90d4039a30 con 0x562bc293f350 2024-09-21T06:23:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 10 client.27665 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 10 client.27665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 20 client.27665 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28fdcd0 0x562bc2d08720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.902+0000 7f90e891a740 4 client.27665 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:39.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.903+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bc2c5edb0 con 0x562bc28fdcd0 2024-09-21T06:23:39.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.903+0000 7f90dbfff640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28fdcd0 0x562bc2d08720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.903+0000 7f90dbfff640 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28fdcd0 0x562bc2d08720 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91052adf9073b3f server_cookie=7c105c213be18db0 in_seq=0 out_seq=0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.904+0000 7f90bb7fe640 10 client.27665 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.904+0000 7f90bb7fe640 1 -- 172.21.15.31:0/4282535929 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bc2c5edb0 con 0x562bc28fdcd0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.904+0000 7f90bb7fe640 10 client.27665 handle_command_reply: tid=0 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.904+0000 7f90e891a740 1 client.27665 shutdown 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 msgr2=0x562bc2d02520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d02520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f90d4000b60 tx=0x7f90d402f9c0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 shutdown_connections 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28fdcd0 0x562bc2d08720 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d02520 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc291e670 0x562bc2cf6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90bb7fe640 0 client.27665 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 --2- 172.21.15.31:0/4282535929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc2916490 0x562bc2cf65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 >> 172.21.15.31:0/4282535929 conn(0x562bc2933740 msgr2=0x562bc2ce2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 shutdown_connections 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 1 -- 172.21.15.31:0/4282535929 wait complete. 2024-09-21T06:23:39.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.905+0000 7f90e891a740 20 client.27665 trim_cache size 0 max 16384 2024-09-21T06:23:39.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.929+0000 7f90e891a740 1 Processor -- start 2024-09-21T06:23:39.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.929+0000 7f90e891a740 1 -- start start 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.929+0000 7f90e891a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ce3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.929+0000 7f90e891a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d02490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90e891a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2cf0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90e891a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90c000c7c0 con 0x562bc291e670 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90e891a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90c000f020 con 0x562bc293f350 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90e891a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90c000ce10 con 0x562bc2916490 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d02490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90dbfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ce3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d02490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34782/0 (socket says 172.21.15.31:34782) 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 -- 172.21.15.31:0/2383215308 learned_addr learned my addr 172.21.15.31:0/2383215308 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:39.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90e0943640 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2cf0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 -- 172.21.15.31:0/2383215308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 msgr2=0x562bc2ce3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ce3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 -- 172.21.15.31:0/2383215308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 msgr2=0x562bc2cf0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2cf0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 -- 172.21.15.31:0/2383215308 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bc2cf7480 con 0x562bc291e670 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90dbfff640 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ce3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90db7fe640 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d02490 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f90c80e49d0 tx=0x7f90c80acfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35be3f9063860bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.930+0000 7f90c5ffb640 1 -- 172.21.15.31:0/2383215308 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8004e50 con 0x562bc291e670 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.931+0000 7f90c5ffb640 1 -- 172.21.15.31:0/2383215308 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c80e6040 con 0x562bc291e670 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.931+0000 7f90c5ffb640 1 -- 172.21.15.31:0/2383215308 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c80b6b50 con 0x562bc291e670 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.931+0000 7f90e891a740 1 -- 172.21.15.31:0/2383215308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 msgr2=0x562bc2d02490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.931+0000 7f90e891a740 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d02490 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f90c80e49d0 tx=0x7f90c80acfc0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 -- 172.21.15.31:0/2383215308 shutdown_connections 2024-09-21T06:23:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2cf0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d02490 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 --2- 172.21.15.31:0/2383215308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2ce3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 -- 172.21.15.31:0/2383215308 >> 172.21.15.31:0/2383215308 conn(0x562bc2933740 msgr2=0x562bc2cefb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 -- 172.21.15.31:0/2383215308 shutdown_connections 2024-09-21T06:23:39.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 -- 172.21.15.31:0/2383215308 wait complete. 2024-09-21T06:23:39.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 -- start start 2024-09-21T06:23:39.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.932+0000 7f90e891a740 1 Processor -- start 2024-09-21T06:23:39.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2d09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d09690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d71e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90c000d4e0 con 0x562bc291e670 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90c000e950 con 0x562bc293f350 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f90c000dbb0 con 0x562bc2916490 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d09690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 10 client.0 _ll_register_callbacks cb 0x7f90d400a960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d09690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34790/0 (socket says 172.21.15.31:34790) 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 -- 172.21.15.31:0/2470174654 learned_addr learned my addr 172.21.15.31:0/2470174654 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90dbfff640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2d09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e0943640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d71e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 -- 172.21.15.31:0/2470174654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 msgr2=0x562bc2d09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2d09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 -- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 msgr2=0x562bc2d71e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:39.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d71e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 -- 172.21.15.31:0/2470174654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bc2d72480 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90db7fe640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d09690 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f90c8007590 tx=0x7f90c80a8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b129216741fe6155 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2470174654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8004e50 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2470174654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f90c8098180 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.933+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c80e6040 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2470174654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90c80acc40 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 10 client.17895 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562bc2d73740 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2470174654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c80e6040 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2470174654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f90c80b2da0 con 0x562bc291e670 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 10 client.17895 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:39.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 20 client.17895 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:39.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28d3080 0x562bc2d75ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:39.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 4 client.17895 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:23:39.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562bc2c5a860 con 0x562bc28d3080 2024-09-21T06:23:39.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.934+0000 7f90dbfff640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28d3080 0x562bc2d75ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:39.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.935+0000 7f90bb7fe640 10 client.17895 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:39.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.935+0000 7f90dbfff640 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28d3080 0x562bc2d75ee0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec59db124d174dc6 server_cookie=63409688bdb50b02 in_seq=0 out_seq=0 2024-09-21T06:23:39.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.937+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2470174654 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562bc2c5a860 con 0x562bc28d3080 2024-09-21T06:23:39.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.937+0000 7f90bb7fe640 10 client.17895 handle_command_reply: tid=0 2024-09-21T06:23:39.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.937+0000 7f90e891a740 1 client.17895 shutdown 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 msgr2=0x562bc2d09690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d09690 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f90c8007590 tx=0x7f90c80a8cb0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 shutdown_connections 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562bc28d3080 0x562bc2d75ee0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562bc293f350 0x562bc2d71e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bc291e670 0x562bc2d09690 secure :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f90c8007590 tx=0x7f90c80a8cb0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 --2- 172.21.15.31:0/2470174654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562bc2916490 0x562bc2d09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 >> 172.21.15.31:0/2470174654 conn(0x562bc2933740 msgr2=0x562bc2cf65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90bb7fe640 0 client.17895 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 shutdown_connections 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.938+0000 7f90e891a740 1 -- 172.21.15.31:0/2470174654 wait complete. 2024-09-21T06:23:39.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.939+0000 7f90e891a740 20 client.17895 trim_cache size 0 max 16384 2024-09-21T06:23:39.943 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:23:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:23:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "541b6c55-3b9d-40c2-9663-35900f4b8f08", 2024-09-21T06:23:39.943 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:23:39.943 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.939+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90cc0b6780 msgr2=0x7f90cc0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.939+0000 7f90e23a6640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90cc0b6780 0x7f90cc0b8c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005d00 tx=0x7f90d0005c90 comp rx=0 tx=0).stop 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.939+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 msgr2=0x7f90dc199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.939+0000 7f90e23a6640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 0x7f90dc199530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c970 tx=0x7f90c800ce30 comp rx=0 tx=0).stop 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 shutdown_connections 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90dc108b40 0x7f90dc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90cc0b6780 0x7f90cc0b8c90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90dc1081f0 0x7f90dc199530 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 --2- 172.21.15.31:0/21779141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90dc1028b0 0x7f90dc198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 >> 172.21.15.31:0/21779141 conn(0x7f90dc0fe320 msgr2=0x7f90dc103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 shutdown_connections 2024-09-21T06:23:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:39.940+0000 7f90e23a6640 1 -- 172.21.15.31:0/21779141 wait complete. 2024-09-21T06:23:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 720.0 s, 2438.4 tps, lat 41.558 ms stddev 50.853 2024-09-21T06:23:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:40 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:23:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:40 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:40 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:23:40.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:40 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:23:40.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:40 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:40.856 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:40 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:23:40.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:23:41.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:40 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:23:41.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:40 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:41.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:40 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:23:41.315 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:23:41.785 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:41 smithi031 ceph-mon[31237]: pgmap v695: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-21T06:23:41.785 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:41 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:41 smithi107 ceph-mon[34380]: pgmap v695: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-21T06:23:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:41 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.921+0000 7f858bc0d640 1 -- 172.21.15.31:0/2453953966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8584102a60 msgr2=0x7f8584102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.921+0000 7f858bc0d640 1 --2- 172.21.15.31:0/2453953966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8584102a60 0x7f8584102e70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8574009900 tx=0x7f857402a410 comp rx=0 tx=0).stop 2024-09-21T06:23:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 -- 172.21.15.31:0/2453953966 shutdown_connections 2024-09-21T06:23:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 --2- 172.21.15.31:0/2453953966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584108cf0 0x7f858410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 --2- 172.21.15.31:0/2453953966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85841083a0 0x7f85841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 --2- 172.21.15.31:0/2453953966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8584102a60 0x7f8584102e70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 -- 172.21.15.31:0/2453953966 >> 172.21.15.31:0/2453953966 conn(0x7f8584079730 msgr2=0x7f8584079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:41.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 -- 172.21.15.31:0/2453953966 shutdown_connections 2024-09-21T06:23:41.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.922+0000 7f858bc0d640 1 -- 172.21.15.31:0/2453953966 wait complete. 2024-09-21T06:23:41.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 Processor -- start 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 -- start start 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584102a60 0x7f8584104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 0x7f8584104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8584108cf0 0x7f8584104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8584112960 con 0x7f85841083a0 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85841127e0 con 0x7f8584108cf0 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858bc0d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8584112ae0 con 0x7f8584102a60 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f85891a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 0x7f8584104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8584108cf0 0x7f8584104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f85891a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 0x7f8584104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34812/0 (socket says 172.21.15.31:34812) 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f85891a8640 1 -- 172.21.15.31:0/1808121884 learned_addr learned my addr 172.21.15.31:0/1808121884 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.923+0000 7f85899a9640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584102a60 0x7f8584104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85899a9640 1 -- 172.21.15.31:0/1808121884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8584108cf0 msgr2=0x7f8584104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85899a9640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8584108cf0 0x7f8584104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85899a9640 1 -- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 msgr2=0x7f8584104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85899a9640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 0x7f8584104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85899a9640 1 -- 172.21.15.31:0/1808121884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85841a87c0 con 0x7f8584102a60 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85891a8640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 0x7f8584104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:41.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8584108cf0 0x7f8584104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:41.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f85899a9640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584102a60 0x7f8584104250 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f857402a930 tx=0x7f8574031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:41.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f857affd640 1 -- 172.21.15.31:0/1808121884 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857403f030 con 0x7f8584102a60 2024-09-21T06:23:41.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85841a8a50 con 0x7f8584102a60 2024-09-21T06:23:41.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f857affd640 1 -- 172.21.15.31:0/1808121884 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8574004490 con 0x7f8584102a60 2024-09-21T06:23:41.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f857affd640 1 -- 172.21.15.31:0/1808121884 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574035070 con 0x7f8584102a60 2024-09-21T06:23:41.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.924+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85841a8f80 con 0x7f8584102a60 2024-09-21T06:23:41.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.928+0000 7f8592181740 1 Processor -- start 2024-09-21T06:23:41.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.928+0000 7f8592181740 1 -- start start 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f8592181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f8592181740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f8592181740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f8592181740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559920249e50 con 0x55991ff0b390 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f8592181740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559920239280 con 0x55991ff031b0 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f8592181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55992023d090 con 0x55991ff2c070 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f857affd640 1 -- 172.21.15.31:0/1808121884 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8574042030 con 0x7f8584102a60 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f85891a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f85891a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202d40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34828/0 (socket says 172.21.15.31:34828) 2024-09-21T06:23:41.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f85891a8640 1 -- 172.21.15.31:0/1449715496 learned_addr learned my addr 172.21.15.31:0/1449715496 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f858a1aa640 1 -- 172.21.15.31:0/1449715496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 msgr2=0x5599202d3b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202d3b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f858a1aa640 1 -- 172.21.15.31:0/1449715496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 msgr2=0x5599202d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202d40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.929+0000 7f858a1aa640 1 -- 172.21.15.31:0/1449715496 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55991ff97650 con 0x55991ff2c070 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.930+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202d3650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f857c005fd0 tx=0x7f857c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e31fa3fe58141c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:41.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.930+0000 7f856ffff640 1 -- 172.21.15.31:0/1449715496 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c004d20 con 0x55991ff2c070 2024-09-21T06:23:41.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.930+0000 7f856ffff640 1 -- 172.21.15.31:0/1449715496 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857c00f9a0 con 0x55991ff2c070 2024-09-21T06:23:41.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.930+0000 7f856ffff640 1 -- 172.21.15.31:0/1449715496 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c004470 con 0x55991ff2c070 2024-09-21T06:23:41.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f857affd640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85700b6680 0x7f85700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f857affd640 1 -- 172.21.15.31:0/1808121884 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f8574107640 con 0x7f8584102a60 2024-09-21T06:23:41.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f85899a9640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85700b6680 0x7f85700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 -- 172.21.15.31:0/1449715496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 msgr2=0x5599202d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202d3650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f857c005fd0 tx=0x7f857c011bf0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 -- 172.21.15.31:0/1449715496 shutdown_connections 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 --2- 172.21.15.31:0/1449715496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202d3650 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.931+0000 7f8592181740 1 -- 172.21.15.31:0/1449715496 >> 172.21.15.31:0/1449715496 conn(0x55991ff20460 msgr2=0x5599202d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.932+0000 7f8592181740 1 -- 172.21.15.31:0/1449715496 shutdown_connections 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.933+0000 7f85899a9640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85700b6680 0x7f85700b8b90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5599202d22d0 tx=0x7f857402a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:41.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.933+0000 7f8592181740 1 -- 172.21.15.31:0/1449715496 wait complete. 2024-09-21T06:23:41.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.933+0000 7f8592181740 1 -- start start 2024-09-21T06:23:41.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.933+0000 7f8592181740 1 Processor -- start 2024-09-21T06:23:41.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55992009a700 con 0x55991ff2c070 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5599201d4110 con 0x55991ff031b0 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5599202e1e30 con 0x55991ff0b390 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f8592181740 10 client.0 _ll_register_callbacks cb 0x5599201ed990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f85891a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202decf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46772/0 (socket says 172.21.15.31:46772) 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.934+0000 7f858a1aa640 1 -- 172.21.15.31:0/76072818 learned_addr learned my addr 172.21.15.31:0/76072818 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:41.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 -- 172.21.15.31:0/76072818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 msgr2=0x5599202df230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202df230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 -- 172.21.15.31:0/76072818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 msgr2=0x5599202eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202eb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 -- 172.21.15.31:0/76072818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559920201c40 con 0x55991ff031b0 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f85891a8640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:41.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f858a1aa640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202decf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f857c005f40 tx=0x7f857c004050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347a05819c17950f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.935+0000 7f856d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.936+0000 7f856d7fa640 1 -- 172.21.15.31:0/76072818 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c02b060 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.936+0000 7f856d7fa640 1 -- 172.21.15.31:0/76072818 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f857c003cb0 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.936+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857c00f9a0 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.936+0000 7f856d7fa640 1 -- 172.21.15.31:0/76072818 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f857c01e040 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.937+0000 7f8592181740 10 client.27680 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.937+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5599202edb30 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.937+0000 7f856d7fa640 1 -- 172.21.15.31:0/76072818 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857c011320 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f856d7fa640 1 -- 172.21.15.31:0/76072818 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f857c024040 con 0x55991ff031b0 2024-09-21T06:23:41.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f8592181740 10 client.27680 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f8592181740 10 client.27680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f8592181740 20 client.27680 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f8592181740 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991feea9f0 0x5599202f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f8592181740 4 client.27680 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f857c00f9a0 con 0x55991feea9f0 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.938+0000 7f85891a8640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991feea9f0 0x5599202f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.939+0000 7f85891a8640 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991feea9f0 0x5599202f0e90 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ebc155421823a6 server_cookie=cf40dd1399e44737 in_seq=0 out_seq=0 2024-09-21T06:23:41.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.939+0000 7f856d7fa640 10 client.27680 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:41.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.940+0000 7f856d7fa640 1 -- 172.21.15.31:0/76072818 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f857c00f9a0 con 0x55991feea9f0 2024-09-21T06:23:41.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.940+0000 7f856d7fa640 10 client.27680 handle_command_reply: tid=0 2024-09-21T06:23:41.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.940+0000 7f8592181740 1 client.27680 shutdown 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 msgr2=0x5599202decf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202decf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f857c005f40 tx=0x7f857c004050 comp rx=0 tx=0).stop 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 shutdown_connections 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991feea9f0 0x5599202f0e90 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202df230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 --2- 172.21.15.31:0/76072818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202decf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 >> 172.21.15.31:0/76072818 conn(0x55991ff20460 msgr2=0x5599202d67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f856d7fa640 0 client.27680 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 shutdown_connections 2024-09-21T06:23:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 1 -- 172.21.15.31:0/76072818 wait complete. 2024-09-21T06:23:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.941+0000 7f8592181740 20 client.27680 trim_cache size 0 max 16384 2024-09-21T06:23:41.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 Processor -- start 2024-09-21T06:23:41.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 -- start start 2024-09-21T06:23:41.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202e0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85841af9f0 con 0x55991ff2c070 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85841b2250 con 0x55991ff031b0 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.966+0000 7f8592181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85841b0040 con 0x55991ff0b390 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f85891a8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202e0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202e0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46776/0 (socket says 172.21.15.31:46776) 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 -- 172.21.15.31:0/910925102 learned_addr learned my addr 172.21.15.31:0/910925102 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 -- 172.21.15.31:0/910925102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 msgr2=0x5599202d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 -- 172.21.15.31:0/910925102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 msgr2=0x5599202d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f858a1aa640 1 -- 172.21.15.31:0/910925102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599202dffb0 con 0x55991ff031b0 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.967+0000 7f85891a8640 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.968+0000 7f858a1aa640 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202e0920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f857c005630 tx=0x7f857c011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839f701122716a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.968+0000 7f856f7fe640 1 -- 172.21.15.31:0/910925102 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c02b040 con 0x55991ff031b0 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.968+0000 7f856f7fe640 1 -- 172.21.15.31:0/910925102 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857c00f8c0 con 0x55991ff031b0 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.968+0000 7f856f7fe640 1 -- 172.21.15.31:0/910925102 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c0040c0 con 0x55991ff031b0 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.969+0000 7f8592181740 1 -- 172.21.15.31:0/910925102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 msgr2=0x5599202e0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.969+0000 7f8592181740 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202e0920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f857c005630 tx=0x7f857c011bf0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.969+0000 7f8592181740 1 -- 172.21.15.31:0/910925102 shutdown_connections 2024-09-21T06:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.969+0000 7f8592181740 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff2c070 0x5599202d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.969+0000 7f8592181740 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff0b390 0x5599202d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.969+0000 7f8592181740 1 --2- 172.21.15.31:0/910925102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202e0920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.970+0000 7f8592181740 1 -- 172.21.15.31:0/910925102 >> 172.21.15.31:0/910925102 conn(0x55991ff20460 msgr2=0x5599202e0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.971+0000 7f8592181740 1 -- 172.21.15.31:0/910925102 shutdown_connections 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.971+0000 7f8592181740 1 -- 172.21.15.31:0/910925102 wait complete. 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.971+0000 7f8592181740 1 -- start start 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 Processor -- start 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202f1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85841b0710 con 0x55991ff0b390 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85841b1b80 con 0x55991ff031b0 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85841b0de0 con 0x55991ff2c070 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.972+0000 7f8592181740 10 client.0 _ll_register_callbacks cb 0x7f8580009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202f1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34846/0 (socket says 172.21.15.31:34846) 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f858a1aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46778/0 (socket says 172.21.15.31:46778) 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 -- 172.21.15.31:0/1901242521 learned_addr learned my addr 172.21.15.31:0/1901242521 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 -- 172.21.15.31:0/1901242521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 msgr2=0x5599202f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 -- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 msgr2=0x5599202f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 -- 172.21.15.31:0/1901242521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599202f2a30 con 0x55991ff0b390 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.973+0000 7f85891a8640 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202f1c70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8580009600 tx=0x7f8580004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42d38db192d35e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.974+0000 7f856d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.974+0000 7f856d7fa640 1 -- 172.21.15.31:0/1901242521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580003a00 con 0x55991ff0b390 2024-09-21T06:23:41.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.974+0000 7f856d7fa640 1 -- 172.21.15.31:0/1901242521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f8580003ba0 con 0x55991ff0b390 2024-09-21T06:23:41.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.974+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857c00f8c0 con 0x55991ff0b390 2024-09-21T06:23:41.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.974+0000 7f856d7fa640 1 -- 172.21.15.31:0/1901242521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8580003da0 con 0x55991ff0b390 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.975+0000 7f8592181740 10 client.17910 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.975+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55992035b630 con 0x55991ff0b390 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f856d7fa640 1 -- 172.21.15.31:0/1901242521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85800054e0 con 0x55991ff0b390 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f856d7fa640 1 -- 172.21.15.31:0/1901242521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8580023070 con 0x55991ff0b390 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f8592181740 10 client.17910 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f8592181740 10 client.17910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f8592181740 20 client.17910 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f8592181740 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991febfda0 0x55992035de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f8592181740 4 client.17910 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:23:41.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.976+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f857c00f8c0 con 0x55991febfda0 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.977+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991febfda0 0x55992035de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.977+0000 7f856d7fa640 10 client.17910 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.977+0000 7f858a1aa640 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991febfda0 0x55992035de50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f3fd608b3200a6 server_cookie=1ba56c31a42f3a77 in_seq=0 out_seq=0 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f856d7fa640 1 -- 172.21.15.31:0/1901242521 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8522 (crc 0 0 0) 0x7f857c00f8c0 con 0x55991febfda0 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f856d7fa640 10 client.17910 handle_command_reply: tid=0 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 client.17910 shutdown 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 msgr2=0x5599202f1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202f1c70 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8580009600 tx=0x7f8580004610 comp rx=0 tx=0).stop 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 shutdown_connections 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55991febfda0 0x55992035de50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55991ff2c070 0x5599202f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55991ff0b390 0x5599202f1c70 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f856d7fa640 0 client.17910 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 --2- 172.21.15.31:0/1901242521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55991ff031b0 0x5599202f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 >> 172.21.15.31:0/1901242521 conn(0x55991ff20460 msgr2=0x5599202d4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.978+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 shutdown_connections 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.979+0000 7f8592181740 1 -- 172.21.15.31:0/1901242521 wait complete. 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.979+0000 7f8592181740 20 client.17910 trim_cache size 0 max 16384 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:23:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "541b6c55-3b9d-40c2-9663-35900f4b8f08", 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:23:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:23:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.997 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:23:41.998 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:41.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:23:42.001 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:23:42.002 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85700b6680 msgr2=0x7f85700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85700b6680 0x7f85700b8b90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5599202d22d0 tx=0x7f857402a410 comp rx=0 tx=0).stop 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584102a60 msgr2=0x7f8584104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584102a60 0x7f8584104250 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f857402a930 tx=0x7f8574031a20 comp rx=0 tx=0).stop 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 shutdown_connections 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8584108cf0 0x7f8584104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f85700b6680 0x7f85700b8b90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85841083a0 0x7f8584104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 --2- 172.21.15.31:0/1808121884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8584102a60 0x7f8584104250 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 >> 172.21.15.31:0/1808121884 conn(0x7f8584079730 msgr2=0x7f8584101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 shutdown_connections 2024-09-21T06:23:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:23:41.981+0000 7f858bc0d640 1 -- 172.21.15.31:0/1808121884 wait complete. 2024-09-21T06:23:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:41 smithi110 ceph-mon[34550]: pgmap v695: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-21T06:23:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:41 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:23:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.256+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/754965946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e9810a660 msgr2=0x7f5e9810aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.256+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/754965946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e9810a660 0x7f5e9810aa70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009860 tx=0x7f5e9002a340 comp rx=0 tx=0).stop 2024-09-21T06:23:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.263+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/754965946 shutdown_connections 2024-09-21T06:23:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.263+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/754965946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e98110350 0x7f5e98110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.263+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/754965946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e9810fa00 0x7f5e9810fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.263+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/754965946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e9810a660 0x7f5e9810aa70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.263+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/754965946 >> 172.21.15.31:0/754965946 conn(0x7f5e98070dd0 msgr2=0x7f5e980711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.269+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/754965946 shutdown_connections 2024-09-21T06:23:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.270+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/754965946 wait complete. 2024-09-21T06:23:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.271+0000 7f5e9ee8b640 1 Processor -- start 2024-09-21T06:23:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.272+0000 7f5e9ee8b640 1 -- start start 2024-09-21T06:23:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.272+0000 7f5e9ee8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e9810a660 0x7f5e981a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.272+0000 7f5e9ee8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e9810fa00 0x7f5e981a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e9810a660 0x7f5e981a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9ee8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e98110350 0x7f5e981aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e9810fa00 0x7f5e981a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e9810fa00 0x7f5e981a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34854/0 (socket says 172.21.15.31:34854) 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e97fff640 1 -- 172.21.15.31:0/730093169 learned_addr learned my addr 172.21.15.31:0/730093169 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5e98122840 con 0x7f5e9810fa00 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981226c0 con 0x7f5e9810a660 2024-09-21T06:23:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981229c0 con 0x7f5e98110350 2024-09-21T06:23:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9d428640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e98110350 0x7f5e981aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9d428640 1 -- 172.21.15.31:0/730093169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e9810a660 msgr2=0x7f5e981a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9d428640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e9810a660 0x7f5e981a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9d428640 1 -- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e9810fa00 msgr2=0x7f5e981a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9d428640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e9810fa00 0x7f5e981a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.273+0000 7f5e9d428640 1 -- 172.21.15.31:0/730093169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e981af1f0 con 0x7f5e98110350 2024-09-21T06:23:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.274+0000 7f5e9d428640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e98110350 0x7f5e981aeaf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800e940 tx=0x7f5e8800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.275+0000 7f5e95ffb640 1 -- 172.21.15.31:0/730093169 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88004d20 con 0x7f5e98110350 2024-09-21T06:23:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.275+0000 7f5e95ffb640 1 -- 172.21.15.31:0/730093169 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e88005020 con 0x7f5e98110350 2024-09-21T06:23:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.275+0000 7f5e95ffb640 1 -- 172.21.15.31:0/730093169 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880116b0 con 0x7f5e98110350 2024-09-21T06:23:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.275+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e981af4e0 con 0x7f5e98110350 2024-09-21T06:23:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.275+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e981b5c70 con 0x7f5e98110350 2024-09-21T06:23:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.279+0000 7f5e95ffb640 1 -- 172.21.15.31:0/730093169 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5e8800ca60 con 0x7f5e98110350 2024-09-21T06:23:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 Processor -- start 2024-09-21T06:23:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 -- start start 2024-09-21T06:23:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55638b71c6f0 con 0x55638b3d92f0 2024-09-21T06:23:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55638b70bb20 con 0x55638b4021b0 2024-09-21T06:23:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55638b70f930 con 0x55638b3e14d0 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.280+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34868/0 (socket says 172.21.15.31:34868) 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 -- 172.21.15.31:0/390466473 learned_addr learned my addr 172.21.15.31:0/390466473 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e97fff640 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e97fff640 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e95ffb640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5e840b6730 0x7f5e840b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 -- 172.21.15.31:0/390466473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 msgr2=0x55638b7a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 -- 172.21.15.31:0/390466473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 msgr2=0x55638b7a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7a79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 -- 172.21.15.31:0/390466473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638b6d1530 con 0x55638b3d92f0 2024-09-21T06:23:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e97fff640 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9d428640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5e840b6730 0x7f5e840b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.281+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a7ef0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e90000c00 tx=0x7f5e90009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad2fbc1d7d1bb4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.282+0000 7f5e8effd640 1 -- 172.21.15.31:0/390466473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90037040 con 0x55638b3d92f0 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.282+0000 7f5e8effd640 1 -- 172.21.15.31:0/390466473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e90042070 con 0x55638b3d92f0 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.282+0000 7f5e9d428640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5e840b6730 0x7f5e840b8c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e880a9b10 tx=0x7f5e880a9c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.282+0000 7f5e95ffb640 1 -- 172.21.15.31:0/730093169 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5e880e76e0 con 0x7f5e98110350 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.283+0000 7f5e8effd640 1 -- 172.21.15.31:0/390466473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90037040 con 0x55638b3d92f0 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 -- 172.21.15.31:0/390466473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 msgr2=0x55638b7a7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a7ef0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5e90000c00 tx=0x7f5e90009540 comp rx=0 tx=0).stop 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 -- 172.21.15.31:0/390466473 shutdown_connections 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a7ef0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/390466473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 -- 172.21.15.31:0/390466473 >> 172.21.15.31:0/390466473 conn(0x55638b3f6300 msgr2=0x55638b7a58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 -- 172.21.15.31:0/390466473 shutdown_connections 2024-09-21T06:23:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.284+0000 7f5ea5df7740 1 -- 172.21.15.31:0/390466473 wait complete. 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 -- start start 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 Processor -- start 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 0x55638b7b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b4021b0 0x55638b7b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55638b643ab0 con 0x55638b3e14d0 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55638b6b1730 con 0x55638b3d92f0 2024-09-21T06:23:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55638b7b6120 con 0x55638b4021b0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.285+0000 7f5ea5df7740 10 client.0 _ll_register_callbacks cb 0x55638b7056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 0x55638b7b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 0x55638b7b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46830/0 (socket says 172.21.15.31:46830) 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 -- 172.21.15.31:0/1248697366 learned_addr learned my addr 172.21.15.31:0/1248697366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b4021b0 0x55638b7b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 -- 172.21.15.31:0/1248697366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b4021b0 msgr2=0x55638b7b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b4021b0 0x55638b7b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 -- 172.21.15.31:0/1248697366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 msgr2=0x55638b7b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.286+0000 7f5e97fff640 1 -- 172.21.15.31:0/1248697366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638b46d790 con 0x55638b3d92f0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 0x55638b7b2f90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5e80011fd0 tx=0x7f5e80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed4c1e088b8fefa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5e77fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5e77fff640 1 -- 172.21.15.31:0/1248697366 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80004460 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5e77fff640 1 -- 172.21.15.31:0/1248697366 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5e80005020 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e90042070 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5e77fff640 1 -- 172.21.15.31:0/1248697366 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5e80029050 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5ea5df7740 10 client.27698 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.287+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55638b4e6620 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5e77fff640 1 -- 172.21.15.31:0/1248697366 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e90042070 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5e77fff640 1 -- 172.21.15.31:0/1248697366 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5e80020040 con 0x55638b3d92f0 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5ea5df7740 10 client.27698 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5ea5df7740 10 client.27698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5ea5df7740 20 client.27698 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3c0d80 0x55638b7c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5ea5df7740 4 client.27698 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55638b71e640 con 0x55638b3c0d80 2024-09-21T06:23:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.288+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3c0d80 0x55638b7c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.289+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3c0d80 0x55638b7c50c0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32b307bc44deac2 server_cookie=b0e7e1c4cb91d499 in_seq=0 out_seq=0 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.289+0000 7f5e77fff640 10 client.27698 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5e77fff640 1 -- 172.21.15.31:0/1248697366 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55638b71e640 con 0x55638b3c0d80 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5e77fff640 10 client.27698 handle_command_reply: tid=0 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 client.27698 shutdown 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 msgr2=0x55638b7b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 0x55638b7b2f90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5e80011fd0 tx=0x7f5e80004730 comp rx=0 tx=0).stop 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 shutdown_connections 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3c0d80 0x55638b7c50c0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b4021b0 0x55638b7b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1248697366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b3d92f0 0x55638b7b2f90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5e77fff640 0 client.27698 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 >> 172.21.15.31:0/1248697366 conn(0x55638b3f6300 msgr2=0x55638b7a89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 shutdown_connections 2024-09-21T06:23:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1248697366 wait complete. 2024-09-21T06:23:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.290+0000 7f5ea5df7740 20 client.27698 trim_cache size 0 max 16384 2024-09-21T06:23:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.316+0000 7f5ea5df7740 1 Processor -- start 2024-09-21T06:23:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.316+0000 7f5ea5df7740 1 -- start start 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bc490 con 0x55638b3d92f0 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981becf0 con 0x55638b4021b0 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bcae0 con 0x55638b3e14d0 2024-09-21T06:23:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46832/0 (socket says 172.21.15.31:46832) 2024-09-21T06:23:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59552/0 (socket says 172.21.15.31:59552) 2024-09-21T06:23:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 -- 172.21.15.31:0/2205223901 learned_addr learned my addr 172.21.15.31:0/2205223901 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 -- 172.21.15.31:0/2205223901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 msgr2=0x55638b7b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 -- 172.21.15.31:0/2205223901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 msgr2=0x55638b7a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.317+0000 7f5e97fff640 1 -- 172.21.15.31:0/2205223901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55638b7a9ee0 con 0x55638b4021b0 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.318+0000 7f5e97fff640 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b06a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5e80004dc0 tx=0x7f5e80000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69503a3f7f010920 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.318+0000 7f5e8e7fc640 1 -- 172.21.15.31:0/2205223901 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800047a0 con 0x55638b4021b0 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.318+0000 7f5e8e7fc640 1 -- 172.21.15.31:0/2205223901 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e80004940 con 0x55638b4021b0 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.318+0000 7f5e8e7fc640 1 -- 172.21.15.31:0/2205223901 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800035f0 con 0x55638b4021b0 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 -- 172.21.15.31:0/2205223901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 msgr2=0x55638b7b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b06a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5e80004dc0 tx=0x7f5e80000bc0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 -- 172.21.15.31:0/2205223901 shutdown_connections 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b06a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3e14d0 0x55638b7b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/2205223901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3d92f0 0x55638b7a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.319+0000 7f5ea5df7740 1 -- 172.21.15.31:0/2205223901 >> 172.21.15.31:0/2205223901 conn(0x55638b3f6300 msgr2=0x55638b7a89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.320+0000 7f5ea5df7740 1 -- 172.21.15.31:0/2205223901 shutdown_connections 2024-09-21T06:23:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.320+0000 7f5ea5df7740 1 -- 172.21.15.31:0/2205223901 wait complete. 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.321+0000 7f5ea5df7740 1 -- start start 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.321+0000 7f5ea5df7740 1 Processor -- start 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 0x55638b7bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bd1b0 con 0x55638b3e14d0 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981be620 con 0x55638b4021b0 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5e981bd880 con 0x55638b3d92f0 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 0x55638b7bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5ea5df7740 10 client.0 _ll_register_callbacks cb 0x7f5e800115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 0x55638b7bbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59568/0 (socket says 172.21.15.31:59568) 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e9cc27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34894/0 (socket says 172.21.15.31:34894) 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 -- 172.21.15.31:0/1966310148 learned_addr learned my addr 172.21.15.31:0/1966310148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 -- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 msgr2=0x55638b7b9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 -- 172.21.15.31:0/1966310148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 msgr2=0x55638b7b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.322+0000 7f5e97fff640 1 -- 172.21.15.31:0/1966310148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638b7b9fb0 con 0x55638b3d92f0 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.323+0000 7f5e97fff640 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 0x55638b7bbcc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5e80025040 tx=0x7f5e80005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=764914d096c87758 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.323+0000 7f5e77fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:23:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.323+0000 7f5e77fff640 1 -- 172.21.15.31:0/1966310148 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80042060 con 0x55638b3d92f0 2024-09-21T06:23:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.323+0000 7f5e77fff640 1 -- 172.21.15.31:0/1966310148 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7f5e80004e40 con 0x55638b3d92f0 2024-09-21T06:23:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.323+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e80004940 con 0x55638b3d92f0 2024-09-21T06:23:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5e77fff640 1 -- 172.21.15.31:0/1966310148 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5e80005040 con 0x55638b3d92f0 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 10 client.27760 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55638b82fb90 con 0x55638b3d92f0 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5e77fff640 1 -- 172.21.15.31:0/1966310148 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e80004940 con 0x55638b3d92f0 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5e77fff640 1 -- 172.21.15.31:0/1966310148 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5e8003a050 con 0x55638b3d92f0 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 10 client.27760 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 10 client.27760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 20 client.27760 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3961c0 0x55638b8323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 4 client.27760 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.324+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55638b71a0f0 con 0x55638b3961c0 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.325+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3961c0 0x55638b8323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.325+0000 7f5e77fff640 10 client.27760 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.325+0000 7f5e9cc27640 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3961c0 0x55638b8323c0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d778d6301e8be762 server_cookie=91e03d646a3e8dba in_seq=0 out_seq=0 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5e77fff640 1 -- 172.21.15.31:0/1966310148 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55638b71a0f0 con 0x55638b3961c0 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5e77fff640 10 client.27760 handle_command_reply: tid=0 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 client.27760 shutdown 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 msgr2=0x55638b7bbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 0x55638b7bbcc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5e80025040 tx=0x7f5e80005c80 comp rx=0 tx=0).stop 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 shutdown_connections 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55638b3961c0 0x55638b8323c0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55638b4021b0 0x55638b7b9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55638b3e14d0 0x55638b7b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 --2- 172.21.15.31:0/1966310148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55638b3d92f0 0x55638b7bbcc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 >> 172.21.15.31:0/1966310148 conn(0x55638b3f6300 msgr2=0x55638b7b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.359+0000 7f5e77fff640 0 client.27760 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.360+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 shutdown_connections 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.360+0000 7f5ea5df7740 1 -- 172.21.15.31:0/1966310148 wait complete. 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.360+0000 7f5ea5df7740 20 client.27760 trim_cache size 0 max 16384 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:23:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:23:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5e840b6730 msgr2=0x7f5e840b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5e840b6730 0x7f5e840b8c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e880a9b10 tx=0x7f5e880a9c90 comp rx=0 tx=0).stop 2024-09-21T06:23:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e98110350 msgr2=0x7f5e981aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e98110350 0x7f5e981aeaf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800e940 tx=0x7f5e8800ed40 comp rx=0 tx=0).stop 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 shutdown_connections 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5e98110350 0x7f5e981aeaf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5e840b6730 0x7f5e840b8c40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e9810fa00 0x7f5e981a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 --2- 172.21.15.31:0/730093169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e9810a660 0x7f5e981a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.362+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 >> 172.21.15.31:0/730093169 conn(0x7f5e98070dd0 msgr2=0x7f5e98073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.363+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 shutdown_connections 2024-09-21T06:23:42.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:42.363+0000 7f5e9ee8b640 1 -- 172.21.15.31:0/730093169 wait complete. 2024-09-21T06:23:42.759 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:42 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:23:42.761 INFO:tasks.cephfs.filesystem:scrub status for tag:541b6c55-3b9d-40c2-9663-35900f4b8f08 - {'path': '/', 'tag': '541b6c55-3b9d-40c2-9663-35900f4b8f08', 'options': 'recursive,force'} 2024-09-21T06:23:43.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:42 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:23:43.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:42 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:23:43.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:43 smithi031 ceph-mon[31237]: pgmap v696: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 14 MiB/s wr, 1.50k op/s 2024-09-21T06:23:44.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:43 smithi110 ceph-mon[34550]: pgmap v696: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 14 MiB/s wr, 1.50k op/s 2024-09-21T06:23:44.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:43 smithi107 ceph-mon[34380]: pgmap v696: 97 pgs: 97 active+clean; 4.4 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 14 MiB/s wr, 1.50k op/s 2024-09-21T06:23:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 725.0 s, 1897.2 tps, lat 52.532 ms stddev 63.131 2024-09-21T06:23:45.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:45 smithi031 ceph-mon[31237]: pgmap v697: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 14 MiB/s wr, 1.54k op/s 2024-09-21T06:23:46.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:45 smithi110 ceph-mon[34550]: pgmap v697: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 14 MiB/s wr, 1.54k op/s 2024-09-21T06:23:46.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:45 smithi107 ceph-mon[34380]: pgmap v697: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 14 MiB/s wr, 1.54k op/s 2024-09-21T06:23:47.830 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:47 smithi110 ceph-mon[34550]: pgmap v698: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 12 MiB/s wr, 1.36k op/s 2024-09-21T06:23:47.840 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:47 smithi107 ceph-mon[34380]: pgmap v698: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 12 MiB/s wr, 1.36k op/s 2024-09-21T06:23:47.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:47 smithi031 ceph-mon[31237]: pgmap v698: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 12 MiB/s wr, 1.36k op/s 2024-09-21T06:23:49.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:49 smithi031 ceph-mon[31237]: pgmap v699: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.6 MiB/s wr, 871 op/s 2024-09-21T06:23:50.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:49 smithi110 ceph-mon[34550]: pgmap v699: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.6 MiB/s wr, 871 op/s 2024-09-21T06:23:50.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:49 smithi107 ceph-mon[34380]: pgmap v699: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 7.6 MiB/s wr, 871 op/s 2024-09-21T06:23:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 730.0 s, 1498.6 tps, lat 65.790 ms stddev 72.588 2024-09-21T06:23:51.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:51 smithi031 ceph-mon[31237]: pgmap v700: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 706 op/s 2024-09-21T06:23:52.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:51 smithi110 ceph-mon[34550]: pgmap v700: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 706 op/s 2024-09-21T06:23:52.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:51 smithi107 ceph-mon[34380]: pgmap v700: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.4 MiB/s wr, 706 op/s 2024-09-21T06:23:53.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:53 smithi031 ceph-mon[31237]: pgmap v701: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.8 MiB/s wr, 757 op/s 2024-09-21T06:23:54.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:53 smithi110 ceph-mon[34550]: pgmap v701: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.8 MiB/s wr, 757 op/s 2024-09-21T06:23:54.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:53 smithi107 ceph-mon[34380]: pgmap v701: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.8 MiB/s wr, 757 op/s 2024-09-21T06:23:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 735.0 s, 1776.0 tps, lat 57.171 ms stddev 71.219 2024-09-21T06:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.507+0000 7efd5109d640 1 -- 172.21.15.31:0/800621215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c1029e0 msgr2=0x7efd4c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.507+0000 7efd5109d640 1 --2- 172.21.15.31:0/800621215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c1029e0 0x7efd4c102df0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009860 tx=0x7efd3c02a340 comp rx=0 tx=0).stop 2024-09-21T06:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.508+0000 7efd5109d640 1 -- 172.21.15.31:0/800621215 shutdown_connections 2024-09-21T06:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.508+0000 7efd5109d640 1 --2- 172.21.15.31:0/800621215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c108c60 0x7efd4c10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.508+0000 7efd5109d640 1 --2- 172.21.15.31:0/800621215 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108310 0x7efd4c108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.508+0000 7efd5109d640 1 --2- 172.21.15.31:0/800621215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c1029e0 0x7efd4c102df0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.508+0000 7efd5109d640 1 -- 172.21.15.31:0/800621215 >> 172.21.15.31:0/800621215 conn(0x7efd4c0fe3c0 msgr2=0x7efd4c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.509+0000 7efd5109d640 1 -- 172.21.15.31:0/800621215 shutdown_connections 2024-09-21T06:23:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.509+0000 7efd5109d640 1 -- 172.21.15.31:0/800621215 wait complete. 2024-09-21T06:23:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.510+0000 7efd5109d640 1 Processor -- start 2024-09-21T06:23:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.510+0000 7efd5109d640 1 -- start start 2024-09-21T06:23:55.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd5109d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 0x7efd4c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd5109d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c108310 0x7efd4c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd5109d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108c60 0x7efd4c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd5109d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c112920 con 0x7efd4c1029e0 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd5109d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1127a0 con 0x7efd4c108310 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd5109d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c112aa0 con 0x7efd4c108c60 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 0x7efd4c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c108310 0x7efd4c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 0x7efd4c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49090/0 (socket says 172.21.15.31:49090) 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4ad9d640 1 -- 172.21.15.31:0/678188385 learned_addr learned my addr 172.21.15.31:0/678188385 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4b59e640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108c60 0x7efd4c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4a59c640 1 -- 172.21.15.31:0/678188385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108c60 msgr2=0x7efd4c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4a59c640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108c60 0x7efd4c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4a59c640 1 -- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 msgr2=0x7efd4c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4a59c640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 0x7efd4c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4a59c640 1 -- 172.21.15.31:0/678188385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c19f270 con 0x7efd4c108310 2024-09-21T06:23:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4b59e640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108c60 0x7efd4c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.511+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 0x7efd4c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.512+0000 7efd4a59c640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c108310 0x7efd4c1998e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efd4c06b5a0 tx=0x7efd4000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:55.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.512+0000 7efd33fff640 1 -- 172.21.15.31:0/678188385 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4000bc80 con 0x7efd4c108310 2024-09-21T06:23:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.512+0000 7efd33fff640 1 -- 172.21.15.31:0/678188385 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4000be20 con 0x7efd4c108310 2024-09-21T06:23:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.512+0000 7efd33fff640 1 -- 172.21.15.31:0/678188385 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4001d780 con 0x7efd4c108310 2024-09-21T06:23:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.512+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c19f560 con 0x7efd4c108310 2024-09-21T06:23:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.512+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c06b980 con 0x7efd4c108310 2024-09-21T06:23:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.515+0000 7efd33fff640 1 -- 172.21.15.31:0/678188385 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efd40015a80 con 0x7efd4c108310 2024-09-21T06:23:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.515+0000 7efd58008740 1 Processor -- start 2024-09-21T06:23:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.515+0000 7efd58008740 1 -- start start 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd58008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd58008740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dabef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd58008740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd58008740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558f62d206f0 con 0x558f629dd2f0 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd58008740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558f62d0fb20 con 0x558f629e54d0 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd58008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f62d13930 con 0x558f62a061b0 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd4b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46254/0 (socket says 172.21.15.31:46254) 2024-09-21T06:23:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.516+0000 7efd4a59c640 1 -- 172.21.15.31:0/3909191596 learned_addr learned my addr 172.21.15.31:0/3909191596 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd33fff640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd340b67d0 0x7efd340b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 -- 172.21.15.31:0/3909191596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 msgr2=0x558f62dac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dabef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 -- 172.21.15.31:0/3909191596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 msgr2=0x558f62dabef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dabef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 -- 172.21.15.31:0/3909191596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f62cd5530 con 0x558f62a061b0 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dabef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4a59c640 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd33fff640 1 -- 172.21.15.31:0/678188385 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7efd400eb6d0 con 0x7efd4c108310 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd340b67d0 0x7efd340b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.517+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dab9b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efd440106c0 tx=0x7efd44010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc721dbafe36ab41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.518+0000 7efd317fa640 1 -- 172.21.15.31:0/3909191596 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4400ead0 con 0x558f62a061b0 2024-09-21T06:23:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.518+0000 7efd317fa640 1 -- 172.21.15.31:0/3909191596 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4400ec70 con 0x558f62a061b0 2024-09-21T06:23:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.518+0000 7efd317fa640 1 -- 172.21.15.31:0/3909191596 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440127e0 con 0x558f62a061b0 2024-09-21T06:23:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 -- 172.21.15.31:0/3909191596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 msgr2=0x558f62dab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd4b59e640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd340b67d0 0x7efd340b8ce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd4401c000 tx=0x7efd44012a60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dab9b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efd440106c0 tx=0x7efd44010b80 comp rx=0 tx=0).stop 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 -- 172.21.15.31:0/3909191596 shutdown_connections 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dabef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 --2- 172.21.15.31:0/3909191596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dab9b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 -- 172.21.15.31:0/3909191596 >> 172.21.15.31:0/3909191596 conn(0x558f629fa300 msgr2=0x558f62da98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 -- 172.21.15.31:0/3909191596 shutdown_connections 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 -- 172.21.15.31:0/3909191596 wait complete. 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 -- start start 2024-09-21T06:23:55.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.519+0000 7efd58008740 1 Processor -- start 2024-09-21T06:23:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dc0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dc0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558f62c47ab0 con 0x558f629dd2f0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558f62cb5730 con 0x558f629e54d0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f62dba130 con 0x558f62a061b0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd58008740 10 client.0 _ll_register_callbacks cb 0x558f62d096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd4b59e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dc0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dc0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46262/0 (socket says 172.21.15.31:46262) 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd4a59c640 1 -- 172.21.15.31:0/2414373226 learned_addr learned my addr 172.21.15.31:0/2414373226 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.520+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dc0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd4a59c640 1 -- 172.21.15.31:0/2414373226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 msgr2=0x558f62dc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd4a59c640 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dc5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd4a59c640 1 -- 172.21.15.31:0/2414373226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 msgr2=0x558f62dc0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd4a59c640 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dc0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd4a59c640 1 -- 172.21.15.31:0/2414373226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f62a71790 con 0x558f629e54d0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd4a59c640 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dc0750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efd400e8ee0 tx=0x7efd400badb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fa1f54ce585466 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd2eff5640 1 -- 172.21.15.31:0/2414373226 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400b1300 con 0x558f629e54d0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd2eff5640 1 -- 172.21.15.31:0/2414373226 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7efd400ace80 con 0x558f629e54d0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.521+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4400ec70 con 0x558f629e54d0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd2eff5640 1 -- 172.21.15.31:0/2414373226 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd4000ee60 con 0x558f629e54d0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 10 client.27716 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558f62aea620 con 0x558f629e54d0 2024-09-21T06:23:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd2eff5640 1 -- 172.21.15.31:0/2414373226 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4400ec70 con 0x558f629e54d0 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd2eff5640 1 -- 172.21.15.31:0/2414373226 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efd40025030 con 0x558f629e54d0 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 10 client.27716 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 10 client.27716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 20 client.27716 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f629c4d80 0x558f62dd1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 4 client.27716 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f62d22640 con 0x558f629c4d80 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.522+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f629c4d80 0x558f62dd1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.523+0000 7efd2eff5640 10 client.27716 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.523+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f629c4d80 0x558f62dd1980 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536ec308cbd357c8 server_cookie=23e76eca4e980c84 in_seq=0 out_seq=0 2024-09-21T06:23:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.524+0000 7efd2eff5640 1 -- 172.21.15.31:0/2414373226 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f62d22640 con 0x558f629c4d80 2024-09-21T06:23:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.524+0000 7efd2eff5640 10 client.27716 handle_command_reply: tid=0 2024-09-21T06:23:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.524+0000 7efd58008740 1 client.27716 shutdown 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 msgr2=0x558f62dc0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dc0750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7efd400e8ee0 tx=0x7efd400badb0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 shutdown_connections 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f629c4d80 0x558f62dd1980 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f62a061b0 0x558f62dc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f629e54d0 0x558f62dc0750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 --2- 172.21.15.31:0/2414373226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629dd2f0 0x558f62dc0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 >> 172.21.15.31:0/2414373226 conn(0x558f629fa300 msgr2=0x558f62dac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd2eff5640 0 client.27716 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 shutdown_connections 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 1 -- 172.21.15.31:0/2414373226 wait complete. 2024-09-21T06:23:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.525+0000 7efd58008740 20 client.27716 trim_cache size 0 max 16384 2024-09-21T06:23:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.544+0000 7efd58008740 1 Processor -- start 2024-09-21T06:23:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.545+0000 7efd58008740 1 -- start start 2024-09-21T06:23:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.545+0000 7efd58008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62db94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.545+0000 7efd58008740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dc55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.545+0000 7efd58008740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd58008740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1ab480 con 0x558f629e54d0 2024-09-21T06:23:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd58008740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1adce0 con 0x558f62a061b0 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd58008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1abad0 con 0x558f629dd2f0 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62db94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dc55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62db94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50646/0 (socket says 172.21.15.31:50646) 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dc55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49114/0 (socket says 172.21.15.31:49114) 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 -- 172.21.15.31:0/1760037083 learned_addr learned my addr 172.21.15.31:0/1760037083 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4b59e640 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 -- 172.21.15.31:0/1760037083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 msgr2=0x558f62dac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 -- 172.21.15.31:0/1760037083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 msgr2=0x558f62dc55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dc55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 -- 172.21.15.31:0/1760037083 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f62dd1ee0 con 0x558f629dd2f0 2024-09-21T06:23:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4a59c640 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dc55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:23:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.546+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62db94b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009980 tx=0x7efd3c030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03b9d9bb2e58788 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd30ff9640 1 -- 172.21.15.31:0/1760037083 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c037040 con 0x558f629dd2f0 2024-09-21T06:23:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd30ff9640 1 -- 172.21.15.31:0/1760037083 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c04a070 con 0x558f629dd2f0 2024-09-21T06:23:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd30ff9640 1 -- 172.21.15.31:0/1760037083 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c032b50 con 0x558f629dd2f0 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 -- 172.21.15.31:0/1760037083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 msgr2=0x558f62db94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62db94b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009980 tx=0x7efd3c030340 comp rx=0 tx=0).stop 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 -- 172.21.15.31:0/1760037083 shutdown_connections 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dc55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 --2- 172.21.15.31:0/1760037083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62db94b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.547+0000 7efd58008740 1 -- 172.21.15.31:0/1760037083 >> 172.21.15.31:0/1760037083 conn(0x558f629fa300 msgr2=0x558f62dbe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.548+0000 7efd58008740 1 -- 172.21.15.31:0/1760037083 shutdown_connections 2024-09-21T06:23:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.548+0000 7efd58008740 1 -- 172.21.15.31:0/1760037083 wait complete. 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.548+0000 7efd58008740 1 -- start start 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.548+0000 7efd58008740 1 Processor -- start 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62dd6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dd7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dc77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1ac1a0 con 0x558f629e54d0 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1ad610 con 0x558f62a061b0 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1ac870 con 0x558f629dd2f0 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dd7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4ad9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62dd6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dd7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49118/0 (socket says 172.21.15.31:49118) 2024-09-21T06:23:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4a59c640 1 -- 172.21.15.31:0/3477566774 learned_addr learned my addr 172.21.15.31:0/3477566774 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62dd6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:50660/0 (socket says 172.21.15.31:50660) 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dc77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd58008740 10 client.0 _ll_register_callbacks cb 0x7efd400e92e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4b59e640 1 -- 172.21.15.31:0/3477566774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 msgr2=0x558f62dd6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62dd6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4b59e640 1 -- 172.21.15.31:0/3477566774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 msgr2=0x558f62dd7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dd7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.549+0000 7efd4b59e640 1 -- 172.21.15.31:0/3477566774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f62dc7de0 con 0x558f62a061b0 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd4b59e640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dc77a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efd4401de90 tx=0x7efd44017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=340e27004a0d4057 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:23:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd2eff5640 1 -- 172.21.15.31:0/3477566774 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44038020 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd2eff5640 1 -- 172.21.15.31:0/3477566774 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8499+0+0 (secure 0 0 0) 0x7efd44012c70 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3c04a070 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd2eff5640 1 -- 172.21.15.31:0/3477566774 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd44043050 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd58008740 10 client.27722 fetch_fsmap learned FSMap version 21 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.550+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558f62dcc2c0 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd2eff5640 1 -- 172.21.15.31:0/3477566774 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c04a070 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd2eff5640 1 -- 172.21.15.31:0/3477566774 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efd4400f040 con 0x558f62a061b0 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd58008740 10 client.27722 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd58008740 10 client.27722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd58008740 20 client.27722 populate_metadata read hostname 'smithi031' 2024-09-21T06:23:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd58008740 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f6299a1c0 0x558f62e3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:23:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd58008740 4 client.27722 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:23:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f62d1e0f0 con 0x558f6299a1c0 2024-09-21T06:23:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f6299a1c0 0x558f62e3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:23:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd2eff5640 10 client.27722 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.551+0000 7efd4ad9d640 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f6299a1c0 0x558f62e3e340 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60290392c8b0aaba server_cookie=5bdb6684d2e31644 in_seq=0 out_seq=0 2024-09-21T06:23:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.577+0000 7efd2eff5640 1 -- 172.21.15.31:0/3477566774 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f62d1e0f0 con 0x558f6299a1c0 2024-09-21T06:23:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.577+0000 7efd2eff5640 10 client.27722 handle_command_reply: tid=0 2024-09-21T06:23:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.577+0000 7efd58008740 1 client.27722 shutdown 2024-09-21T06:23:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.577+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 msgr2=0x558f62dc77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.577+0000 7efd58008740 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dc77a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efd4401de90 tx=0x7efd44017000 comp rx=0 tx=0).stop 2024-09-21T06:23:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 shutdown_connections 2024-09-21T06:23:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558f6299a1c0 0x558f62e3e340 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd2eff5640 0 client.27722 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:23:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558f62a061b0 0x558f62dc77a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f629e54d0 0x558f62dd7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 --2- 172.21.15.31:0/3477566774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f629dd2f0 0x558f62dd6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 >> 172.21.15.31:0/3477566774 conn(0x558f629fa300 msgr2=0x558f62dac9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 shutdown_connections 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 1 -- 172.21.15.31:0/3477566774 wait complete. 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd58008740 20 client.27722 trim_cache size 0 max 16384 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:23:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:23:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd340b67d0 msgr2=0x7efd340b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd340b67d0 0x7efd340b8ce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd4401c000 tx=0x7efd44012a60 comp rx=0 tx=0).stop 2024-09-21T06:23:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c108310 msgr2=0x7efd4c1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:23:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c108310 0x7efd4c1998e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7efd4c06b5a0 tx=0x7efd4000ed30 comp rx=0 tx=0).stop 2024-09-21T06:23:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 shutdown_connections 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd4c108c60 0x7efd4c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd4c108310 0x7efd4c1998e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd340b67d0 0x7efd340b8ce0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 --2- 172.21.15.31:0/678188385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd4c1029e0 0x7efd4c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.578+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 >> 172.21.15.31:0/678188385 conn(0x7efd4c0fe3c0 msgr2=0x7efd4c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.579+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 shutdown_connections 2024-09-21T06:23:55.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:23:55.579+0000 7efd5109d640 1 -- 172.21.15.31:0/678188385 wait complete. 2024-09-21T06:23:55.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:55 smithi031 ceph-mon[31237]: pgmap v702: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.3 MiB/s wr, 721 op/s 2024-09-21T06:23:56.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:55 smithi110 ceph-mon[34550]: pgmap v702: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.3 MiB/s wr, 721 op/s 2024-09-21T06:23:56.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:55 smithi107 ceph-mon[34380]: pgmap v702: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.3 MiB/s wr, 721 op/s 2024-09-21T06:23:56.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:23:56.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:23:56.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:23:56.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:56 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:23:57.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:23:57.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:23:57.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:23:57.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:56 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:23:57.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:23:57.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:23:57.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:23:57.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:56 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:23:57.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:57 smithi031 ceph-mon[31237]: pgmap v703: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 704 op/s 2024-09-21T06:23:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:57 smithi110 ceph-mon[34550]: pgmap v703: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 704 op/s 2024-09-21T06:23:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:57 smithi107 ceph-mon[34380]: pgmap v703: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 704 op/s 2024-09-21T06:23:59.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:23:59 smithi031 ceph-mon[31237]: pgmap v704: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.0 MiB/s wr, 715 op/s 2024-09-21T06:24:00.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:23:59 smithi110 ceph-mon[34550]: pgmap v704: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.0 MiB/s wr, 715 op/s 2024-09-21T06:24:00.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:23:59 smithi107 ceph-mon[34380]: pgmap v704: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.0 MiB/s wr, 715 op/s 2024-09-21T06:24:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 740.0 s, 1548.4 tps, lat 64.157 ms stddev 73.053 2024-09-21T06:24:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:01 smithi031 ceph-mon[31237]: pgmap v705: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-09-21T06:24:02.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:01 smithi110 ceph-mon[34550]: pgmap v705: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-09-21T06:24:02.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:01 smithi107 ceph-mon[34380]: pgmap v705: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.3 MiB/s wr, 646 op/s 2024-09-21T06:24:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:03 smithi031 ceph-mon[31237]: pgmap v706: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 645 op/s 2024-09-21T06:24:04.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:03 smithi110 ceph-mon[34550]: pgmap v706: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 645 op/s 2024-09-21T06:24:04.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:03 smithi107 ceph-mon[34380]: pgmap v706: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 645 op/s 2024-09-21T06:24:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 745.0 s, 428.6 tps, lat 229.140 ms stddev 205.548 2024-09-21T06:24:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:05 smithi031 ceph-mon[31237]: pgmap v707: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.2 MiB/s wr, 675 op/s 2024-09-21T06:24:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:05 smithi110 ceph-mon[34550]: pgmap v707: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.2 MiB/s wr, 675 op/s 2024-09-21T06:24:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:05 smithi107 ceph-mon[34380]: pgmap v707: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.2 MiB/s wr, 675 op/s 2024-09-21T06:24:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:07 smithi031 ceph-mon[31237]: pgmap v708: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 611 op/s 2024-09-21T06:24:08.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:07 smithi110 ceph-mon[34550]: pgmap v708: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 611 op/s 2024-09-21T06:24:08.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:07 smithi107 ceph-mon[34380]: pgmap v708: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 611 op/s 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 -- 172.21.15.31:0/4177972477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3741029e0 msgr2=0x7fe374102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/4177972477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3741029e0 0x7fe374102df0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe370009900 tx=0x7fe37002a410 comp rx=0 tx=0).stop 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 -- 172.21.15.31:0/4177972477 shutdown_connections 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/4177972477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe374108c60 0x7fe37410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/4177972477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 0x7fe374108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/4177972477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3741029e0 0x7fe374102df0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.685+0000 7fe37bfdf640 1 -- 172.21.15.31:0/4177972477 >> 172.21.15.31:0/4177972477 conn(0x7fe37407a6e0 msgr2=0x7fe37407aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.686+0000 7fe37bfdf640 1 -- 172.21.15.31:0/4177972477 shutdown_connections 2024-09-21T06:24:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.686+0000 7fe37bfdf640 1 -- 172.21.15.31:0/4177972477 wait complete. 2024-09-21T06:24:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.686+0000 7fe37bfdf640 1 Processor -- start 2024-09-21T06:24:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.686+0000 7fe37bfdf640 1 -- start start 2024-09-21T06:24:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37bfdf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 0x7fe3740ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37bfdf640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 0x7fe3740ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37bfdf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe374108c60 0x7fe3740fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37bfdf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe374112920 con 0x7fe374108c60 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37bfdf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741127a0 con 0x7fe3741029e0 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37bfdf640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe374112aa0 con 0x7fe374108310 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37a57c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe374108c60 0x7fe3740fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37957a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 0x7fe3740ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 0x7fe3740ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 0x7fe3740ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48270/0 (socket says 172.21.15.31:48270) 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe37957a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 0x7fe3740ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42012/0 (socket says 172.21.15.31:42012) 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 -- 172.21.15.31:0/106327993 learned_addr learned my addr 172.21.15.31:0/106327993 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 -- 172.21.15.31:0/106327993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 msgr2=0x7fe3740ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 0x7fe3740ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 -- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe374108c60 msgr2=0x7fe3740fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe374108c60 0x7fe3740fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.687+0000 7fe379d7b640 1 -- 172.21.15.31:0/106327993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741a8740 con 0x7fe3741029e0 2024-09-21T06:24:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.688+0000 7fe379d7b640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 0x7fe3740ff550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe370005940 tx=0x7fe3700040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.688+0000 7fe36affd640 1 -- 172.21.15.31:0/106327993 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37003f030 con 0x7fe3741029e0 2024-09-21T06:24:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.688+0000 7fe36affd640 1 -- 172.21.15.31:0/106327993 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3700044c0 con 0x7fe3741029e0 2024-09-21T06:24:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.688+0000 7fe36affd640 1 -- 172.21.15.31:0/106327993 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370035070 con 0x7fe3741029e0 2024-09-21T06:24:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.688+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3741a89d0 con 0x7fe3741029e0 2024-09-21T06:24:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.688+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3741a8f10 con 0x7fe3741029e0 2024-09-21T06:24:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.691+0000 7fe36affd640 1 -- 172.21.15.31:0/106327993 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe370042030 con 0x7fe3741029e0 2024-09-21T06:24:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 Processor -- start 2024-09-21T06:24:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 -- start start 2024-09-21T06:24:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c61430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561420bd56f0 con 0x56142089a4d0 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561420bc4b20 con 0x5614208922f0 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe383026740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561420bc8930 con 0x5614208bb1b0 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe37a57c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.692+0000 7fe37957a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c61430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:42024/0 (socket says 172.21.15.31:42024) 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37957a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c61430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34828/0 (socket says 172.21.15.31:34828) 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 -- 172.21.15.31:0/117208826 learned_addr learned my addr 172.21.15.31:0/117208826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe379d7b640 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe36affd640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3600b6730 0x7fe3600b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 -- 172.21.15.31:0/117208826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 msgr2=0x561420c60ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c60ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 -- 172.21.15.31:0/117208826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 msgr2=0x561420c61430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c61430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 -- 172.21.15.31:0/117208826 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561420b8a530 con 0x5614208bb1b0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe379d7b640 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3600b6730 0x7fe3600b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe36affd640 1 -- 172.21.15.31:0/106327993 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fe3701079a0 con 0x7fe3741029e0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.693+0000 7fe37a57c640 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c609b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0115c0 tx=0x7fe36c011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503c7deea5eecb57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe35ffff640 1 -- 172.21.15.31:0/117208826 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c017050 con 0x5614208bb1b0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe37a57c640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3600b6730 0x7fe3600b8c40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe36c022000 tx=0x7fe36c023000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe35ffff640 1 -- 172.21.15.31:0/117208826 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36c0046f0 con 0x5614208bb1b0 2024-09-21T06:24:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe35ffff640 1 -- 172.21.15.31:0/117208826 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c01bb70 con 0x5614208bb1b0 2024-09-21T06:24:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 -- 172.21.15.31:0/117208826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 msgr2=0x561420c609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c609b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0115c0 tx=0x7fe36c011a80 comp rx=0 tx=0).stop 2024-09-21T06:24:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 -- 172.21.15.31:0/117208826 shutdown_connections 2024-09-21T06:24:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c61430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 --2- 172.21.15.31:0/117208826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c609b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.694+0000 7fe383026740 1 -- 172.21.15.31:0/117208826 >> 172.21.15.31:0/117208826 conn(0x5614208af300 msgr2=0x561420c5e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.695+0000 7fe383026740 1 -- 172.21.15.31:0/117208826 shutdown_connections 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.695+0000 7fe383026740 1 -- 172.21.15.31:0/117208826 wait complete. 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.695+0000 7fe383026740 1 -- start start 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.695+0000 7fe383026740 1 Processor -- start 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c6c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c78420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561420afcab0 con 0x56142089a4d0 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561420b6a730 con 0x5614208922f0 2024-09-21T06:24:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561420c6f130 con 0x5614208bb1b0 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe383026740 10 client.0 _ll_register_callbacks cb 0x561420bbe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c6c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c6c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34840/0 (socket says 172.21.15.31:34840) 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 -- 172.21.15.31:0/2926610391 learned_addr learned my addr 172.21.15.31:0/2926610391 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37a57c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c78420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c6c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48292/0 (socket says 172.21.15.31:48292) 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 -- 172.21.15.31:0/2926610391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 msgr2=0x561420c78420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c78420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 -- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 msgr2=0x561420c6c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c6c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 -- 172.21.15.31:0/2926610391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561420926790 con 0x56142089a4d0 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe379d7b640 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c6c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe37957a640 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c6c610 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe364012700 tx=0x7fe364012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961e7d63fefcd06f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe35d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe35d7fa640 1 -- 172.21.15.31:0/2926610391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364004020 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.696+0000 7fe35d7fa640 1 -- 172.21.15.31:0/2926610391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fe3640041c0 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36c0046f0 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe35d7fa640 1 -- 172.21.15.31:0/2926610391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe3640043c0 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 10 client.17955 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56142099f620 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe35d7fa640 1 -- 172.21.15.31:0/2926610391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36c0046f0 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe35d7fa640 1 -- 172.21.15.31:0/2926610391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe364003a10 con 0x56142089a4d0 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 10 client.17955 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 10 client.17955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 20 client.17955 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561420879d80 0x561420c7e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 4 client.17955 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561420bd7640 con 0x561420879d80 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.697+0000 7fe379d7b640 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561420879d80 0x561420c7e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.698+0000 7fe379d7b640 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561420879d80 0x561420c7e220 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b6cf4c1cef3cb3 server_cookie=b5c5f90659ab1706 in_seq=0 out_seq=0 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.698+0000 7fe35d7fa640 10 client.17955 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.698+0000 7fe35d7fa640 1 -- 172.21.15.31:0/2926610391 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561420bd7640 con 0x561420879d80 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.698+0000 7fe35d7fa640 10 client.17955 handle_command_reply: tid=0 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.698+0000 7fe383026740 1 client.17955 shutdown 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 msgr2=0x561420c6c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c6c610 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe364012700 tx=0x7fe364012bc0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 shutdown_connections 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561420879d80 0x561420c7e220 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208bb1b0 0x561420c78420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe35d7fa640 0 client.17955 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56142089a4d0 0x561420c6c610 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 --2- 172.21.15.31:0/2926610391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5614208922f0 0x561420c6c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 >> 172.21.15.31:0/2926610391 conn(0x5614208af300 msgr2=0x561420c619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 shutdown_connections 2024-09-21T06:24:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 1 -- 172.21.15.31:0/2926610391 wait complete. 2024-09-21T06:24:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.699+0000 7fe383026740 20 client.17955 trim_cache size 0 max 16384 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 Processor -- start 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 -- start start 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561420879d80 0x561420c6c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 0x561420c623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420c6daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741af990 con 0x5614208922f0 2024-09-21T06:24:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741b21f0 con 0x56142089a4d0 2024-09-21T06:24:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe383026740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741affe0 con 0x561420879d80 2024-09-21T06:24:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe37957a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 0x561420c623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561420879d80 0x561420c6c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe37957a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 0x561420c623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34842/0 (socket says 172.21.15.31:34842) 2024-09-21T06:24:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe37957a640 1 -- 172.21.15.31:0/256041533 learned_addr learned my addr 172.21.15.31:0/256041533 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.720+0000 7fe37a57c640 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420c6daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37957a640 1 -- 172.21.15.31:0/256041533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561420879d80 msgr2=0x561420c6c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37957a640 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561420879d80 0x561420c6c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37957a640 1 -- 172.21.15.31:0/256041533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 msgr2=0x561420c6daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37957a640 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420c6daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37957a640 1 -- 172.21.15.31:0/256041533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561420c59d50 con 0x5614208922f0 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37a57c640 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420c6daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe37957a640 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 0x561420c623a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe36400a010 tx=0x7fe36400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e7d14c6d1e10a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe35f7fe640 1 -- 172.21.15.31:0/256041533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364028030 con 0x5614208922f0 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe35f7fe640 1 -- 172.21.15.31:0/256041533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36400fdd0 con 0x5614208922f0 2024-09-21T06:24:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.721+0000 7fe35f7fe640 1 -- 172.21.15.31:0/256041533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364003a70 con 0x5614208922f0 2024-09-21T06:24:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 -- 172.21.15.31:0/256041533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 msgr2=0x561420c623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 0x561420c623a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe36400a010 tx=0x7fe36400f810 comp rx=0 tx=0).stop 2024-09-21T06:24:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 -- 172.21.15.31:0/256041533 shutdown_connections 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420c6daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5614208922f0 0x561420c623a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 --2- 172.21.15.31:0/256041533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561420879d80 0x561420c6c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.722+0000 7fe383026740 1 -- 172.21.15.31:0/256041533 >> 172.21.15.31:0/256041533 conn(0x5614208af300 msgr2=0x561420c6d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.723+0000 7fe383026740 1 -- 172.21.15.31:0/256041533 shutdown_connections 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.723+0000 7fe383026740 1 -- 172.21.15.31:0/256041533 wait complete. 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.723+0000 7fe383026740 1 -- start start 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.723+0000 7fe383026740 1 Processor -- start 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 0x561420c6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208922f0 0x561420c700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420ce6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741b06b0 con 0x561420879d80 2024-09-21T06:24:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741b1b20 con 0x56142089a4d0 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3741b0d80 con 0x5614208922f0 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 0x561420c6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe37957a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208922f0 0x561420c700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 0x561420c6fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34856/0 (socket says 172.21.15.31:34856) 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 -- 172.21.15.31:0/3230123148 learned_addr learned my addr 172.21.15.31:0/3230123148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe37a57c640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420ce6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe383026740 10 client.0 _ll_register_callbacks cb 0x7fe364009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 -- 172.21.15.31:0/3230123148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208922f0 msgr2=0x561420c700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208922f0 0x561420c700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 -- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 msgr2=0x561420ce6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420ce6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe379d7b640 1 -- 172.21.15.31:0/3230123148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561420ce7130 con 0x561420879d80 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.724+0000 7fe37957a640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208922f0 0x561420c700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe379d7b640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 0x561420c6fb80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe3700c9e60 tx=0x7fe37003b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a5e2cd1105dfd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe35d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe35d7fa640 1 -- 172.21.15.31:0/3230123148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370049040 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe35d7fa640 1 -- 172.21.15.31:0/3230123148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fe3700d6d70 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36400fdd0 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe35d7fa640 1 -- 172.21.15.31:0/3230123148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe3700cd660 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe383026740 10 client.17961 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561420ce8350 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe35d7fa640 1 -- 172.21.15.31:0/3230123148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36400fdd0 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe35d7fa640 1 -- 172.21.15.31:0/3230123148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe3700ceb10 con 0x561420879d80 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe383026740 10 client.17961 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe383026740 10 client.17961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.725+0000 7fe383026740 20 client.17961 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.726+0000 7fe383026740 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614208bb1b0 0x561420ceab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.726+0000 7fe383026740 4 client.17961 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:24:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.726+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561420bd30f0 con 0x5614208bb1b0 2024-09-21T06:24:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.726+0000 7fe37957a640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614208bb1b0 0x561420ceab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.726+0000 7fe35d7fa640 10 client.17961 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:08.726+0000 7fe37957a640 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614208bb1b0 0x561420ceab10 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389f5251652e3d70 server_cookie=d7fdf928e5ed6ee5 in_seq=0 out_seq=0 2024-09-21T06:24:08.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:08 smithi031 ceph-mon[31237]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T06:24:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:08 smithi110 ceph-mon[34550]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T06:24:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:08 smithi107 ceph-mon[34380]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T06:24:09.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:09 smithi031 ceph-mon[31237]: pgmap v710: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 526 op/s 2024-09-21T06:24:10.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:09 smithi110 ceph-mon[34550]: pgmap v710: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 526 op/s 2024-09-21T06:24:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:09 smithi107 ceph-mon[34380]: pgmap v710: 97 pgs: 97 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.6 MiB/s wr, 526 op/s 2024-09-21T06:24:10.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 750.0 s, 103.2 tps, lat 482.416 ms stddev 634.469 2024-09-21T06:24:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.363+0000 7fe35d7fa640 1 -- 172.21.15.31:0/3230123148 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561420bd30f0 con 0x5614208bb1b0 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.363+0000 7fe35d7fa640 10 client.17961 handle_command_reply: tid=0 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.363+0000 7fe383026740 1 client.17961 shutdown 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.363+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 msgr2=0x561420c6fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.363+0000 7fe383026740 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 0x561420c6fb80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe3700c9e60 tx=0x7fe37003b4f0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 shutdown_connections 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5614208bb1b0 0x561420ceab10 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe35d7fa640 0 client.17961 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56142089a4d0 0x561420ce6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614208922f0 0x561420c700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 --2- 172.21.15.31:0/3230123148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561420879d80 0x561420c6fb80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 >> 172.21.15.31:0/3230123148 conn(0x5614208af300 msgr2=0x561420c62170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 shutdown_connections 2024-09-21T06:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.364+0000 7fe383026740 1 -- 172.21.15.31:0/3230123148 wait complete. 2024-09-21T06:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.365+0000 7fe383026740 20 client.17961 trim_cache size 0 max 16384 2024-09-21T06:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:24:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3600b6730 msgr2=0x7fe3600b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3600b6730 0x7fe3600b8c40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe36c022000 tx=0x7fe36c023000 comp rx=0 tx=0).stop 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 msgr2=0x7fe3740ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 0x7fe3740ff550 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe370005940 tx=0x7fe3700040c0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 shutdown_connections 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe3600b6730 0x7fe3600b8c40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe374108c60 0x7fe3740fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe374108310 0x7fe3740ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 --2- 172.21.15.31:0/106327993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3741029e0 0x7fe3740ff550 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.366+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 >> 172.21.15.31:0/106327993 conn(0x7fe37407a6e0 msgr2=0x7fe3741019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.367+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 shutdown_connections 2024-09-21T06:24:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:11.367+0000 7fe37bfdf640 1 -- 172.21.15.31:0/106327993 wait complete. 2024-09-21T06:24:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:11 smithi031 ceph-mon[31237]: pgmap v711: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 961 op/s 2024-09-21T06:24:12.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:11 smithi110 ceph-mon[34550]: pgmap v711: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 961 op/s 2024-09-21T06:24:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:11 smithi107 ceph-mon[34380]: pgmap v711: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 961 op/s 2024-09-21T06:24:12.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:24:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3799460882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 msgr2=0x7f2aa8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3799460882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa8102e70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009900 tx=0x7f2a9802a410 comp rx=0 tx=0).stop 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3799460882 shutdown_connections 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3799460882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa8108cf0 0x7f2aa810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3799460882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa81083a0 0x7f2aa81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3799460882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa8102e70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3799460882 >> 172.21.15.31:0/3799460882 conn(0x7f2aa8079730 msgr2=0x7f2aa8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3799460882 shutdown_connections 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3799460882 wait complete. 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.667+0000 7f2aae5fb640 1 Processor -- start 2024-09-21T06:24:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 -- start start 2024-09-21T06:24:13.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa81083a0 0x7f2aa819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa8108cf0 0x7f2aa819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8112960 con 0x7f2aa8102a60 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81127e0 con 0x7f2aa81083a0 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aae5fb640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8112ae0 con 0x7f2aa8108cf0 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa81083a0 0x7f2aa819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48384/0 (socket says 172.21.15.31:48384) 2024-09-21T06:24:13.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.668+0000 7f2aa7fff640 1 -- 172.21.15.31:0/3911319275 learned_addr learned my addr 172.21.15.31:0/3911319275 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aacb98640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa8108cf0 0x7f2aa819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa77fe640 1 -- 172.21.15.31:0/3911319275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa8108cf0 msgr2=0x7f2aa819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa8108cf0 0x7f2aa819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa77fe640 1 -- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 msgr2=0x7f2aa819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa77fe640 1 -- 172.21.15.31:0/3911319275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa81a6500 con 0x7f2aa81083a0 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aacb98640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa8108cf0 0x7f2aa819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa81083a0 0x7f2aa819b7b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e9c0 tx=0x7f2a9c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa57fa640 1 -- 172.21.15.31:0/3911319275 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00ccc0 con 0x7f2aa81083a0 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa57fa640 1 -- 172.21.15.31:0/3911319275 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c0108e0 con 0x7f2aa81083a0 2024-09-21T06:24:13.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa81990c0 con 0x7f2aa81083a0 2024-09-21T06:24:13.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aa57fa640 1 -- 172.21.15.31:0/3911319275 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0187c0 con 0x7f2aa81083a0 2024-09-21T06:24:13.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.669+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa8199650 con 0x7f2aa81083a0 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 Processor -- start 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 -- start start 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082d7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 0x558b082d77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558b0824de50 con 0x558b07f0f390 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558b0823d280 con 0x558b07f071b0 2024-09-21T06:24:13.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558b08241090 con 0x558b07f30070 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.672+0000 7f2aa57fa640 1 -- 172.21.15.31:0/3911319275 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2a9c010430 con 0x7f2aa81083a0 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 0x558b082d77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aacb98640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 0x558b082d77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:40230/0 (socket says 172.21.15.31:40230) 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2744670737 learned_addr learned my addr 172.21.15.31:0/2744670737 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2744670737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 msgr2=0x558b082d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2744670737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 msgr2=0x558b082d7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082d7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2744670737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b07f9b650 con 0x558b07f071b0 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aacb98640 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.673+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 0x558b082d77b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006010 tx=0x7f2a980058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfaa1c7b13c2920e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.674+0000 7f2aa57fa640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2a900b6730 0x7f2a900b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.674+0000 7f2a8e7fc640 1 -- 172.21.15.31:0/2744670737 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98035040 con 0x558b07f071b0 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.674+0000 7f2a8e7fc640 1 -- 172.21.15.31:0/2744670737 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9803ecf0 con 0x558b07f071b0 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.674+0000 7f2a8e7fc640 1 -- 172.21.15.31:0/2744670737 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98005310 con 0x558b07f071b0 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.674+0000 7f2aacb98640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2a900b6730 0x7f2a900b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.674+0000 7f2aa57fa640 1 -- 172.21.15.31:0/3911319275 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f2a9c0e7af0 con 0x7f2aa81083a0 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2744670737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 msgr2=0x558b082d77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2aacb98640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2a900b6730 0x7f2a900b8c40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0005fd0 tx=0x7f2aa0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 0x558b082d77b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006010 tx=0x7f2a980058d0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2744670737 shutdown_connections 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082d7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f071b0 0x558b082d77b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2744670737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2744670737 >> 172.21.15.31:0/2744670737 conn(0x558b07f24460 msgr2=0x558b082d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2744670737 shutdown_connections 2024-09-21T06:24:13.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2744670737 wait complete. 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.675+0000 7f2ab4b6f740 1 -- start start 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 Processor -- start 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 0x558b082e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f30070 0x558b082eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558b0809e700 con 0x558b07f30070 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558b081d8110 con 0x558b07f0f390 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558b082e5990 con 0x558b07f071b0 2024-09-21T06:24:13.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2ab4b6f740 10 client.0 _ll_register_callbacks cb 0x558b081f1990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 0x558b082e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2aacb98640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f30070 0x558b082eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 0x558b082e2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:51172/0 (socket says 172.21.15.31:51172) 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2503974335 learned_addr learned my addr 172.21.15.31:0/2503974335 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.676+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 msgr2=0x558b082e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2503974335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f30070 msgr2=0x558b082eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f30070 0x558b082eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa7fff640 1 -- 172.21.15.31:0/2503974335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b08205c40 con 0x558b07f071b0 2024-09-21T06:24:13.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:13.684 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:13 smithi031 ceph-mon[31237]: pgmap v712: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.44k op/s 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 0x558b082e2910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2a98031a00 tx=0x7f2a9802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d142a8ad9998f1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2a877fe640 1 -- 172.21.15.31:0/2503974335 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98058070 con 0x558b07f071b0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2a877fe640 1 -- 172.21.15.31:0/2503974335 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f2a980052a0 con 0x558b07f071b0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.677+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9803ecf0 con 0x558b07f071b0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2a877fe640 1 -- 172.21.15.31:0/2503974335 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a98062050 con 0x558b07f071b0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 10 client.27805 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558b082f16b0 con 0x558b07f071b0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2a877fe640 1 -- 172.21.15.31:0/2503974335 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9803ecf0 con 0x558b07f071b0 2024-09-21T06:24:13.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2a877fe640 1 -- 172.21.15.31:0/2503974335 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2a9805d440 con 0x558b07f071b0 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 10 client.27805 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 10 client.27805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 20 client.27805 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b07eee9f0 0x558b082f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 4 client.27805 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.678+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b0824fda0 con 0x558b07eee9f0 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.679+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b07eee9f0 0x558b082f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.679+0000 7f2a877fe640 10 client.27805 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.679+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b07eee9f0 0x558b082f4d10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8d6f994cff94d3 server_cookie=fcc98b9fb4b911af in_seq=0 out_seq=0 2024-09-21T06:24:13.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.680+0000 7f2a877fe640 1 -- 172.21.15.31:0/2503974335 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558b0824fda0 con 0x558b07eee9f0 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.680+0000 7f2a877fe640 10 client.27805 handle_command_reply: tid=0 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.680+0000 7f2ab4b6f740 1 client.27805 shutdown 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.680+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 msgr2=0x558b082e2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.680+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 0x558b082e2910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2a98031a00 tx=0x7f2a9802a410 comp rx=0 tx=0).stop 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 shutdown_connections 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b07eee9f0 0x558b082f4d10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f30070 0x558b082eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/2503974335 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f071b0 0x558b082e2910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 >> 172.21.15.31:0/2503974335 conn(0x558b07f24460 msgr2=0x558b082da400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2a877fe640 0 client.27805 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 shutdown_connections 2024-09-21T06:24:13.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/2503974335 wait complete. 2024-09-21T06:24:13.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.681+0000 7f2ab4b6f740 20 client.27805 trim_cache size 0 max 16384 2024-09-21T06:24:13.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 Processor -- start 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 -- start start 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082e2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07ec3da0 0x558b082dfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ab530 con 0x558b07f0f390 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81add90 con 0x558b07ec3da0 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.699+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81abb80 con 0x558b07f30070 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082e2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082e2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48416/0 (socket says 172.21.15.31:48416) 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 -- 172.21.15.31:0/3408689110 learned_addr learned my addr 172.21.15.31:0/3408689110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aacb98640 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07ec3da0 0x558b082dfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 -- 172.21.15.31:0/3408689110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 msgr2=0x558b082eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 -- 172.21.15.31:0/3408689110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07ec3da0 msgr2=0x558b082dfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07ec3da0 0x558b082dfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 -- 172.21.15.31:0/3408689110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b082e1870 con 0x558b07f0f390 2024-09-21T06:24:13.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082e2dd0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2a9805c3d0 tx=0x7f2a98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bc96247d59c6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2a8dffb640 1 -- 172.21.15.31:0/3408689110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98058050 con 0x558b07f0f390 2024-09-21T06:24:13.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2a8dffb640 1 -- 172.21.15.31:0/3408689110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9805de50 con 0x558b07f0f390 2024-09-21T06:24:13.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.700+0000 7f2a8dffb640 1 -- 172.21.15.31:0/3408689110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98035070 con 0x558b07f0f390 2024-09-21T06:24:13.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/3408689110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 msgr2=0x558b082e2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082e2dd0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2a9805c3d0 tx=0x7f2a98000bc0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/3408689110 shutdown_connections 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07ec3da0 0x558b082dfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b082eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/3408689110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07f0f390 0x558b082e2dd0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/3408689110 >> 172.21.15.31:0/3408689110 conn(0x558b07f24460 msgr2=0x558b082e26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/3408689110 shutdown_connections 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.701+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/3408689110 wait complete. 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 -- start start 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 Processor -- start 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 0x558b082e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b083577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ac250 con 0x558b07ec3da0 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ad6c0 con 0x558b07f0f390 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ac920 con 0x558b07f30070 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2ab4b6f740 10 client.0 _ll_register_callbacks cb 0x7f2a9c0eb470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 0x558b082e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 0x558b082e7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48420/0 (socket says 172.21.15.31:48420) 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.702+0000 7f2aa7fff640 1 -- 172.21.15.31:0/4107261158 learned_addr learned my addr 172.21.15.31:0/4107261158 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aacb98640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b083577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa7fff640 1 -- 172.21.15.31:0/4107261158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 msgr2=0x558b083577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b083577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa7fff640 1 -- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 msgr2=0x558b082e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa7fff640 1 -- 172.21.15.31:0/4107261158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b08357e20 con 0x558b07ec3da0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2aa7fff640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 0x558b082e7b60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2a980658e0 tx=0x7f2a98005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d3537474cc0852 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2a877fe640 1 -- 172.21.15.31:0/4107261158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98076060 con 0x558b07ec3da0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2a877fe640 1 -- 172.21.15.31:0/4107261158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f2a9803ea70 con 0x558b07ec3da0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9805de50 con 0x558b07ec3da0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2a877fe640 1 -- 172.21.15.31:0/4107261158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2a98035070 con 0x558b07ec3da0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2ab4b6f740 10 client.17979 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.703+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558b0835f170 con 0x558b07ec3da0 2024-09-21T06:24:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2a877fe640 1 -- 172.21.15.31:0/4107261158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9805de50 con 0x558b07ec3da0 2024-09-21T06:24:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2a877fe640 1 -- 172.21.15.31:0/4107261158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2a98070070 con 0x558b07ec3da0 2024-09-21T06:24:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2ab4b6f740 10 client.17979 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:13.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2ab4b6f740 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2ab4b6f740 20 client.17979 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b0835f800 0x558b08361c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2ab4b6f740 4 client.17979 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b0824b850 con 0x558b0835f800 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.704+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b0835f800 0x558b08361c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2a877fe640 10 client.17979 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2aa77fe640 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b0835f800 0x558b08361c20 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d5b68db63437c8 server_cookie=52faf785693638ce in_seq=0 out_seq=0 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2a877fe640 1 -- 172.21.15.31:0/4107261158 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8326 (crc 0 0 0) 0x558b0824b850 con 0x558b0835f800 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2a877fe640 10 client.17979 handle_command_reply: tid=0 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2ab4b6f740 1 client.17979 shutdown 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 msgr2=0x558b082e7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 0x558b082e7b60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2a980658e0 tx=0x7f2a98005fd0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.705+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 shutdown_connections 2024-09-21T06:24:13.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558b0835f800 0x558b08361c20 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558b07f30070 0x558b083577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558b07f0f390 0x558b082e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 --2- 172.21.15.31:0/4107261158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558b07ec3da0 0x558b082e7b60 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 >> 172.21.15.31:0/4107261158 conn(0x558b07f24460 msgr2=0x558b082d82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2a877fe640 0 client.17979 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 shutdown_connections 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 1 -- 172.21.15.31:0/4107261158 wait complete. 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.706+0000 7f2ab4b6f740 20 client.17979 trim_cache size 0 max 16384 2024-09-21T06:24:13.712 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:24:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:24:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:24:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:24:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:24:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:13.730 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2a900b6730 msgr2=0x7f2a900b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2a900b6730 0x7f2a900b8c40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0005fd0 tx=0x7f2aa0009500 comp rx=0 tx=0).stop 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa81083a0 msgr2=0x7f2aa819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa81083a0 0x7f2aa819b7b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e9c0 tx=0x7f2a9c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 shutdown_connections 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2aa8108cf0 0x7f2aa819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa81083a0 0x7f2aa819b7b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2a900b6730 0x7f2a900b8c40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 --2- 172.21.15.31:0/3911319275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2aa8102a60 0x7f2aa819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 >> 172.21.15.31:0/3911319275 conn(0x7f2aa8079730 msgr2=0x7f2aa81013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:13.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 shutdown_connections 2024-09-21T06:24:13.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:13.707+0000 7f2aae5fb640 1 -- 172.21.15.31:0/3911319275 wait complete. 2024-09-21T06:24:14.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:13 smithi110 ceph-mon[34550]: pgmap v712: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.44k op/s 2024-09-21T06:24:14.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:13 smithi107 ceph-mon[34380]: pgmap v712: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 4.5 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 19 MiB/s wr, 1.44k op/s 2024-09-21T06:24:14.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:24:14.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:24:14.955 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 755.0 s, 385.4 tps, lat 394.587 ms stddev 1130.087 2024-09-21T06:24:15.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 -- 172.21.15.31:0/3316733718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f54102a60 msgr2=0x7f2f54102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 --2- 172.21.15.31:0/3316733718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f54102a60 0x7f2f54102e70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009900 tx=0x7f2f4402a410 comp rx=0 tx=0).stop 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 -- 172.21.15.31:0/3316733718 shutdown_connections 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 --2- 172.21.15.31:0/3316733718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f54108cf0 0x7f2f5410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 --2- 172.21.15.31:0/3316733718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f541083a0 0x7f2f541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 --2- 172.21.15.31:0/3316733718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f54102a60 0x7f2f54102e70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 -- 172.21.15.31:0/3316733718 >> 172.21.15.31:0/3316733718 conn(0x7f2f54079730 msgr2=0x7f2f54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 -- 172.21.15.31:0/3316733718 shutdown_connections 2024-09-21T06:24:15.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.527+0000 7f2f59f07640 1 -- 172.21.15.31:0/3316733718 wait complete. 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.528+0000 7f2f59f07640 1 Processor -- start 2024-09-21T06:24:15.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.528+0000 7f2f59f07640 1 -- start start 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f59f07640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f54102a60 0x7f2f5419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f59f07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 0x7f2f5419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f59f07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f54108cf0 0x7f2f5419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f59f07640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113a80 con 0x7f2f541083a0 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f59f07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113900 con 0x7f2f54108cf0 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f59f07640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54113c00 con 0x7f2f54102a60 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f52ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 0x7f2f5419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f52ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 0x7f2f5419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48438/0 (socket says 172.21.15.31:48438) 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f52ffd640 1 -- 172.21.15.31:0/195181666 learned_addr learned my addr 172.21.15.31:0/195181666 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f537fe640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f54102a60 0x7f2f5419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f53fff640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f54108cf0 0x7f2f5419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f537fe640 1 -- 172.21.15.31:0/195181666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f54108cf0 msgr2=0x7f2f5419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f537fe640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f54108cf0 0x7f2f5419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f537fe640 1 -- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 msgr2=0x7f2f5419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.529+0000 7f2f537fe640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 0x7f2f5419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.530+0000 7f2f537fe640 1 -- 172.21.15.31:0/195181666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f541a0480 con 0x7f2f54102a60 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.530+0000 7f2f52ffd640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 0x7f2f5419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.531+0000 7f2f537fe640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f54102a60 0x7f2f5419a510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2f4402a930 tx=0x7f2f44031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.531+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4403f030 con 0x7f2f54102a60 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.531+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f44004490 con 0x7f2f54102a60 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.531+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44035070 con 0x7f2f54102a60 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.531+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f541a0710 con 0x7f2f54102a60 2024-09-21T06:24:15.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.531+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541a6ed0 con 0x7f2f54102a60 2024-09-21T06:24:15.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.532+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f54102e70 con 0x7f2f54102a60 2024-09-21T06:24:15.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.534+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2f44042030 con 0x7f2f54102a60 2024-09-21T06:24:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.535+0000 7f2f50ff9640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f3c0b6700 0x7f2f3c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:15.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.536+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f2f44107f70 con 0x7f2f54102a60 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.536+0000 7f2f52ffd640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f3c0b6700 0x7f2f3c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:15.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.537+0000 7f2f52ffd640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f3c0b6700 0x7f2f3c0b8c10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f54100be0 tx=0x7f2f48005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:15.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.538+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f44049030 con 0x7f2f54102a60 2024-09-21T06:24:15.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:15 smithi031 ceph-mon[31237]: pgmap v713: 97 pgs: 19 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 47 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2024-09-21T06:24:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.888+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f5410aac0 con 0x7f2f54102a60 2024-09-21T06:24:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.889+0000 7f2f50ff9640 1 -- 172.21.15.31:0/195181666 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f2f440ce450 con 0x7f2f54102a60 2024-09-21T06:24:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:24:15.892 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:24:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:24:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f3c0b6700 msgr2=0x7f2f3c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f3c0b6700 0x7f2f3c0b8c10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2f54100be0 tx=0x7f2f48005f90 comp rx=0 tx=0).stop 2024-09-21T06:24:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f54102a60 msgr2=0x7f2f5419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f54102a60 0x7f2f5419a510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2f4402a930 tx=0x7f2f44031a20 comp rx=0 tx=0).stop 2024-09-21T06:24:15.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 shutdown_connections 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f54108cf0 0x7f2f5419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2f3c0b6700 0x7f2f3c0b8c10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f541083a0 0x7f2f5419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.894+0000 7f2f59f07640 1 --2- 172.21.15.31:0/195181666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f54102a60 0x7f2f5419a510 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.895+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 >> 172.21.15.31:0/195181666 conn(0x7f2f54079730 msgr2=0x7f2f541014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.895+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 shutdown_connections 2024-09-21T06:24:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:15.895+0000 7f2f59f07640 1 -- 172.21.15.31:0/195181666 wait complete. 2024-09-21T06:24:16.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:15 smithi110 ceph-mon[34550]: pgmap v713: 97 pgs: 19 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 47 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2024-09-21T06:24:16.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:15 smithi107 ceph-mon[34380]: pgmap v713: 97 pgs: 19 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 47 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2024-09-21T06:24:16.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:24:16.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:16 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/195181666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:17.000 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:17.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:16 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/195181666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:17.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:16 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/195181666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.609+0000 7fdeaa486640 1 -- 172.21.15.31:0/593272501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea411cc30 msgr2=0x7fdea411f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.609+0000 7fdeaa486640 1 --2- 172.21.15.31:0/593272501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea411cc30 0x7fdea411f140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fde98009860 tx=0x7fde9802a380 comp rx=0 tx=0).stop 2024-09-21T06:24:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.610+0000 7fdeaa486640 1 -- 172.21.15.31:0/593272501 shutdown_connections 2024-09-21T06:24:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.610+0000 7fdeaa486640 1 --2- 172.21.15.31:0/593272501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea411cc30 0x7fdea411f140 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.610+0000 7fdeaa486640 1 --2- 172.21.15.31:0/593272501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdea410a9e0 0x7fdea410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.610+0000 7fdeaa486640 1 --2- 172.21.15.31:0/593272501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.610+0000 7fdeaa486640 1 -- 172.21.15.31:0/593272501 >> 172.21.15.31:0/593272501 conn(0x7fdea4070ec0 msgr2=0x7fdea40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.613+0000 7fdeaa486640 1 -- 172.21.15.31:0/593272501 shutdown_connections 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.615+0000 7fdeaa486640 1 -- 172.21.15.31:0/593272501 wait complete. 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.616+0000 7fdeaa486640 1 Processor -- start 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.617+0000 7fdeaa486640 1 -- start start 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.618+0000 7fdeaa486640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea410a9e0 0x7fdea41a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.618+0000 7fdeaa486640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea41a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.618+0000 7fdeaa486640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdea411cc30 0x7fdea41aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.618+0000 7fdeaa486640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdea41228b0 con 0x7fdea4111f40 2024-09-21T06:24:17.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.618+0000 7fdeaa486640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdea4122730 con 0x7fdea410a9e0 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.618+0000 7fdeaa486640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdea4122a30 con 0x7fdea411cc30 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.619+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea41a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.619+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea41a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48454/0 (socket says 172.21.15.31:48454) 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.619+0000 7fdea37fe640 1 -- 172.21.15.31:0/3627279554 learned_addr learned my addr 172.21.15.31:0/3627279554 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.619+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea410a9e0 0x7fdea41a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.620+0000 7fdea37fe640 1 -- 172.21.15.31:0/3627279554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdea411cc30 msgr2=0x7fdea41aeaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.620+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdea411cc30 0x7fdea41aeaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.620+0000 7fdea37fe640 1 -- 172.21.15.31:0/3627279554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea410a9e0 msgr2=0x7fdea41a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.620+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea410a9e0 0x7fdea41a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.620+0000 7fdea37fe640 1 -- 172.21.15.31:0/3627279554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea41af1d0 con 0x7fdea4111f40 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.620+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea41a9800 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e9c0 tx=0x7fde9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.622+0000 7fdea17fa640 1 -- 172.21.15.31:0/3627279554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9400ccc0 con 0x7fdea4111f40 2024-09-21T06:24:17.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.622+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea41af4c0 con 0x7fdea4111f40 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.623+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea41b5c20 con 0x7fdea4111f40 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.623+0000 7fdea17fa640 1 -- 172.21.15.31:0/3627279554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde940035f0 con 0x7fdea4111f40 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.623+0000 7fdea17fa640 1 -- 172.21.15.31:0/3627279554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9400ce60 con 0x7fdea4111f40 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.640+0000 7fdeb09fa740 1 Processor -- start 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.642+0000 7fdeb09fa740 1 -- start start 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.642+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101a1070 0x55a3105486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.642+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310548be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.643+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310549120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.643+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a3104beea0 con 0x55a3101781b0 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.643+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a3104ae2d0 con 0x55a3101a1070 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.643+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a3104b20e0 con 0x55a310180390 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.644+0000 7fdea17fa640 1 -- 172.21.15.31:0/3627279554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fde94004000 con 0x7fdea4111f40 2024-09-21T06:24:17.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.645+0000 7fdea17fa640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde8c0b6800 0x7fde8c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.646+0000 7fdea17fa640 1 -- 172.21.15.31:0/3627279554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fde9401fdc0 con 0x7fdea4111f40 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.646+0000 7fdea8a23640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101a1070 0x55a3105486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.646+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310548be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.646+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310549120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.646+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310548be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48464/0 (socket says 172.21.15.31:48464) 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.646+0000 7fdea3fff640 1 -- 172.21.15.31:0/140914376 learned_addr learned my addr 172.21.15.31:0/140914376 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.647+0000 7fdea3fff640 1 -- 172.21.15.31:0/140914376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 msgr2=0x55a310549120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.647+0000 7fdea3fff640 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310549120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.647+0000 7fdea3fff640 1 -- 172.21.15.31:0/140914376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101a1070 msgr2=0x55a3105486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.647+0000 7fdea3fff640 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101a1070 0x55a3105486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.647+0000 7fdea3fff640 1 -- 172.21.15.31:0/140914376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31020c650 con 0x55a3101781b0 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.647+0000 7fdea3fff640 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310548be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fde9c010480 tx=0x7fde9c010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c0c9aa88ef4e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.648+0000 7fde8a7fc640 1 -- 172.21.15.31:0/140914376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c017070 con 0x55a3101781b0 2024-09-21T06:24:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.648+0000 7fde8a7fc640 1 -- 172.21.15.31:0/140914376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9c011c10 con 0x55a3101781b0 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.649+0000 7fdea8a23640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde8c0b6800 0x7fde8c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 -- 172.21.15.31:0/140914376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 msgr2=0x55a310548be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310548be0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fde9c010480 tx=0x7fde9c010940 comp rx=0 tx=0).stop 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdea8a23640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde8c0b6800 0x7fde8c0b8d10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fde9802a970 tx=0x7fde98005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 -- 172.21.15.31:0/140914376 shutdown_connections 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310549120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310548be0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/140914376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101a1070 0x55a3105486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 -- 172.21.15.31:0/140914376 >> 172.21.15.31:0/140914376 conn(0x55a310195460 msgr2=0x55a310546590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 -- 172.21.15.31:0/140914376 shutdown_connections 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.650+0000 7fdeb09fa740 1 -- 172.21.15.31:0/140914376 wait complete. 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.651+0000 7fdeb09fa740 1 -- start start 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.651+0000 7fdeb09fa740 1 Processor -- start 2024-09-21T06:24:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a3101781b0 0x55a310553be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 0x55a310554120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101a1070 0x55a310554660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a31030f700 con 0x55a3101a1070 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a310449160 con 0x55a310180390 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a310556ed0 con 0x55a3101781b0 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.652+0000 7fdeb09fa740 10 client.0 _ll_register_callbacks cb 0x55a3104629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.653+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 0x55a310554120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.653+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 0x55a310554120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:40294/0 (socket says 172.21.15.31:40294) 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.653+0000 7fdea37fe640 1 -- 172.21.15.31:0/3653654476 learned_addr learned my addr 172.21.15.31:0/3653654476 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.653+0000 7fdea8a23640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101a1070 0x55a310554660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a3101781b0 0x55a310553be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea37fe640 1 -- 172.21.15.31:0/3653654476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a3101781b0 msgr2=0x55a310553be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a3101781b0 0x55a310553be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea37fe640 1 -- 172.21.15.31:0/3653654476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101a1070 msgr2=0x55a310554660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101a1070 0x55a310554660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea37fe640 1 -- 172.21.15.31:0/3653654476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a310476c90 con 0x55a310180390 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.654+0000 7fdea8a23640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101a1070 0x55a310554660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.655+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 0x55a310554120 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fde94009230 tx=0x7fde940b6bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15d479b209e5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.655+0000 7fde837fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.655+0000 7fde837fe640 1 -- 172.21.15.31:0/3653654476 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940a9e80 con 0x55a310180390 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.656+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9c011c10 con 0x55a310180390 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.656+0000 7fde837fe640 1 -- 172.21.15.31:0/3653654476 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fde9401fdc0 con 0x55a310180390 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.656+0000 7fde837fe640 1 -- 172.21.15.31:0/3653654476 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fde940a9e80 con 0x55a310180390 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.656+0000 7fdeb09fa740 10 client.27773 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a310562ae0 con 0x55a310180390 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fde837fe640 1 -- 172.21.15.31:0/3653654476 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9c011c10 con 0x55a310180390 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fde837fe640 1 -- 172.21.15.31:0/3653654476 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fde940c8ce0 con 0x55a310180390 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 10 client.27773 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 10 client.27773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 20 client.27773 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a31015f9f0 0x55a310566370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 4 client.27773 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.657+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3104c0df0 con 0x55a31015f9f0 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.658+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a31015f9f0 0x55a310566370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.660+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a31015f9f0 0x55a310566370 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95f95bd27413a7ba server_cookie=f59aa85622dc78ec in_seq=0 out_seq=0 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.660+0000 7fde837fe640 10 client.27773 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.661+0000 7fde837fe640 1 -- 172.21.15.31:0/3653654476 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a3104c0df0 con 0x55a31015f9f0 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.661+0000 7fde837fe640 10 client.27773 handle_command_reply: tid=0 2024-09-21T06:24:17.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.661+0000 7fdeb09fa740 1 client.27773 shutdown 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.661+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 msgr2=0x55a310554120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.661+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 0x55a310554120 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fde94009230 tx=0x7fde940b6bb0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 shutdown_connections 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a31015f9f0 0x55a310566370 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fde837fe640 0 client.27773 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101a1070 0x55a310554660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a310180390 0x55a310554120 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3653654476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a3101781b0 0x55a310553be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 >> 172.21.15.31:0/3653654476 conn(0x55a310195460 msgr2=0x55a31054b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 shutdown_connections 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.662+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3653654476 wait complete. 2024-09-21T06:24:17.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.663+0000 7fdeb09fa740 20 client.27773 trim_cache size 0 max 16384 2024-09-21T06:24:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.681+0000 7fdeb09fa740 1 Processor -- start 2024-09-21T06:24:17.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.681+0000 7fdeb09fa740 1 -- start start 2024-09-21T06:24:17.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 0x55a310555810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101781b0 0x55a310550dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310551310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000caf0 con 0x55a31015f9f0 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000f3d0 con 0x55a3101781b0 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000d1c0 con 0x55a310180390 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 0x55a310555810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea8a23640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310551310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 0x55a310555810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48480/0 (socket says 172.21.15.31:48480) 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 -- 172.21.15.31:0/34563006 learned_addr learned my addr 172.21.15.31:0/34563006 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea37fe640 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101781b0 0x55a310550dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 -- 172.21.15.31:0/34563006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 msgr2=0x55a310551310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310551310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 -- 172.21.15.31:0/34563006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101781b0 msgr2=0x55a310550dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101781b0 0x55a310550dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.682+0000 7fdea3fff640 1 -- 172.21.15.31:0/34563006 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3105549a0 con 0x55a31015f9f0 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.683+0000 7fdea3fff640 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 0x55a310555810 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fde9c010e40 tx=0x7fde9c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2de8f4917aa8ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.683+0000 7fde89ffb640 1 -- 172.21.15.31:0/34563006 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c017040 con 0x55a31015f9f0 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.683+0000 7fde89ffb640 1 -- 172.21.15.31:0/34563006 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9c003970 con 0x55a31015f9f0 2024-09-21T06:24:17.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.683+0000 7fde89ffb640 1 -- 172.21.15.31:0/34563006 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c0117a0 con 0x55a31015f9f0 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 -- 172.21.15.31:0/34563006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 msgr2=0x55a310555810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 0x55a310555810 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fde9c010e40 tx=0x7fde9c009c40 comp rx=0 tx=0).stop 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 -- 172.21.15.31:0/34563006 shutdown_connections 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310551310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a3101781b0 0x55a310550dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/34563006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a31015f9f0 0x55a310555810 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 -- 172.21.15.31:0/34563006 >> 172.21.15.31:0/34563006 conn(0x55a310195460 msgr2=0x55a310555100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 -- 172.21.15.31:0/34563006 shutdown_connections 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 -- 172.21.15.31:0/34563006 wait complete. 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.684+0000 7fdeb09fa740 1 -- start start 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.685+0000 7fdeb09fa740 1 Processor -- start 2024-09-21T06:24:17.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.685+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a31015f9f0 0x55a31055cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.685+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310557fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.685+0000 7fdeb09fa740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310558660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdeb09fa740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000d890 con 0x55a3101781b0 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310557fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310557fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48492/0 (socket says 172.21.15.31:48492) 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea37fe640 1 -- 172.21.15.31:0/3760301712 learned_addr learned my addr 172.21.15.31:0/3760301712 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000ed00 con 0x55a31015f9f0 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a31015f9f0 0x55a31055cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000df60 con 0x55a310180390 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea8a23640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310558660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdeb09fa740 10 client.0 _ll_register_callbacks cb 0x7fde9401fd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea3fff640 1 -- 172.21.15.31:0/3760301712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 msgr2=0x55a310558660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310558660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea3fff640 1 -- 172.21.15.31:0/3760301712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 msgr2=0x55a310557fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310557fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.686+0000 7fdea3fff640 1 -- 172.21.15.31:0/3760301712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a310558dd0 con 0x55a31015f9f0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.687+0000 7fdea3fff640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a31015f9f0 0x55a31055cb10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fde9c01d040 tx=0x7fde9c003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2dfc9a2b8e27d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.687+0000 7fde837fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.687+0000 7fde837fe640 1 -- 172.21.15.31:0/3760301712 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c017050 con 0x55a31015f9f0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.687+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9c003970 con 0x55a31015f9f0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.687+0000 7fde837fe640 1 -- 172.21.15.31:0/3760301712 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fde9c033070 con 0x55a31015f9f0 2024-09-21T06:24:17.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.688+0000 7fde837fe640 1 -- 172.21.15.31:0/3760301712 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fde9c0049e0 con 0x55a31015f9f0 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.688+0000 7fdeb09fa740 10 client.27779 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.688+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a3105d0d70 con 0x55a31015f9f0 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.688+0000 7fde837fe640 1 -- 172.21.15.31:0/3760301712 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9c003970 con 0x55a31015f9f0 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fde837fe640 1 -- 172.21.15.31:0/3760301712 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fde9c0119f0 con 0x55a31015f9f0 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdeb09fa740 10 client.27779 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdeb09fa740 10 client.27779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdeb09fa740 20 client.27779 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a3101a1070 0x55a3105d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdeb09fa740 4 client.27779 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a31054af50 con 0x55a3101a1070 2024-09-21T06:24:17.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a3101a1070 0x55a3105d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.689+0000 7fde837fe640 10 client.27779 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fdea37fe640 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a3101a1070 0x55a3105d3180 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d15ef63e1ea145 server_cookie=7c331c970e69504a in_seq=0 out_seq=0 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fde837fe640 1 -- 172.21.15.31:0/3760301712 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a31054af50 con 0x55a3101a1070 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fde837fe640 10 client.27779 handle_command_reply: tid=0 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fdeb09fa740 1 client.27779 shutdown 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a31015f9f0 msgr2=0x55a31055cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a31015f9f0 0x55a31055cb10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fde9c01d040 tx=0x7fde9c003bd0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.690+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 shutdown_connections 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a3101a1070 0x55a3105d3180 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a310180390 0x55a310558660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a3101781b0 0x55a310557fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fde837fe640 0 client.27779 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:17.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 --2- 172.21.15.31:0/3760301712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a31015f9f0 0x55a31055cb10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 >> 172.21.15.31:0/3760301712 conn(0x55a310195460 msgr2=0x55a3105520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 shutdown_connections 2024-09-21T06:24:17.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 1 -- 172.21.15.31:0/3760301712 wait complete. 2024-09-21T06:24:17.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeb09fa740 20 client.27779 trim_cache size 0 max 16384 2024-09-21T06:24:17.695 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:24:17.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.691+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde8c0b6800 msgr2=0x7fde8c0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde8c0b6800 0x7fde8c0b8d10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fde9802a970 tx=0x7fde98005f90 comp rx=0 tx=0).stop 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 msgr2=0x7fdea41a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea41a9800 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e9c0 tx=0x7fde9400ee80 comp rx=0 tx=0).stop 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 shutdown_connections 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdea411cc30 0x7fdea41aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fde8c0b6800 0x7fde8c0b8d10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea4111f40 0x7fdea41a9800 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 --2- 172.21.15.31:0/3627279554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdea410a9e0 0x7fdea41a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 >> 172.21.15.31:0/3627279554 conn(0x7fdea4070ec0 msgr2=0x7fdea411e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 shutdown_connections 2024-09-21T06:24:17.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:17.692+0000 7fdeaa486640 1 -- 172.21.15.31:0/3627279554 wait complete. 2024-09-21T06:24:17.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:17 smithi031 ceph-mon[31237]: pgmap v714: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2024-09-21T06:24:17.929 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:17 smithi110 ceph-mon[34550]: pgmap v714: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2024-09-21T06:24:17.940 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:17 smithi107 ceph-mon[34380]: pgmap v714: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 MiB/s wr, 1.59k op/s 2024-09-21T06:24:19.380 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:24:19.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:24:19.813 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:19 smithi031 ceph-mon[31237]: pgmap v715: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 1.52k op/s 2024-09-21T06:24:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:19 smithi110 ceph-mon[34550]: pgmap v715: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 1.52k op/s 2024-09-21T06:24:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:19 smithi107 ceph-mon[34380]: pgmap v715: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 MiB/s wr, 1.52k op/s 2024-09-21T06:24:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 760.0 s, 1409.2 tps, lat 70.151 ms stddev 78.053 2024-09-21T06:24:20.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 -- 172.21.15.31:0/4048923806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b81027a0 msgr2=0x7f60b8102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 --2- 172.21.15.31:0/4048923806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b81027a0 0x7f60b8102bb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f60a801ace0 tx=0x7f60a803b990 comp rx=0 tx=0).stop 2024-09-21T06:24:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 -- 172.21.15.31:0/4048923806 shutdown_connections 2024-09-21T06:24:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 --2- 172.21.15.31:0/4048923806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60b8107be0 0x7f60b81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 --2- 172.21.15.31:0/4048923806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81030f0 0x7f60b8107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 --2- 172.21.15.31:0/4048923806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b81027a0 0x7f60b8102bb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.345+0000 7f60c01c3640 1 -- 172.21.15.31:0/4048923806 >> 172.21.15.31:0/4048923806 conn(0x7f60b8079730 msgr2=0x7f60b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.349+0000 7f60c01c3640 1 -- 172.21.15.31:0/4048923806 shutdown_connections 2024-09-21T06:24:20.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.349+0000 7f60c01c3640 1 -- 172.21.15.31:0/4048923806 wait complete. 2024-09-21T06:24:20.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 Processor -- start 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 -- start start 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 0x7f60b80743b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60b81030f0 0x7f60b8074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107be0 0x7f60b810f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810b580 con 0x7f60b8107be0 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810b400 con 0x7f60b81027a0 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.350+0000 7f60c01c3640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810b700 con 0x7f60b81030f0 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.351+0000 7f60be760640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107be0 0x7f60b810f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.351+0000 7f60bd75e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60b81030f0 0x7f60b8074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.351+0000 7f60bdf5f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 0x7f60b80743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.351+0000 7f60bdf5f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 0x7f60b80743b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53600/0 (socket says 172.21.15.31:53600) 2024-09-21T06:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.351+0000 7f60bdf5f640 1 -- 172.21.15.31:0/2047301225 learned_addr learned my addr 172.21.15.31:0/2047301225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.353+0000 7f60bdf5f640 1 -- 172.21.15.31:0/2047301225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60b81030f0 msgr2=0x7f60b8074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.354+0000 7f60bdf5f640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60b81030f0 0x7f60b8074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.354+0000 7f60bdf5f640 1 -- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107be0 msgr2=0x7f60b810f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.354+0000 7f60bdf5f640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107be0 0x7f60b810f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.354+0000 7f60bdf5f640 1 -- 172.21.15.31:0/2047301225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b810f940 con 0x7f60b81027a0 2024-09-21T06:24:20.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.359+0000 7f60bdf5f640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 0x7f60b80743b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f60a8003ba0 tx=0x7f60a8003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.362+0000 7f60b6ffd640 1 -- 172.21.15.31:0/2047301225 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8057070 con 0x7f60b81027a0 2024-09-21T06:24:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.362+0000 7f60b6ffd640 1 -- 172.21.15.31:0/2047301225 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a8040ae0 con 0x7f60b81027a0 2024-09-21T06:24:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.362+0000 7f60b6ffd640 1 -- 172.21.15.31:0/2047301225 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8045040 con 0x7f60b81027a0 2024-09-21T06:24:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.362+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b810fb70 con 0x7f60b81027a0 2024-09-21T06:24:20.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.362+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81154d0 con 0x7f60b81027a0 2024-09-21T06:24:20.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.366+0000 7f60c6737740 1 Processor -- start 2024-09-21T06:24:20.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 -- start start 2024-09-21T06:24:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 0x563f84e44430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e44970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e44eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f84dbae60 con 0x563f84a7c670 2024-09-21T06:24:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f84daa290 con 0x563f84a9d350 2024-09-21T06:24:20.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60c6737740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f84dae0a0 con 0x563f84a74490 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60be760640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 0x563f84e44430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60be760640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 0x563f84e44430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53602/0 (socket says 172.21.15.31:53602) 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.367+0000 7f60be760640 1 -- 172.21.15.31:0/1981195988 learned_addr learned my addr 172.21.15.31:0/1981195988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60bd75e640 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e44eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60be760640 1 -- 172.21.15.31:0/1981195988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 msgr2=0x563f84e44970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60be760640 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e44970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60be760640 1 -- 172.21.15.31:0/1981195988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 msgr2=0x563f84e44eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60b6ffd640 1 -- 172.21.15.31:0/2047301225 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f60a8048020 con 0x7f60b81027a0 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60be760640 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e44eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.368+0000 7f60be760640 1 -- 172.21.15.31:0/1981195988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f84b08930 con 0x563f84a9d350 2024-09-21T06:24:20.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.369+0000 7f60bd75e640 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e44eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:24:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.369+0000 7f60b6ffd640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60a00b6800 0x7f60a00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.370+0000 7f60b6ffd640 1 -- 172.21.15.31:0/2047301225 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f60a8118600 con 0x7f60b81027a0 2024-09-21T06:24:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.370+0000 7f60bdf5f640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60a00b6800 0x7f60a00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.370+0000 7f60be760640 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 0x563f84e44430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f60b00099c0 tx=0x7f60b0009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b665a59ebd3e26d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.371+0000 7f609ffff640 1 -- 172.21.15.31:0/1981195988 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0017070 con 0x563f84a9d350 2024-09-21T06:24:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.371+0000 7f609ffff640 1 -- 172.21.15.31:0/1981195988 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b0013470 con 0x563f84a9d350 2024-09-21T06:24:20.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.371+0000 7f609ffff640 1 -- 172.21.15.31:0/1981195988 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0012630 con 0x563f84a9d350 2024-09-21T06:24:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 -- 172.21.15.31:0/1981195988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 msgr2=0x563f84e44430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 0x563f84e44430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f60b00099c0 tx=0x7f60b0009e80 comp rx=0 tx=0).stop 2024-09-21T06:24:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 -- 172.21.15.31:0/1981195988 shutdown_connections 2024-09-21T06:24:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e44eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e44970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 --2- 172.21.15.31:0/1981195988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a9d350 0x563f84e44430 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60c6737740 1 -- 172.21.15.31:0/1981195988 >> 172.21.15.31:0/1981195988 conn(0x563f84a91740 msgr2=0x563f84e42320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.372+0000 7f60bdf5f640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60a00b6800 0x7f60a00b8d10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x563f84e430b0 tx=0x7f60a8003f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.373+0000 7f60c6737740 1 -- 172.21.15.31:0/1981195988 shutdown_connections 2024-09-21T06:24:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.373+0000 7f60c6737740 1 -- 172.21.15.31:0/1981195988 wait complete. 2024-09-21T06:24:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.373+0000 7f60c6737740 1 -- start start 2024-09-21T06:24:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.374+0000 7f60c6737740 1 Processor -- start 2024-09-21T06:24:20.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.374+0000 7f60c6737740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a74490 0x563f84e50110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.374+0000 7f60c6737740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.374+0000 7f60c6737740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a9d350 0x563f84e5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.374+0000 7f60c6737740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x563f84c0b890 con 0x563f84a7c670 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60bd75e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60bd75e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60400/0 (socket says 172.21.15.31:60400) 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60bd75e640 1 -- 172.21.15.31:0/1191874746 learned_addr learned my addr 172.21.15.31:0/1191874746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f84d44d40 con 0x563f84a74490 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563f84e52c10 con 0x563f84a9d350 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a74490 0x563f84e50110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a9d350 0x563f84e5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60c6737740 10 client.0 _ll_register_callbacks cb 0x563f84da8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 -- 172.21.15.31:0/1191874746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a9d350 msgr2=0x563f84e5bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a9d350 0x563f84e5bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 -- 172.21.15.31:0/1191874746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 msgr2=0x563f84e506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 -- 172.21.15.31:0/1191874746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f84e5c600 con 0x563f84a74490 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.375+0000 7f60be760640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a74490 0x563f84e50110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f60b0013d30 tx=0x7f60b0013ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4822f2613c2303a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f609d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f609d7fa640 1 -- 172.21.15.31:0/1191874746 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b002d060 con 0x563f84a74490 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f609d7fa640 1 -- 172.21.15.31:0/1191874746 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f60b001c890 con 0x563f84a74490 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60b0013470 con 0x563f84a74490 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f609d7fa640 1 -- 172.21.15.31:0/1191874746 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f60b0017050 con 0x563f84a74490 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f60c6737740 10 client.27788 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.376+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f84e5ee60 con 0x563f84a74490 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.379+0000 7f609d7fa640 1 -- 172.21.15.31:0/1191874746 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b0013470 con 0x563f84a74490 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.379+0000 7f609d7fa640 1 -- 172.21.15.31:0/1191874746 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f60b0031b90 con 0x563f84a74490 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.379+0000 7f60c6737740 10 client.27788 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.379+0000 7f60c6737740 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.379+0000 7f60c6737740 20 client.27788 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.380+0000 7f60c6737740 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a5bcd0 0x563f84e621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.380+0000 7f60c6737740 4 client.27788 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.380+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f84dbcdb0 con 0x563f84a5bcd0 2024-09-21T06:24:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.380+0000 7f60bd75e640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a5bcd0 0x563f84e621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.380+0000 7f609d7fa640 10 client.27788 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.381+0000 7f60bd75e640 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a5bcd0 0x563f84e621c0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44f835c0f3776dc server_cookie=7e1d6e5803fab70b in_seq=0 out_seq=0 2024-09-21T06:24:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.381+0000 7f609d7fa640 1 -- 172.21.15.31:0/1191874746 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f84dbcdb0 con 0x563f84a5bcd0 2024-09-21T06:24:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.381+0000 7f609d7fa640 10 client.27788 handle_command_reply: tid=0 2024-09-21T06:24:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.381+0000 7f60c6737740 1 client.27788 shutdown 2024-09-21T06:24:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a74490 msgr2=0x563f84e50110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a74490 0x563f84e50110 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f60b0013d30 tx=0x7f60b0013ef0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 shutdown_connections 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a5bcd0 0x563f84e621c0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a9d350 0x563f84e5bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f609d7fa640 0 client.27788 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 --2- 172.21.15.31:0/1191874746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a74490 0x563f84e50110 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 >> 172.21.15.31:0/1191874746 conn(0x563f84a91740 msgr2=0x563f84e475c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 shutdown_connections 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.382+0000 7f60c6737740 1 -- 172.21.15.31:0/1191874746 wait complete. 2024-09-21T06:24:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.383+0000 7f60c6737740 20 client.27788 trim_cache size 0 max 16384 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.403+0000 7f60c6737740 1 Processor -- start 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.403+0000 7f60c6737740 1 -- start start 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60c6737740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a5bcd0 0x563f84e5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60c6737740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e42390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60c6737740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60c6737740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400c890 con 0x563f84a7c670 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60c6737740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400f0f0 con 0x563f84a5bcd0 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60c6737740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400cee0 con 0x563f84a74490 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60be760640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60be760640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60402/0 (socket says 172.21.15.31:60402) 2024-09-21T06:24:20.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60be760640 1 -- 172.21.15.31:0/2053515662 learned_addr learned my addr 172.21.15.31:0/2053515662 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:20.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60bd75e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e42390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60be760640 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a5bcd0 0x563f84e5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60bd75e640 1 -- 172.21.15.31:0/2053515662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a5bcd0 msgr2=0x563f84e5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60bd75e640 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a5bcd0 0x563f84e5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60bd75e640 1 -- 172.21.15.31:0/2053515662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 msgr2=0x563f84e494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60bd75e640 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.404+0000 7f60bd75e640 1 -- 172.21.15.31:0/2053515662 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f84e62530 con 0x563f84a74490 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.405+0000 7f60bd75e640 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e42390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x563f84e62700 tx=0x7f60ac008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c2012172a92f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.405+0000 7f609f7fe640 1 -- 172.21.15.31:0/2053515662 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac038a10 con 0x563f84a74490 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.405+0000 7f609f7fe640 1 -- 172.21.15.31:0/2053515662 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ac042030 con 0x563f84a74490 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.405+0000 7f609f7fe640 1 -- 172.21.15.31:0/2053515662 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac046c60 con 0x563f84a74490 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 -- 172.21.15.31:0/2053515662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 msgr2=0x563f84e42390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e42390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x563f84e62700 tx=0x7f60ac008040 comp rx=0 tx=0).stop 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 -- 172.21.15.31:0/2053515662 shutdown_connections 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a7c670 0x563f84e494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e42390 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 --2- 172.21.15.31:0/2053515662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a5bcd0 0x563f84e5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 -- 172.21.15.31:0/2053515662 >> 172.21.15.31:0/2053515662 conn(0x563f84a91740 msgr2=0x563f84e41c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 -- 172.21.15.31:0/2053515662 shutdown_connections 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 -- 172.21.15.31:0/2053515662 wait complete. 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 -- start start 2024-09-21T06:24:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.406+0000 7f60c6737740 1 Processor -- start 2024-09-21T06:24:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a5bcd0 0x563f84e58100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e58680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a7c670 0x563f84e52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400d5b0 con 0x563f84a5bcd0 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400ea20 con 0x563f84a7c670 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400dc80 con 0x563f84a74490 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60c6737740 10 client.0 _ll_register_callbacks cb 0x7f60b0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60be760640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a5bcd0 0x563f84e58100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60be760640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a5bcd0 0x563f84e58100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60416/0 (socket says 172.21.15.31:60416) 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60be760640 1 -- 172.21.15.31:0/324787545 learned_addr learned my addr 172.21.15.31:0/324787545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.407+0000 7f60be760640 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a7c670 0x563f84e52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.408+0000 7f60be760640 1 -- 172.21.15.31:0/324787545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 msgr2=0x563f84e58680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:24:20.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.408+0000 7f60be760640 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e58680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.408+0000 7f60be760640 1 -- 172.21.15.31:0/324787545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a5bcd0 msgr2=0x563f84e58100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.408+0000 7f60be760640 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a5bcd0 0x563f84e58100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.408+0000 7f60be760640 1 -- 172.21.15.31:0/324787545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f84e535a0 con 0x563f84a7c670 2024-09-21T06:24:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.409+0000 7f60be760640 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a7c670 0x563f84e52eb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f60b00139b0 tx=0x7f60b0009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4771d597e7704db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f609d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f609d7fa640 1 -- 172.21.15.31:0/324787545 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b002d070 con 0x563f84a7c670 2024-09-21T06:24:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f609d7fa640 1 -- 172.21.15.31:0/324787545 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7f60b0013cd0 con 0x563f84a7c670 2024-09-21T06:24:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ac042030 con 0x563f84a7c670 2024-09-21T06:24:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f609d7fa640 1 -- 172.21.15.31:0/324787545 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f60b0013470 con 0x563f84a7c670 2024-09-21T06:24:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f60c6737740 10 client.27794 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.411+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f84ecd2f0 con 0x563f84a7c670 2024-09-21T06:24:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f609d7fa640 1 -- 172.21.15.31:0/324787545 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ac042030 con 0x563f84a7c670 2024-09-21T06:24:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f609d7fa640 1 -- 172.21.15.31:0/324787545 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f60b0006020 con 0x563f84a7c670 2024-09-21T06:24:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60c6737740 10 client.27794 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60c6737740 10 client.27794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60c6737740 20 client.27794 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60c6737740 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a9d350 0x563f84ecfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60c6737740 4 client.27794 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f84db8860 con 0x563f84a9d350 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60bd75e640 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a9d350 0x563f84ecfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f609d7fa640 10 client.27794 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:20.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.412+0000 7f60bd75e640 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a9d350 0x563f84ecfa30 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8539fafcae3416b0 server_cookie=9f13e3d729599183 in_seq=0 out_seq=0 2024-09-21T06:24:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.414+0000 7f609d7fa640 1 -- 172.21.15.31:0/324787545 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f84db8860 con 0x563f84a9d350 2024-09-21T06:24:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.414+0000 7f609d7fa640 10 client.27794 handle_command_reply: tid=0 2024-09-21T06:24:20.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.414+0000 7f60c6737740 1 client.27794 shutdown 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.414+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a7c670 msgr2=0x563f84e52eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.414+0000 7f60c6737740 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a7c670 0x563f84e52eb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f60b00139b0 tx=0x7f60b0009040 comp rx=0 tx=0).stop 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 shutdown_connections 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x563f84a9d350 0x563f84ecfa30 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f84a7c670 0x563f84e52eb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563f84a74490 0x563f84e58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 --2- 172.21.15.31:0/324787545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f84a5bcd0 0x563f84e58100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f609d7fa640 0 client.27794 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 >> 172.21.15.31:0/324787545 conn(0x563f84a91740 msgr2=0x563f84e45410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 shutdown_connections 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 1 -- 172.21.15.31:0/324787545 wait complete. 2024-09-21T06:24:20.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c6737740 20 client.27794 trim_cache size 0 max 16384 2024-09-21T06:24:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:24:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:24:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "06bdcfb5-152c-425c-b4aa-a2e742f70d78", 2024-09-21T06:24:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:24:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60a00b6800 msgr2=0x7f60a00b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c01c3640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60a00b6800 0x7f60a00b8d10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x563f84e430b0 tx=0x7f60a8003f40 comp rx=0 tx=0).stop 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.415+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 msgr2=0x7f60b80743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 0x7f60b80743b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f60a8003ba0 tx=0x7f60a8003bd0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 shutdown_connections 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f60a00b6800 0x7f60a00b8d10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60b8107be0 0x7f60b810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60b81030f0 0x7f60b8074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 --2- 172.21.15.31:0/2047301225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60b81027a0 0x7f60b80743b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 >> 172.21.15.31:0/2047301225 conn(0x7f60b8079730 msgr2=0x7f60b81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 shutdown_connections 2024-09-21T06:24:20.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:20.416+0000 7f60c01c3640 1 -- 172.21.15.31:0/2047301225 wait complete. 2024-09-21T06:24:21.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:24:21.554 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:21.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:21 smithi031 ceph-mon[31237]: pgmap v716: 97 pgs: 97 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2024-09-21T06:24:21.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:21 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:24:21.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:21 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:21.999 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:21 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:24:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:21 smithi110 ceph-mon[34550]: pgmap v716: 97 pgs: 97 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2024-09-21T06:24:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:21 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:24:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:21 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:21 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:24:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:21 smithi107 ceph-mon[34380]: pgmap v716: 97 pgs: 97 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 1.32k op/s 2024-09-21T06:24:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:21 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:24:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:21 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:21 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:24:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.165+0000 7fd120693640 1 -- 172.21.15.31:0/1350217511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 msgr2=0x7fd11810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.165+0000 7fd120693640 1 --2- 172.21.15.31:0/1350217511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 0x7fd11810a3c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd108009860 tx=0x7fd10802a360 comp rx=0 tx=0).stop 2024-09-21T06:24:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.166+0000 7fd120693640 1 -- 172.21.15.31:0/1350217511 shutdown_connections 2024-09-21T06:24:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.166+0000 7fd120693640 1 --2- 172.21.15.31:0/1350217511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd11811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.166+0000 7fd120693640 1 --2- 172.21.15.31:0/1350217511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 0x7fd11810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.166+0000 7fd120693640 1 --2- 172.21.15.31:0/1350217511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 0x7fd11810a3c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.166+0000 7fd120693640 1 -- 172.21.15.31:0/1350217511 >> 172.21.15.31:0/1350217511 conn(0x7fd118070ec0 msgr2=0x7fd1180712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.169+0000 7fd120693640 1 -- 172.21.15.31:0/1350217511 shutdown_connections 2024-09-21T06:24:22.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.172+0000 7fd120693640 1 -- 172.21.15.31:0/1350217511 wait complete. 2024-09-21T06:24:22.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.173+0000 7fd120693640 1 Processor -- start 2024-09-21T06:24:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.174+0000 7fd120693640 1 -- start start 2024-09-21T06:24:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd120693640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 0x7fd1181a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd120693640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 0x7fd1181a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd120693640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd1181aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd120693640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd118122930 con 0x7fd11810a9e0 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd120693640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181227b0 con 0x7fd11811cc30 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd120693640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd118122ab0 con 0x7fd118111f40 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd11ec30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd1181aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.175+0000 7fd11ec30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd1181aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53662/0 (socket says 172.21.15.31:53662) 2024-09-21T06:24:22.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.176+0000 7fd11ec30640 1 -- 172.21.15.31:0/535789970 learned_addr learned my addr 172.21.15.31:0/535789970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.176+0000 7fd11e42f640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 0x7fd1181a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.176+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 0x7fd1181a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.176+0000 7fd11ec30640 1 -- 172.21.15.31:0/535789970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 msgr2=0x7fd1181a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.177+0000 7fd11ec30640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 0x7fd1181a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.177+0000 7fd11ec30640 1 -- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 msgr2=0x7fd1181a92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.177+0000 7fd11ec30640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 0x7fd1181a92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.177+0000 7fd11ec30640 1 -- 172.21.15.31:0/535789970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1181af230 con 0x7fd11811cc30 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.178+0000 7fd11e42f640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 0x7fd1181a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.178+0000 7fd11ec30640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd1181aeb00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd11400d750 tx=0x7fd11400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.179+0000 7fd10f7fe640 1 -- 172.21.15.31:0/535789970 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114017070 con 0x7fd11811cc30 2024-09-21T06:24:22.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.179+0000 7fd10f7fe640 1 -- 172.21.15.31:0/535789970 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1140054d0 con 0x7fd11811cc30 2024-09-21T06:24:22.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.179+0000 7fd10f7fe640 1 -- 172.21.15.31:0/535789970 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114004c60 con 0x7fd11811cc30 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.181+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1181af520 con 0x7fd11811cc30 2024-09-21T06:24:22.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.181+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1181b5c80 con 0x7fd11811cc30 2024-09-21T06:24:22.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.184+0000 7fd126c07740 1 Processor -- start 2024-09-21T06:24:22.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.185+0000 7fd10f7fe640 1 -- 172.21.15.31:0/535789970 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd114009bd0 con 0x7fd11811cc30 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 -- start start 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f19650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f19b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b51390 0x560722f1a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560722e8fe50 con 0x560722b491b0 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560722e7f280 con 0x560722b72070 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd126c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560722e83090 con 0x560722b51390 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f19650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f19650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53676/0 (socket says 172.21.15.31:53676) 2024-09-21T06:24:22.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd11dc2e640 1 -- 172.21.15.31:0/554837834 learned_addr learned my addr 172.21.15.31:0/554837834 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.186+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b51390 0x560722f1a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 -- 172.21.15.31:0/554837834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b51390 msgr2=0x560722f1a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b51390 0x560722f1a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 -- 172.21.15.31:0/554837834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 msgr2=0x560722f19b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd10f7fe640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1040b6700 0x7fd1040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f19b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 -- 172.21.15.31:0/554837834 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560722bdd650 con 0x560722b72070 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd10f7fe640 1 -- 172.21.15.31:0/535789970 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fd1140e73d0 con 0x7fd11811cc30 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f19650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fd0 tx=0x7fd110009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38b1b7bb5497d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.187+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1040b6700 0x7fd1040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.188+0000 7fd10cff9640 1 -- 172.21.15.31:0/554837834 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000f820 con 0x560722b72070 2024-09-21T06:24:22.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.188+0000 7fd10cff9640 1 -- 172.21.15.31:0/554837834 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd110022430 con 0x560722b72070 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.188+0000 7fd10cff9640 1 -- 172.21.15.31:0/554837834 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000c660 con 0x560722b72070 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 -- 172.21.15.31:0/554837834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 msgr2=0x560722f19650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f19650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fd0 tx=0x7fd110009640 comp rx=0 tx=0).stop 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1040b6700 0x7fd1040b8c10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd110011fd0 tx=0x7fd11000cd30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 -- 172.21.15.31:0/554837834 shutdown_connections 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b51390 0x560722f1a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f19b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 --2- 172.21.15.31:0/554837834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f19650 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.189+0000 7fd126c07740 1 -- 172.21.15.31:0/554837834 >> 172.21.15.31:0/554837834 conn(0x560722b66460 msgr2=0x560722f17540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.190+0000 7fd126c07740 1 -- 172.21.15.31:0/554837834 shutdown_connections 2024-09-21T06:24:22.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.190+0000 7fd126c07740 1 -- 172.21.15.31:0/554837834 wait complete. 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.190+0000 7fd126c07740 1 -- start start 2024-09-21T06:24:22.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.190+0000 7fd126c07740 1 Processor -- start 2024-09-21T06:24:22.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f24cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f25730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560722ce0700 con 0x560722b51390 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560722e1a110 con 0x560722b72070 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560722f27e80 con 0x560722b491b0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.196+0000 7fd126c07740 10 client.0 _ll_register_callbacks cb 0x560722e33990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.197+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.197+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f251f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60458/0 (socket says 172.21.15.31:60458) 2024-09-21T06:24:22.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.197+0000 7fd11dc2e640 1 -- 172.21.15.31:0/1700661954 learned_addr learned my addr 172.21.15.31:0/1700661954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.203+0000 7fd11e42f640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f24cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11ec30640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f25730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11dc2e640 1 -- 172.21.15.31:0/1700661954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 msgr2=0x560722f24cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f24cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11dc2e640 1 -- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 msgr2=0x560722f25730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f25730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11dc2e640 1 -- 172.21.15.31:0/1700661954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560722e47c40 con 0x560722b51390 2024-09-21T06:24:22.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.205+0000 7fd11e42f640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f24cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.216+0000 7fd11dc2e640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f251f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd110010010 tx=0x7fd11000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd2c062f761b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.216+0000 7fd101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.221+0000 7fd101ffb640 1 -- 172.21.15.31:0/1700661954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000ce60 con 0x560722b51390 2024-09-21T06:24:22.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.222+0000 7fd101ffb640 1 -- 172.21.15.31:0/1700661954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fd11000eb60 con 0x560722b51390 2024-09-21T06:24:22.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.223+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110022430 con 0x560722b51390 2024-09-21T06:24:22.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.227+0000 7fd101ffb640 1 -- 172.21.15.31:0/1700661954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd11003b050 con 0x560722b51390 2024-09-21T06:24:22.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.227+0000 7fd126c07740 10 client.18027 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:22.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.227+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560722f44a90 con 0x560722b51390 2024-09-21T06:24:22.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd101ffb640 1 -- 172.21.15.31:0/1700661954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd110022430 con 0x560722b51390 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd101ffb640 1 -- 172.21.15.31:0/1700661954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd11000b070 con 0x560722b51390 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd126c07740 10 client.18027 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd126c07740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd126c07740 20 client.18027 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd126c07740 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b309f0 0x560722f47f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd126c07740 4 client.18027 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:22.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.228+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560722e91da0 con 0x560722b309f0 2024-09-21T06:24:22.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.230+0000 7fd11e42f640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b309f0 0x560722f47f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.233+0000 7fd11e42f640 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b309f0 0x560722f47f00 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142d298f7152c854 server_cookie=2eb378c6d6af71ba in_seq=0 out_seq=0 2024-09-21T06:24:22.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.233+0000 7fd101ffb640 10 client.18027 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:22.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.235+0000 7fd101ffb640 1 -- 172.21.15.31:0/1700661954 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560722e91da0 con 0x560722b309f0 2024-09-21T06:24:22.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.235+0000 7fd101ffb640 10 client.18027 handle_command_reply: tid=0 2024-09-21T06:24:22.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.235+0000 7fd126c07740 1 client.18027 shutdown 2024-09-21T06:24:22.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 msgr2=0x560722f251f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f251f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd110010010 tx=0x7fd11000f000 comp rx=0 tx=0).stop 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 shutdown_connections 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b309f0 0x560722f47f00 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f25730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f251f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd101ffb640 0 client.18027 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 --2- 172.21.15.31:0/1700661954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f24cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.236+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 >> 172.21.15.31:0/1700661954 conn(0x560722b66460 msgr2=0x560722f1a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.237+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 shutdown_connections 2024-09-21T06:24:22.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.238+0000 7fd126c07740 1 -- 172.21.15.31:0/1700661954 wait complete. 2024-09-21T06:24:22.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.238+0000 7fd126c07740 20 client.18027 trim_cache size 0 max 16384 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.264+0000 7fd126c07740 1 Processor -- start 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.264+0000 7fd126c07740 1 -- start start 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd126c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd126c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd126c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f1ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60474/0 (socket says 172.21.15.31:60474) 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f26b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49406/0 (socket says 172.21.15.31:49406) 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11dc2e640 1 -- 172.21.15.31:0/2926859037 learned_addr learned my addr 172.21.15.31:0/2926859037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:22.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd11810adf0 con 0x560722b51390 2024-09-21T06:24:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b67b0 con 0x560722b72070 2024-09-21T06:24:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd118119320 con 0x560722b491b0 2024-09-21T06:24:22.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 -- 172.21.15.31:0/2926859037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 msgr2=0x560722f16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 -- 172.21.15.31:0/2926859037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 msgr2=0x560722f1ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f1ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.265+0000 7fd11e42f640 1 -- 172.21.15.31:0/2926859037 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560722f25f70 con 0x560722b491b0 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.266+0000 7fd11e42f640 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f26b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd10802a880 tx=0x7fd108005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7473bc0ee2f9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.266+0000 7fd103fff640 1 -- 172.21.15.31:0/2926859037 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd108037040 con 0x560722b491b0 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.266+0000 7fd103fff640 1 -- 172.21.15.31:0/2926859037 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd108038040 con 0x560722b491b0 2024-09-21T06:24:22.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.266+0000 7fd103fff640 1 -- 172.21.15.31:0/2926859037 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1080046c0 con 0x560722b491b0 2024-09-21T06:24:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 msgr2=0x560722f26b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f26b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd10802a880 tx=0x7fd108005870 comp rx=0 tx=0).stop 2024-09-21T06:24:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 shutdown_connections 2024-09-21T06:24:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b72070 0x560722f16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b51390 0x560722f1ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.271 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 --2- 172.21.15.31:0/2926859037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b491b0 0x560722f26b90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.267+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 >> 172.21.15.31:0/2926859037 conn(0x560722b66460 msgr2=0x560722f1a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.268+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 shutdown_connections 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.268+0000 7fd126c07740 1 -- 172.21.15.31:0/2926859037 wait complete. 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.268+0000 7fd126c07740 1 -- start start 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.268+0000 7fd126c07740 1 Processor -- start 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b51390 0x560722f29700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 0x560722f29c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181199f0 con 0x560722b491b0 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1181b60e0 con 0x560722b51390 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd11811a0c0 con 0x560722b72070 2024-09-21T06:24:22.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd11dc2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b51390 0x560722f29700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd11ec30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 0x560722f29c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd11ec30640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 0x560722f29c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49418/0 (socket says 172.21.15.31:49418) 2024-09-21T06:24:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd11ec30640 1 -- 172.21.15.31:0/821610971 learned_addr learned my addr 172.21.15.31:0/821610971 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd126c07740 10 client.0 _ll_register_callbacks cb 0x7fd110011420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:22.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.269+0000 7fd11e42f640 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.270+0000 7fd11ec30640 1 -- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b51390 msgr2=0x560722f29700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.270+0000 7fd11ec30640 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b51390 0x560722f29700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.270+0000 7fd11ec30640 1 -- 172.21.15.31:0/821610971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 msgr2=0x560722f291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.270+0000 7fd11ec30640 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.270+0000 7fd11ec30640 1 -- 172.21.15.31:0/821610971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560722fb1890 con 0x560722b72070 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.270+0000 7fd11ec30640 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 0x560722f29c40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd114009610 tx=0x7fd1140acb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f07b2c5a8c6f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.271+0000 7fd101ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.271+0000 7fd101ffb640 1 -- 172.21.15.31:0/821610971 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd114005680 con 0x560722b72070 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.271+0000 7fd101ffb640 1 -- 172.21.15.31:0/821610971 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8543+0+0 (secure 0 0 0) 0x7fd1140abee0 con 0x560722b72070 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.271+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd108038040 con 0x560722b72070 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.271+0000 7fd101ffb640 1 -- 172.21.15.31:0/821610971 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd1140acd20 con 0x560722b72070 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.271+0000 7fd126c07740 10 client.27853 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560722fb2b00 con 0x560722b72070 2024-09-21T06:24:22.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd101ffb640 1 -- 172.21.15.31:0/821610971 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1140ad910 con 0x560722b72070 2024-09-21T06:24:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd101ffb640 1 -- 172.21.15.31:0/821610971 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd1140d0b90 con 0x560722b72070 2024-09-21T06:24:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 10 client.27853 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 10 client.27853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 20 client.27853 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b05da0 0x560722fb52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:22.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 4 client.27853 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.272+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd108038040 con 0x560722b05da0 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.273+0000 7fd11e42f640 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b05da0 0x560722fb52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd101ffb640 10 client.27853 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd11e42f640 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b05da0 0x560722fb52a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592ee36ec5f0ed2a server_cookie=a5895fc025b7a47d in_seq=0 out_seq=0 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd101ffb640 1 -- 172.21.15.31:0/821610971 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8610 (crc 0 0 0) 0x7fd108038040 con 0x560722b05da0 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd101ffb640 10 client.27853 handle_command_reply: tid=0 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd126c07740 1 client.27853 shutdown 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 msgr2=0x560722f29c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.274+0000 7fd126c07740 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 0x560722f29c40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd114009610 tx=0x7fd1140acb60 comp rx=0 tx=0).stop 2024-09-21T06:24:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 shutdown_connections 2024-09-21T06:24:22.278 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560722b05da0 0x560722fb52a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560722b72070 0x560722f29c40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560722b51390 0x560722f29700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd101ffb640 0 client.27853 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 --2- 172.21.15.31:0/821610971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560722b491b0 0x560722f291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 >> 172.21.15.31:0/821610971 conn(0x560722b66460 msgr2=0x560722f30a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 shutdown_connections 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 1 -- 172.21.15.31:0/821610971 wait complete. 2024-09-21T06:24:22.279 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.275+0000 7fd126c07740 20 client.27853 trim_cache size 0 max 16384 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.280 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:24:22.281 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:24:22.284 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.286 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.287 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:24:22.289 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.292 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.293 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:24:22.295 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:22.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:22.299 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:24:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1040b6700 msgr2=0x7fd1040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1040b6700 0x7fd1040b8c10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd110011fd0 tx=0x7fd11000cd30 comp rx=0 tx=0).stop 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 msgr2=0x7fd1181aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd1181aeb00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd11400d750 tx=0x7fd11400dc10 comp rx=0 tx=0).stop 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 shutdown_connections 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd11811cc30 0x7fd1181aeb00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd118111f40 0x7fd1181a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd1040b6700 0x7fd1040b8c10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 --2- 172.21.15.31:0/535789970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd11810a9e0 0x7fd1181a92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 >> 172.21.15.31:0/535789970 conn(0x7fd118070ec0 msgr2=0x7fd11811d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 shutdown_connections 2024-09-21T06:24:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:22.277+0000 7fd120693640 1 -- 172.21.15.31:0/535789970 wait complete. 2024-09-21T06:24:23.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:22 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:23.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:22 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:24:23.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:22 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:23.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:22 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:24:23.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:22 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:23.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:22 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:24:23.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:24:23.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:24:23.739 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:24.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:23 smithi031 ceph-mon[31237]: pgmap v717: 97 pgs: 97 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-21T06:24:24.013 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:23 smithi031 ceph-mon[31237]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T06:24:24.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:23 smithi110 ceph-mon[34550]: pgmap v717: 97 pgs: 97 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-21T06:24:24.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:23 smithi110 ceph-mon[34550]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T06:24:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:23 smithi107 ceph-mon[34380]: pgmap v717: 97 pgs: 97 active+clean; 4.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2024-09-21T06:24:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:23 smithi107 ceph-mon[34380]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.334+0000 7f63f151c640 1 -- 172.21.15.31:0/326721270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 msgr2=0x7f63ec10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.334+0000 7f63f151c640 1 --2- 172.21.15.31:0/326721270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 0x7f63ec10f550 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f63e400bc00 tx=0x7f63e402e690 comp rx=0 tx=0).stop 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.335+0000 7f63f151c640 1 -- 172.21.15.31:0/326721270 shutdown_connections 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.335+0000 7f63f151c640 1 --2- 172.21.15.31:0/326721270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 0x7f63ec10f550 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.335+0000 7f63f151c640 1 --2- 172.21.15.31:0/326721270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec1083d0 0x7f63ec1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.335+0000 7f63f151c640 1 --2- 172.21.15.31:0/326721270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec102ae0 0x7f63ec102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.335+0000 7f63f151c640 1 -- 172.21.15.31:0/326721270 >> 172.21.15.31:0/326721270 conn(0x7f63ec079790 msgr2=0x7f63ec079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.339+0000 7f63f151c640 1 -- 172.21.15.31:0/326721270 shutdown_connections 2024-09-21T06:24:24.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.342+0000 7f63f151c640 1 -- 172.21.15.31:0/326721270 wait complete. 2024-09-21T06:24:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.343+0000 7f63f151c640 1 Processor -- start 2024-09-21T06:24:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.343+0000 7f63f151c640 1 -- start start 2024-09-21T06:24:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.343+0000 7f63f151c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec102ae0 0x7f63ec074e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.343+0000 7f63f151c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec1083d0 0x7f63ec06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.344+0000 7f63f151c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 0x7f63ec06e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.344+0000 7f63f151c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec1118e0 con 0x7f63ec102ae0 2024-09-21T06:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.344+0000 7f63f151c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec111760 con 0x7f63ec1083d0 2024-09-21T06:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.344+0000 7f63f151c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec111a60 con 0x7f63ec108d20 2024-09-21T06:24:24.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.345+0000 7f63eaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec102ae0 0x7f63ec074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.345+0000 7f63eaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec102ae0 0x7f63ec074e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60506/0 (socket says 172.21.15.31:60506) 2024-09-21T06:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.345+0000 7f63eaffd640 1 -- 172.21.15.31:0/3668533082 learned_addr learned my addr 172.21.15.31:0/3668533082 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.346+0000 7f63ea7fc640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec1083d0 0x7f63ec06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.346+0000 7f63eb7fe640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 0x7f63ec06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.348+0000 7f63ea7fc640 1 -- 172.21.15.31:0/3668533082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 msgr2=0x7f63ec06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.348+0000 7f63ea7fc640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 0x7f63ec06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.348+0000 7f63ea7fc640 1 -- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec102ae0 msgr2=0x7f63ec074e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.348+0000 7f63ea7fc640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec102ae0 0x7f63ec074e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.348+0000 7f63ea7fc640 1 -- 172.21.15.31:0/3668533082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec06ee50 con 0x7f63ec1083d0 2024-09-21T06:24:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.354+0000 7f63ea7fc640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec1083d0 0x7f63ec06e0f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f63ec06ab20 tx=0x7f63e000db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.354+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0017070 con 0x7f63ec1083d0 2024-09-21T06:24:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.354+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63e0009820 con 0x7f63ec1083d0 2024-09-21T06:24:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.354+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e0004b90 con 0x7f63ec1083d0 2024-09-21T06:24:24.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.355+0000 7f63f151c640 1 -- 172.21.15.31:0/3668533082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec073af0 con 0x7f63ec1083d0 2024-09-21T06:24:24.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.356+0000 7f63f151c640 1 -- 172.21.15.31:0/3668533082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec074080 con 0x7f63ec1083d0 2024-09-21T06:24:24.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.360+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ec102ef0 con 0x7f63ec1083d0 2024-09-21T06:24:24.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.363+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f63e0004d30 con 0x7f63ec1083d0 2024-09-21T06:24:24.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.364+0000 7f63d3fff640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63d40b67b0 0x7f63d40b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.365+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f63e00e7110 con 0x7f63ec1083d0 2024-09-21T06:24:24.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.367+0000 7f63eaffd640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63d40b67b0 0x7f63d40b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.369+0000 7f63eaffd640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63d40b67b0 0x7f63d40b8cc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc00b2c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.373+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63e00099d0 con 0x7f63ec1083d0 2024-09-21T06:24:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 -- 172.21.15.31:0/887251693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 msgr2=0x7efd2c10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 --2- 172.21.15.31:0/887251693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c10aa70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efd24009ca0 tx=0x7efd2402a8e0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 -- 172.21.15.31:0/887251693 shutdown_connections 2024-09-21T06:24:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 --2- 172.21.15.31:0/887251693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd2c110350 0x7efd2c110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 --2- 172.21.15.31:0/887251693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c10fa00 0x7efd2c10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 --2- 172.21.15.31:0/887251693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c10aa70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.538+0000 7efd328f9640 1 -- 172.21.15.31:0/887251693 >> 172.21.15.31:0/887251693 conn(0x7efd2c070dd0 msgr2=0x7efd2c0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.543+0000 7efd328f9640 1 -- 172.21.15.31:0/887251693 shutdown_connections 2024-09-21T06:24:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.545+0000 7efd328f9640 1 -- 172.21.15.31:0/887251693 wait complete. 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 Processor -- start 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 -- start start 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c1204e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd2c10fa00 0x7efd2c1197a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c110350 0x7efd2c119d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c1228a0 con 0x7efd2c110350 2024-09-21T06:24:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c122720 con 0x7efd2c10fa00 2024-09-21T06:24:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.546+0000 7efd328f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd2c122a20 con 0x7efd2c10a660 2024-09-21T06:24:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c1204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c1204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49454/0 (socket says 172.21.15.31:49454) 2024-09-21T06:24:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 -- 172.21.15.31:0/2341911978 learned_addr learned my addr 172.21.15.31:0/2341911978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd30e96640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c110350 0x7efd2c119d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 -- 172.21.15.31:0/2341911978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd2c10fa00 msgr2=0x7efd2c1197a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:24:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd2c10fa00 0x7efd2c1197a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 -- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c110350 msgr2=0x7efd2c119d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c110350 0x7efd2c119d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.547+0000 7efd2bfff640 1 -- 172.21.15.31:0/2341911978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2c11a5b0 con 0x7efd2c10a660 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.551+0000 7efd2bfff640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c1204e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efd2402aed0 tx=0x7efd240039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.551+0000 7efd297fa640 1 -- 172.21.15.31:0/2341911978 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24003de0 con 0x7efd2c10a660 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.552+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2c11f200 con 0x7efd2c10a660 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.552+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2c11f7f0 con 0x7efd2c10a660 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.552+0000 7efd297fa640 1 -- 172.21.15.31:0/2341911978 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd240044f0 con 0x7efd2c10a660 2024-09-21T06:24:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.552+0000 7efd297fa640 1 -- 172.21.15.31:0/2341911978 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24030d20 con 0x7efd2c10a660 2024-09-21T06:24:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.557+0000 7efd39867740 1 Processor -- start 2024-09-21T06:24:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 -- start start 2024-09-21T06:24:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd65ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd66430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ccdbcda6f0 con 0x55ccdb9c01b0 2024-09-21T06:24:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ccdbcc9b20 con 0x55ccdb9972f0 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd39867740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ccdbccd930 con 0x55ccdb99f4d0 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd297fa640 1 -- 172.21.15.31:0/2341911978 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7efd24004040 con 0x7efd2c10a660 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd65ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd65ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53752/0 (socket says 172.21.15.31:53752) 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.560+0000 7efd2bfff640 1 -- 172.21.15.31:0/2825899478 learned_addr learned my addr 172.21.15.31:0/2825899478 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.561+0000 7efd2bfff640 1 -- 172.21.15.31:0/2825899478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 msgr2=0x55ccdbd66430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.561+0000 7efd2bfff640 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd66430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.561+0000 7efd2bfff640 1 -- 172.21.15.31:0/2825899478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9c01b0 msgr2=0x55ccdbd659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.561+0000 7efd2bfff640 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.561+0000 7efd2bfff640 1 -- 172.21.15.31:0/2825899478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdbc8f530 con 0x55ccdb9972f0 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.561+0000 7efd2bfff640 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd65ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd24130a10 tx=0x7efd24130ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c41de852026ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.563+0000 7efd297fa640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd140b6780 0x7efd140b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.563+0000 7efd297fa640 1 -- 172.21.15.31:0/2341911978 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7efd24108940 con 0x7efd2c10a660 2024-09-21T06:24:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.564+0000 7efd30e96640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd140b6780 0x7efd140b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.565+0000 7efd127fc640 1 -- 172.21.15.31:0/2825899478 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2413a020 con 0x55ccdb9972f0 2024-09-21T06:24:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.565+0000 7efd30e96640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd140b6780 0x7efd140b8c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009bb0 tx=0x7efd1c009370 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.565+0000 7efd127fc640 1 -- 172.21.15.31:0/2825899478 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd24036030 con 0x55ccdb9972f0 2024-09-21T06:24:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.565+0000 7efd127fc640 1 -- 172.21.15.31:0/2825899478 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd241353d0 con 0x55ccdb9972f0 2024-09-21T06:24:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.566+0000 7efd39867740 1 -- 172.21.15.31:0/2825899478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 msgr2=0x55ccdbd65ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.566+0000 7efd39867740 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd65ef0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd24130a10 tx=0x7efd24130ed0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.567+0000 7efd39867740 1 -- 172.21.15.31:0/2825899478 shutdown_connections 2024-09-21T06:24:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.567+0000 7efd39867740 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd66430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.567+0000 7efd39867740 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd65ef0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.567+0000 7efd39867740 1 --2- 172.21.15.31:0/2825899478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.567+0000 7efd39867740 1 -- 172.21.15.31:0/2825899478 >> 172.21.15.31:0/2825899478 conn(0x55ccdb9b4300 msgr2=0x55ccdbd638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.568+0000 7efd39867740 1 -- 172.21.15.31:0/2825899478 shutdown_connections 2024-09-21T06:24:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.569+0000 7efd39867740 1 -- 172.21.15.31:0/2825899478 wait complete. 2024-09-21T06:24:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 -- start start 2024-09-21T06:24:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 Processor -- start 2024-09-21T06:24:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd7d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55ccdbc01ab0 con 0x55ccdb99f4d0 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.570+0000 7efd39867740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ccdbc6f730 con 0x55ccdb9972f0 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd39867740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ccdbd73ef0 con 0x55ccdb9c01b0 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd39867740 10 client.0 _ll_register_callbacks cb 0x55ccdbcc36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd71080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53768/0 (socket says 172.21.15.31:53768) 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 -- 172.21.15.31:0/1796183620 learned_addr learned my addr 172.21.15.31:0/1796183620 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:24.575 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd30e96640 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd7d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 -- 172.21.15.31:0/1796183620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb9c01b0 msgr2=0x55ccdbd7d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd7d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 -- 172.21.15.31:0/1796183620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb99f4d0 msgr2=0x55ccdbd715c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd715c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.571+0000 7efd2bfff640 1 -- 172.21.15.31:0/1796183620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdba2b790 con 0x55ccdb9972f0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.572+0000 7efd2bfff640 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd71080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd240ce6d0 tx=0x7efd2402a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a8024dc7c20e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.572+0000 7efd0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.572+0000 7efd0b7fe640 1 -- 172.21.15.31:0/1796183620 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2413a020 con 0x55ccdb9972f0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.572+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd24036030 con 0x55ccdb9972f0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.572+0000 7efd0b7fe640 1 -- 172.21.15.31:0/1796183620 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7efd240305e0 con 0x55ccdb9972f0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.573+0000 7efd0b7fe640 1 -- 172.21.15.31:0/1796183620 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd240ca5e0 con 0x55ccdb9972f0 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.573+0000 7efd39867740 10 client.27821 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:24.576 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.573+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ccdbaa4620 con 0x55ccdb9972f0 2024-09-21T06:24:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.574+0000 7efd0b7fe640 1 -- 172.21.15.31:0/1796183620 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd24036030 con 0x55ccdb9972f0 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.574+0000 7efd0b7fe640 1 -- 172.21.15.31:0/1796183620 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efd24132b60 con 0x55ccdb9972f0 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.577+0000 7efd39867740 10 client.27821 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.577+0000 7efd39867740 10 client.27821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.577+0000 7efd39867740 20 client.27821 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.577+0000 7efd39867740 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb97ed80 0x55ccdbd82dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.577+0000 7efd39867740 4 client.27821 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.577+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccdbcdc640 con 0x55ccdb97ed80 2024-09-21T06:24:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.578+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb97ed80 0x55ccdbd82dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.578+0000 7efd0b7fe640 10 client.27821 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.578+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb97ed80 0x55ccdbd82dc0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a969647c66e57c74 server_cookie=8ffa6ee34524d77e in_seq=0 out_seq=0 2024-09-21T06:24:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.579+0000 7efd0b7fe640 1 -- 172.21.15.31:0/1796183620 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccdbcdc640 con 0x55ccdb97ed80 2024-09-21T06:24:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.579+0000 7efd0b7fe640 10 client.27821 handle_command_reply: tid=0 2024-09-21T06:24:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.579+0000 7efd39867740 1 client.27821 shutdown 2024-09-21T06:24:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.580+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 msgr2=0x55ccdbd71080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.580+0000 7efd39867740 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd71080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd240ce6d0 tx=0x7efd2402a8e0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 shutdown_connections 2024-09-21T06:24:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb97ed80 0x55ccdbd82dc0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd7d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 --2- 172.21.15.31:0/1796183620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9972f0 0x55ccdbd71080 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 >> 172.21.15.31:0/1796183620 conn(0x55ccdb9b4300 msgr2=0x55ccdbd669e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd0b7fe640 0 client.27821 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 shutdown_connections 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 1 -- 172.21.15.31:0/1796183620 wait complete. 2024-09-21T06:24:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.581+0000 7efd39867740 20 client.27821 trim_cache size 0 max 16384 2024-09-21T06:24:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.614+0000 7efd39867740 1 Processor -- start 2024-09-21T06:24:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.614+0000 7efd39867740 1 -- start start 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.614+0000 7efd39867740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd39867740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60546/0 (socket says 172.21.15.31:60546) 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd2b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd2b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd7d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49496/0 (socket says 172.21.15.31:49496) 2024-09-21T06:24:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd2bfff640 1 -- 172.21.15.31:0/2688542007 learned_addr learned my addr 172.21.15.31:0/2688542007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd39867740 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd6e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800c890 con 0x55ccdb9972f0 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800f0f0 con 0x55ccdb9c01b0 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd30e96640 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd6e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.615+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800cee0 con 0x55ccdb99f4d0 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.616+0000 7efd2b7fe640 1 -- 172.21.15.31:0/2688542007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 msgr2=0x55ccdbd6e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.616+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd6e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.616+0000 7efd2b7fe640 1 -- 172.21.15.31:0/2688542007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 msgr2=0x55ccdbd670f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.616+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd670f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.616+0000 7efd2b7fe640 1 -- 172.21.15.31:0/2688542007 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccdb97efb0 con 0x55ccdb99f4d0 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.617+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd7d0e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55ccdbd83440 tx=0x7efd20009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac94269bb2ce18d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.617+0000 7efd11ffb640 1 -- 172.21.15.31:0/2688542007 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20012540 con 0x55ccdb99f4d0 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.617+0000 7efd11ffb640 1 -- 172.21.15.31:0/2688542007 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd20012b90 con 0x55ccdb99f4d0 2024-09-21T06:24:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.617+0000 7efd11ffb640 1 -- 172.21.15.31:0/2688542007 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2001baa0 con 0x55ccdb99f4d0 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.618+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 msgr2=0x55ccdbd7d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.618+0000 7efd39867740 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd7d0e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55ccdbd83440 tx=0x7efd20009b30 comp rx=0 tx=0).stop 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.618+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 shutdown_connections 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.618+0000 7efd39867740 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbd6e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.618+0000 7efd39867740 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd7d0e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.619+0000 7efd39867740 1 --2- 172.21.15.31:0/2688542007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd670f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.619+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 >> 172.21.15.31:0/2688542007 conn(0x55ccdb9b4300 msgr2=0x55ccdbd669e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.619+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 shutdown_connections 2024-09-21T06:24:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.619+0000 7efd39867740 1 -- 172.21.15.31:0/2688542007 wait complete. 2024-09-21T06:24:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.620+0000 7efd39867740 1 -- start start 2024-09-21T06:24:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.620+0000 7efd39867740 1 Processor -- start 2024-09-21T06:24:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd39867740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd39867740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd77250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd2b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd77250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd2b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd77250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49498/0 (socket says 172.21.15.31:49498) 2024-09-21T06:24:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd76c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60552/0 (socket says 172.21.15.31:60552) 2024-09-21T06:24:24.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.621+0000 7efd39867740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbde4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd2b7fe640 1 -- 172.21.15.31:0/359274685 learned_addr learned my addr 172.21.15.31:0/359274685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800d5b0 con 0x55ccdb9972f0 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800ea20 con 0x55ccdb9c01b0 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd30e96640 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbde4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800dc80 con 0x55ccdb99f4d0 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd2b7fe640 1 -- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 msgr2=0x55ccdbde4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbde4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd2b7fe640 1 -- 172.21.15.31:0/359274685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 msgr2=0x55ccdbd76c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd76c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.622+0000 7efd2b7fe640 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccdbde4dd0 con 0x55ccdb99f4d0 2024-09-21T06:24:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.623+0000 7efd39867740 10 client.0 _ll_register_callbacks cb 0x7efd24130890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.623+0000 7efd2b7fe640 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd77250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd20012fa0 tx=0x7efd20012fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2502a9f10fe730e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.623+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd20012b90 con 0x55ccdb99f4d0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.624+0000 7efd0b7fe640 10 client.27871 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.624+0000 7efd0b7fe640 1 -- 172.21.15.31:0/359274685 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2001b610 con 0x55ccdb99f4d0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.624+0000 7efd0b7fe640 1 -- 172.21.15.31:0/359274685 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7efd20012750 con 0x55ccdb99f4d0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.625+0000 7efd0b7fe640 1 -- 172.21.15.31:0/359274685 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd20012950 con 0x55ccdb99f4d0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.625+0000 7efd39867740 10 client.27871 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.625+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55ccdbded110 con 0x55ccdb99f4d0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.626+0000 7efd0b7fe640 1 -- 172.21.15.31:0/359274685 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efd2001b980 con 0x55ccdb99f4d0 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.626+0000 7efd39867740 10 client.27871 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.627+0000 7efd39867740 10 client.27871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:24.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.627+0000 7efd39867740 20 client.27871 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.627+0000 7efd39867740 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb9541c0 0x55ccdbdef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.627+0000 7efd39867740 4 client.27871 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:24:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.627+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7efd20012b90 con 0x55ccdb9541c0 2024-09-21T06:24:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.627+0000 7efd2bfff640 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb9541c0 0x55ccdbdef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.628+0000 7efd0b7fe640 10 client.27871 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.628+0000 7efd2bfff640 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb9541c0 0x55ccdbdef6d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10770521aa3c431 server_cookie=855f697de0920b20 in_seq=0 out_seq=0 2024-09-21T06:24:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.646+0000 7efd0b7fe640 1 -- 172.21.15.31:0/359274685 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7efd20012b90 con 0x55ccdb9541c0 2024-09-21T06:24:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.646+0000 7efd0b7fe640 10 client.27871 handle_command_reply: tid=0 2024-09-21T06:24:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.646+0000 7efd39867740 1 client.27871 shutdown 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 msgr2=0x55ccdbd77250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd77250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efd20012fa0 tx=0x7efd20012fd0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 shutdown_connections 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55ccdb9541c0 0x55ccdbdef6d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ccdb9c01b0 0x55ccdbde4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccdb99f4d0 0x55ccdbd77250 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd0b7fe640 0 client.27871 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 --2- 172.21.15.31:0/359274685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccdb9972f0 0x55ccdbd76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 >> 172.21.15.31:0/359274685 conn(0x55ccdb9b4300 msgr2=0x55ccdbd63880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 shutdown_connections 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 1 -- 172.21.15.31:0/359274685 wait complete. 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.648+0000 7efd39867740 20 client.27871 trim_cache size 0 max 16384 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:24:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.649+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd140b6780 msgr2=0x7efd140b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.649+0000 7efd328f9640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd140b6780 0x7efd140b8c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009bb0 tx=0x7efd1c009370 comp rx=0 tx=0).stop 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.649+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 msgr2=0x7efd2c1204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.649+0000 7efd328f9640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c1204e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efd2402aed0 tx=0x7efd240039f0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.650+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 shutdown_connections 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.650+0000 7efd328f9640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7efd140b6780 0x7efd140b8c90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.650+0000 7efd328f9640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd2c110350 0x7efd2c119d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.650+0000 7efd328f9640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd2c10fa00 0x7efd2c1197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.650+0000 7efd328f9640 1 --2- 172.21.15.31:0/2341911978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd2c10a660 0x7efd2c1204e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.650+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 >> 172.21.15.31:0/2341911978 conn(0x7efd2c070dd0 msgr2=0x7efd2c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.651+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 shutdown_connections 2024-09-21T06:24:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:24.651+0000 7efd328f9640 1 -- 172.21.15.31:0/2341911978 wait complete. 2024-09-21T06:24:24.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.814+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63ec0743a0 con 0x7f63ec1083d0 2024-09-21T06:24:24.817 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:24:24.817 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.815+0000 7f63d3fff640 1 -- 172.21.15.31:0/3668533082 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f63e00ad610 con 0x7f63ec1083d0 2024-09-21T06:24:24.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63d40b67b0 msgr2=0x7f63d40b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63d40b67b0 0x7f63d40b8cc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc00b2c0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec1083d0 msgr2=0x7f63ec06e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec1083d0 0x7f63ec06e0f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f63ec06ab20 tx=0x7f63e000db30 comp rx=0 tx=0).stop 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 shutdown_connections 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63ec108d20 0x7f63ec06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ec1083d0 0x7f63ec06e0f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f63d40b67b0 0x7f63d40b8cc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 --2- 172.21.15.31:0/3668533082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63ec102ae0 0x7f63ec074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:24.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.819+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 >> 172.21.15.31:0/3668533082 conn(0x7f63ec079790 msgr2=0x7f63ec1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:24.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.820+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 shutdown_connections 2024-09-21T06:24:24.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:24.822+0000 7f63d1ffb640 1 -- 172.21.15.31:0/3668533082 wait complete. 2024-09-21T06:24:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 765.0 s, 1351.4 tps, lat 74.898 ms stddev 84.175 2024-09-21T06:24:25.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:24:25.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:25 smithi031 ceph-mon[31237]: pgmap v719: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.6 MiB/s wr, 580 op/s 2024-09-21T06:24:25.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:25 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3668533082' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:26.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:25 smithi110 ceph-mon[34550]: pgmap v719: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.6 MiB/s wr, 580 op/s 2024-09-21T06:24:26.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:25 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3668533082' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:26.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:25 smithi107 ceph-mon[34380]: pgmap v719: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.6 MiB/s wr, 580 op/s 2024-09-21T06:24:26.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:25 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3668533082' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:24:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:26.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.914+0000 7fa6d3a37640 1 -- 172.21.15.31:0/2761066591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc111f40 msgr2=0x7fa6cc10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.914+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/2761066591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc111f40 0x7fa6cc10a3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009860 tx=0x7fa6c802a340 comp rx=0 tx=0).stop 2024-09-21T06:24:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.915+0000 7fa6d3a37640 1 -- 172.21.15.31:0/2761066591 shutdown_connections 2024-09-21T06:24:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.915+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/2761066591 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 0x7fa6cc11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.915+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/2761066591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6cc10a9e0 0x7fa6cc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.915+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/2761066591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc111f40 0x7fa6cc10a3c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.915+0000 7fa6d3a37640 1 -- 172.21.15.31:0/2761066591 >> 172.21.15.31:0/2761066591 conn(0x7fa6cc0712f0 msgr2=0x7fa6cc073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.916+0000 7fa6d3a37640 1 -- 172.21.15.31:0/2761066591 shutdown_connections 2024-09-21T06:24:26.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.918+0000 7fa6d3a37640 1 -- 172.21.15.31:0/2761066591 wait complete. 2024-09-21T06:24:26.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.922+0000 7fa6d3a37640 1 Processor -- start 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.926+0000 7fa6d3a37640 1 -- start start 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d3a37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc10a9e0 0x7fa6cc1a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d3a37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6cc111f40 0x7fa6cc1a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d3a37640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 0x7fa6cc1aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d3a37640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc122900 con 0x7fa6cc111f40 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d3a37640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc122780 con 0x7fa6cc10a9e0 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d3a37640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6cc122a80 con 0x7fa6cc11cc30 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d0fd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6cc111f40 0x7fa6cc1a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d17d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc10a9e0 0x7fa6cc1a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d1fd4640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 0x7fa6cc1aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d17d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc10a9e0 0x7fa6cc1a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53784/0 (socket says 172.21.15.31:53784) 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d17d3640 1 -- 172.21.15.31:0/3255633584 learned_addr learned my addr 172.21.15.31:0/3255633584 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d1fd4640 1 -- 172.21.15.31:0/3255633584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc10a9e0 msgr2=0x7fa6cc1a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d1fd4640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc10a9e0 0x7fa6cc1a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d1fd4640 1 -- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6cc111f40 msgr2=0x7fa6cc1a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.927+0000 7fa6d1fd4640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6cc111f40 0x7fa6cc1a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6d1fd4640 1 -- 172.21.15.31:0/3255633584 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6cc1af350 con 0x7fa6cc11cc30 2024-09-21T06:24:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6d1fd4640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 0x7fa6cc1aec50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00e710 tx=0x7fa6bc00ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6c27fc640 1 -- 172.21.15.31:0/3255633584 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc004230 con 0x7fa6cc11cc30 2024-09-21T06:24:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6cc1af640 con 0x7fa6cc11cc30 2024-09-21T06:24:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6cc112380 con 0x7fa6cc11cc30 2024-09-21T06:24:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6c27fc640 1 -- 172.21.15.31:0/3255633584 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6bc0043d0 con 0x7fa6cc11cc30 2024-09-21T06:24:26.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.928+0000 7fa6c27fc640 1 -- 172.21.15.31:0/3255633584 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc0047c0 con 0x7fa6cc11cc30 2024-09-21T06:24:26.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.933+0000 7fa6d9fab740 1 Processor -- start 2024-09-21T06:24:26.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6c27fc640 1 -- 172.21.15.31:0/3255633584 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa6bc004ac0 con 0x7fa6cc11cc30 2024-09-21T06:24:26.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 -- start start 2024-09-21T06:24:26.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe328390 0x561bbe6f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561bbe666ea0 con 0x561bbe328390 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561bbe6562d0 con 0x561bbe349070 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.935+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561bbe65a0e0 con 0x561bbe3201b0 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53794/0 (socket says 172.21.15.31:53794) 2024-09-21T06:24:26.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/2186539468 learned_addr learned my addr 172.21.15.31:0/2186539468 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe328390 0x561bbe6f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/2186539468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 msgr2=0x561bbe6f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/2186539468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe328390 msgr2=0x561bbe6f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe328390 0x561bbe6f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/2186539468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bbe3b4650 con 0x561bbe349070 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.936+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f06a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4011790 tx=0x7fa6c4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9893b4bbb068802 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.937+0000 7fa6b77fe640 1 -- 172.21.15.31:0/2186539468 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003a30 con 0x561bbe349070 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.937+0000 7fa6b77fe640 1 -- 172.21.15.31:0/2186539468 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c400fcf0 con 0x561bbe349070 2024-09-21T06:24:26.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.937+0000 7fa6b77fe640 1 -- 172.21.15.31:0/2186539468 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40045a0 con 0x561bbe349070 2024-09-21T06:24:26.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.938+0000 7fa6c27fc640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa6b80b67b0 0x7fa6b80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.938+0000 7fa6d9fab740 1 -- 172.21.15.31:0/2186539468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 msgr2=0x561bbe6f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.938+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f06a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4011790 tx=0x7fa6c4004a40 comp rx=0 tx=0).stop 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.939+0000 7fa6d9fab740 1 -- 172.21.15.31:0/2186539468 shutdown_connections 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.939+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe328390 0x561bbe6f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.939+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.939+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/2186539468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f06a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.939+0000 7fa6d9fab740 1 -- 172.21.15.31:0/2186539468 >> 172.21.15.31:0/2186539468 conn(0x561bbe33d460 msgr2=0x561bbe6ee590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.939+0000 7fa6c27fc640 1 -- 172.21.15.31:0/3255633584 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fa6bc0e75e0 con 0x7fa6cc11cc30 2024-09-21T06:24:26.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.940+0000 7fa6d9fab740 1 -- 172.21.15.31:0/2186539468 shutdown_connections 2024-09-21T06:24:26.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.940+0000 7fa6d9fab740 1 -- 172.21.15.31:0/2186539468 wait complete. 2024-09-21T06:24:26.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.941+0000 7fa6d9fab740 1 -- start start 2024-09-21T06:24:26.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.941+0000 7fa6d9fab740 1 Processor -- start 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe3201b0 0x561bbe6fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe328390 0x561bbe6fc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe708270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561bbe4b7700 con 0x561bbe3201b0 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561bbe5f1160 con 0x561bbe349070 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561bbe6fef20 con 0x561bbe328390 2024-09-21T06:24:26.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.942+0000 7fa6d9fab740 10 client.0 _ll_register_callbacks cb 0x561bbe60a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d1fd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe708270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d1fd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe708270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53800/0 (socket says 172.21.15.31:53800) 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d1fd4640 1 -- 172.21.15.31:0/3600609842 learned_addr learned my addr 172.21.15.31:0/3600609842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe328390 0x561bbe6fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa6b80b67b0 0x7fa6b80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe3201b0 0x561bbe6fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 msgr2=0x561bbe708270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe708270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/3600609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe3201b0 msgr2=0x561bbe6fbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe3201b0 0x561bbe6fbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.943+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/3600609842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bbe61ec90 con 0x561bbe328390 2024-09-21T06:24:26.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.944+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe328390 0x561bbe6fc370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4004120 tx=0x7fa6c4003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b011b7e3f64bff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.944+0000 7fa6b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.944+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa6b80b67b0 0x7fa6b80b8cc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa6c802a930 tx=0x7fa6c8009490 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.945+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3600609842 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4004730 con 0x561bbe328390 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.945+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3600609842 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fa6c4004ab0 con 0x561bbe328390 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.945+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c400fcf0 con 0x561bbe328390 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.946+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3600609842 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa6c4004cd0 con 0x561bbe328390 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.946+0000 7fa6d9fab740 10 client.27880 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:26.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.946+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561bbe70ac90 con 0x561bbe328390 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3600609842 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c400fcf0 con 0x561bbe328390 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3600609842 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa6c4027410 con 0x561bbe328390 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6d9fab740 10 client.27880 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6d9fab740 10 client.27880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6d9fab740 20 client.27880 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe3079f0 0x561bbe70dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6d9fab740 4 client.27880 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:26.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.948+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bbe668df0 con 0x561bbe3079f0 2024-09-21T06:24:26.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.949+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe3079f0 0x561bbe70dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.950+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe3079f0 0x561bbe70dff0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa85a419cd9e90a0 server_cookie=18c10bfcf0f0492d in_seq=0 out_seq=0 2024-09-21T06:24:26.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.950+0000 7fa6b4ff9640 10 client.27880 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:26.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.950+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3600609842 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561bbe668df0 con 0x561bbe3079f0 2024-09-21T06:24:26.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.950+0000 7fa6b4ff9640 10 client.27880 handle_command_reply: tid=0 2024-09-21T06:24:26.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.950+0000 7fa6d9fab740 1 client.27880 shutdown 2024-09-21T06:24:26.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe328390 msgr2=0x561bbe6fc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe328390 0x561bbe6fc370 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4004120 tx=0x7fa6c4003ce0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 shutdown_connections 2024-09-21T06:24:26.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe3079f0 0x561bbe70dff0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe708270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe328390 0x561bbe6fc370 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3600609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe3201b0 0x561bbe6fbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 >> 172.21.15.31:0/3600609842 conn(0x561bbe33d460 msgr2=0x561bbe6f3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6b4ff9640 0 client.27880 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:26.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 shutdown_connections 2024-09-21T06:24:26.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3600609842 wait complete. 2024-09-21T06:24:26.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.951+0000 7fa6d9fab740 20 client.27880 trim_cache size 0 max 16384 2024-09-21T06:24:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 Processor -- start 2024-09-21T06:24:26.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 -- start start 2024-09-21T06:24:26.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:26 smithi031 ceph-mon[31237]: pgmap v720: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 5.8 MiB/s wr, 724 op/s 2024-09-21T06:24:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe2dcda0 0x561bbe713620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b000c800 con 0x561bbe2dcda0 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b000f060 con 0x561bbe349070 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b000ce50 con 0x561bbe3201b0 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d17d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d17d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49534/0 (socket says 172.21.15.31:49534) 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.977+0000 7fa6d17d3640 1 -- 172.21.15.31:0/1700885314 learned_addr learned my addr 172.21.15.31:0/1700885314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d17d3640 1 -- 172.21.15.31:0/1700885314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 msgr2=0x561bbe6f9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d17d3640 1 -- 172.21.15.31:0/1700885314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe2dcda0 msgr2=0x561bbe713620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe2dcda0 0x561bbe713620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d17d3640 1 -- 172.21.15.31:0/1700885314 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bbe6f47b0 con 0x561bbe3201b0 2024-09-21T06:24:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f9020 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009980 tx=0x7fa6c803a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47c2b1f7df5a683f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6b6ffd640 1 -- 172.21.15.31:0/1700885314 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8037040 con 0x561bbe3201b0 2024-09-21T06:24:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6b6ffd640 1 -- 172.21.15.31:0/1700885314 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c80054f0 con 0x561bbe3201b0 2024-09-21T06:24:26.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.978+0000 7fa6b6ffd640 1 -- 172.21.15.31:0/1700885314 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8004ae0 con 0x561bbe3201b0 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 -- 172.21.15.31:0/1700885314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 msgr2=0x561bbe6f9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f9020 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009980 tx=0x7fa6c803a000 comp rx=0 tx=0).stop 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 -- 172.21.15.31:0/1700885314 shutdown_connections 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe2dcda0 0x561bbe713620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe349070 0x561bbe6f9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/1700885314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe3201b0 0x561bbe6f9020 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.979+0000 7fa6d9fab740 1 -- 172.21.15.31:0/1700885314 >> 172.21.15.31:0/1700885314 conn(0x561bbe33d460 msgr2=0x561bbe6eddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.980+0000 7fa6d9fab740 1 -- 172.21.15.31:0/1700885314 shutdown_connections 2024-09-21T06:24:26.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.980+0000 7fa6d9fab740 1 -- 172.21.15.31:0/1700885314 wait complete. 2024-09-21T06:24:26.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.980+0000 7fa6d9fab740 1 -- start start 2024-09-21T06:24:26.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.980+0000 7fa6d9fab740 1 Processor -- start 2024-09-21T06:24:26.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 0x561bbe704b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe3201b0 0x561bbe6f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe349070 0x561bbe6f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b000d520 con 0x561bbe349070 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b000e990 con 0x561bbe3201b0 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d17d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 0x561bbe704b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b000dbf0 con 0x561bbe2dcda0 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d17d3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 0x561bbe704b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:49548/0 (socket says 172.21.15.31:49548) 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d17d3640 1 -- 172.21.15.31:0/3928124518 learned_addr learned my addr 172.21.15.31:0/3928124518 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.981+0000 7fa6d9fab740 10 client.0 _ll_register_callbacks cb 0x7fa6c4011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe3201b0 0x561bbe6f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/3928124518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 msgr2=0x561bbe704b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:26.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 0x561bbe704b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/3928124518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe349070 msgr2=0x561bbe6f94d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:24:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe349070 0x561bbe6f94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d0fd2640 1 -- 172.21.15.31:0/3928124518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bbe6f9c60 con 0x561bbe3201b0 2024-09-21T06:24:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.982+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 0x561bbe704b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.983+0000 7fa6d0fd2640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe3201b0 0x561bbe6f8e50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4027950 tx=0x7fa6c4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb4be42e79c0d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:26.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.983+0000 7fa6b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.983+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3928124518 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40279d0 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.984+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c80054f0 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.984+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3928124518 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fa6c4027b70 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.984+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3928124518 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa6c4004530 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.984+0000 7fa6d9fab740 10 client.27845 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.984+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561bbe778880 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.984+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3928124518 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c80054f0 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3928124518 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa6c4010430 con 0x561bbe3201b0 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d9fab740 10 client.27845 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d9fab740 10 client.27845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d9fab740 20 client.27845 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:26.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe778f70 0x561bbe77b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d9fab740 4 client.27845 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bbe6648a0 con 0x561bbe778f70 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe778f70 0x561bbe77b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6b4ff9640 10 client.27845 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.985+0000 7fa6d17d3640 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe778f70 0x561bbe77b390 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a138f09dbd98d897 server_cookie=e49c4a450f328394 in_seq=0 out_seq=0 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6b4ff9640 1 -- 172.21.15.31:0/3928124518 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bbe6648a0 con 0x561bbe778f70 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6b4ff9640 10 client.27845 handle_command_reply: tid=0 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 client.27845 shutdown 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe3201b0 msgr2=0x561bbe6f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe3201b0 0x561bbe6f8e50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4027950 tx=0x7fa6c4009500 comp rx=0 tx=0).stop 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 shutdown_connections 2024-09-21T06:24:26.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561bbe778f70 0x561bbe77b390 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561bbe349070 0x561bbe6f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bbe3201b0 0x561bbe6f8e50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6b4ff9640 0 client.27845 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:26.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 --2- 172.21.15.31:0/3928124518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561bbe2dcda0 0x561bbe704b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 >> 172.21.15.31:0/3928124518 conn(0x561bbe33d460 msgr2=0x561bbe6fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 shutdown_connections 2024-09-21T06:24:26.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 1 -- 172.21.15.31:0/3928124518 wait complete. 2024-09-21T06:24:26.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.986+0000 7fa6d9fab740 20 client.27845 trim_cache size 0 max 16384 2024-09-21T06:24:26.991 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa6b80b67b0 msgr2=0x7fa6b80b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa6b80b67b0 0x7fa6b80b8cc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa6c802a930 tx=0x7fa6c8009490 comp rx=0 tx=0).stop 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 msgr2=0x7fa6cc1aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 0x7fa6cc1aec50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc00e710 tx=0x7fa6bc00ebd0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 shutdown_connections 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6cc11cc30 0x7fa6cc1aec50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fa6b80b67b0 0x7fa6b80b8cc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6cc111f40 0x7fa6cc1a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 --2- 172.21.15.31:0/3255633584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6cc10a9e0 0x7fa6cc1a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:26.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 >> 172.21.15.31:0/3255633584 conn(0x7fa6cc0712f0 msgr2=0x7fa6cc11d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:26.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 shutdown_connections 2024-09-21T06:24:26.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:26.988+0000 7fa6d3a37640 1 -- 172.21.15.31:0/3255633584 wait complete. 2024-09-21T06:24:27.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:26 smithi110 ceph-mon[34550]: pgmap v720: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 5.8 MiB/s wr, 724 op/s 2024-09-21T06:24:27.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:26 smithi107 ceph-mon[34380]: pgmap v720: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 5.8 MiB/s wr, 724 op/s 2024-09-21T06:24:28.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:24:28.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:24:29.006 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:29 smithi031 ceph-mon[31237]: pgmap v721: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.1 MiB/s wr, 758 op/s 2024-09-21T06:24:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:29.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:29 smithi110 ceph-mon[34550]: pgmap v721: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.1 MiB/s wr, 758 op/s 2024-09-21T06:24:29.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:29 smithi107 ceph-mon[34380]: pgmap v721: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.1 MiB/s wr, 758 op/s 2024-09-21T06:24:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.706+0000 7fabc882e640 1 -- 172.21.15.31:0/463358448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410a590 msgr2=0x7fabc410a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.706+0000 7fabc882e640 1 --2- 172.21.15.31:0/463358448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410a590 0x7fabc410a9a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009900 tx=0x7fabb402a420 comp rx=0 tx=0).stop 2024-09-21T06:24:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.707+0000 7fabc882e640 1 -- 172.21.15.31:0/463358448 shutdown_connections 2024-09-21T06:24:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.707+0000 7fabc882e640 1 --2- 172.21.15.31:0/463358448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc4116640 0x7fabc4118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.707+0000 7fabc882e640 1 --2- 172.21.15.31:0/463358448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410aee0 0x7fabc4115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.707+0000 7fabc882e640 1 --2- 172.21.15.31:0/463358448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410a590 0x7fabc410a9a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.707+0000 7fabc882e640 1 -- 172.21.15.31:0/463358448 >> 172.21.15.31:0/463358448 conn(0x7fabc4070ec0 msgr2=0x7fabc40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:29.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.708+0000 7fabc882e640 1 -- 172.21.15.31:0/463358448 shutdown_connections 2024-09-21T06:24:29.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.708+0000 7fabc882e640 1 -- 172.21.15.31:0/463358448 wait complete. 2024-09-21T06:24:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.709+0000 7fabc882e640 1 Processor -- start 2024-09-21T06:24:29.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.709+0000 7fabc882e640 1 -- start start 2024-09-21T06:24:29.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.710+0000 7fabc882e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 0x7fabc41aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.710+0000 7fabc882e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 0x7fabc41aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.710+0000 7fabc882e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc4116640 0x7fabc41aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.710+0000 7fabc882e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabc411c3b0 con 0x7fabc410aee0 2024-09-21T06:24:29.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc882e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabc411c230 con 0x7fabc410a590 2024-09-21T06:24:29.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 0x7fabc41aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 0x7fabc41aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53622/0 (socket says 172.21.15.31:53622) 2024-09-21T06:24:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 0x7fabc41aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 0x7fabc41aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44196/0 (socket says 172.21.15.31:44196) 2024-09-21T06:24:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc1d9b640 1 -- 172.21.15.31:0/1368430757 learned_addr learned my addr 172.21.15.31:0/1368430757 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabc411c530 con 0x7fabc4116640 2024-09-21T06:24:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.711+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc4116640 0x7fabc41aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.712+0000 7fabc259c640 1 -- 172.21.15.31:0/1368430757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc4116640 msgr2=0x7fabc41aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.712+0000 7fabc259c640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc4116640 0x7fabc41aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.712+0000 7fabc259c640 1 -- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 msgr2=0x7fabc41aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.712+0000 7fabc259c640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 0x7fabc41aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.712+0000 7fabc259c640 1 -- 172.21.15.31:0/1368430757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc41b53f0 con 0x7fabc410a590 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.712+0000 7fabc1d9b640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 0x7fabc41aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.713+0000 7fabc259c640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 0x7fabc41aa090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fabb402aa10 tx=0x7fabb4003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.713+0000 7fabb37fe640 1 -- 172.21.15.31:0/1368430757 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb403f070 con 0x7fabc410a590 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.713+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc41b5680 con 0x7fabc410a590 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.713+0000 7fabb37fe640 1 -- 172.21.15.31:0/1368430757 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb4003e40 con 0x7fabc410a590 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.713+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc41b5b80 con 0x7fabc410a590 2024-09-21T06:24:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.713+0000 7fabb37fe640 1 -- 172.21.15.31:0/1368430757 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4035030 con 0x7fabc410a590 2024-09-21T06:24:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.717+0000 7fabceda2740 1 Processor -- start 2024-09-21T06:24:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.717+0000 7fabceda2740 1 -- start start 2024-09-21T06:24:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.718+0000 7fabb37fe640 1 -- 172.21.15.31:0/1368430757 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fabb4042070 con 0x7fabc410a590 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.719+0000 7fabb37fe640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabac0b67b0 0x7fabac0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.719+0000 7fabb37fe640 1 -- 172.21.15.31:0/1368430757 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fabb4107ae0 con 0x7fabc410a590 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabceda2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabceda2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabceda2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabceda2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b93061ee60 con 0x55b9302e0670 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabceda2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b93060e290 con 0x55b930301350 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabceda2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b9306120a0 con 0x55b9302d8490 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabc1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabc1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306a8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52188/0 (socket says 172.21.15.31:52188) 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabc1d9b640 1 -- 172.21.15.31:0/2110870348 learned_addr learned my addr 172.21.15.31:0/2110870348 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.720+0000 7fabc259c640 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.721+0000 7fabc259c640 1 -- 172.21.15.31:0/2110870348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 msgr2=0x55b9306a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.721+0000 7fabc259c640 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.721+0000 7fabc259c640 1 -- 172.21.15.31:0/2110870348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 msgr2=0x55b9306a8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.721+0000 7fabc259c640 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.721+0000 7fabc259c640 1 -- 172.21.15.31:0/2110870348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93036c930 con 0x55b930301350 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.721+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabac0b67b0 0x7fabac0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.722+0000 7fabc259c640 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306a8430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fabb41051e0 tx=0x7fabb403b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1477804d116037b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.722+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabac0b67b0 0x7fabac0b8cc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00a5f0 tx=0x7fabbc009990 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.722+0000 7fabb0ff9640 1 -- 172.21.15.31:0/2110870348 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4049070 con 0x55b930301350 2024-09-21T06:24:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.722+0000 7fabb0ff9640 1 -- 172.21.15.31:0/2110870348 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb40cdfe0 con 0x55b930301350 2024-09-21T06:24:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.724+0000 7fabceda2740 1 -- 172.21.15.31:0/2110870348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 msgr2=0x55b9306a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.724+0000 7fabb0ff9640 1 -- 172.21.15.31:0/2110870348 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4049070 con 0x55b930301350 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.724+0000 7fabceda2740 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306a8430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fabb41051e0 tx=0x7fabb403b790 comp rx=0 tx=0).stop 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.724+0000 7fabceda2740 1 -- 172.21.15.31:0/2110870348 shutdown_connections 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.724+0000 7fabceda2740 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306a8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.725+0000 7fabceda2740 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.725+0000 7fabceda2740 1 --2- 172.21.15.31:0/2110870348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306a8430 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.725+0000 7fabceda2740 1 -- 172.21.15.31:0/2110870348 >> 172.21.15.31:0/2110870348 conn(0x55b9302f5740 msgr2=0x55b9306a6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.725+0000 7fabceda2740 1 -- 172.21.15.31:0/2110870348 shutdown_connections 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.725+0000 7fabceda2740 1 -- 172.21.15.31:0/2110870348 wait complete. 2024-09-21T06:24:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.726+0000 7fabceda2740 1 -- start start 2024-09-21T06:24:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.726+0000 7fabceda2740 1 Processor -- start 2024-09-21T06:24:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.727+0000 7fabceda2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabceda2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302e0670 0x55b9306b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabceda2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b930301350 0x55b9306bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53644/0 (socket says 172.21.15.31:53644) 2024-09-21T06:24:29.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabc259c640 1 -- 172.21.15.31:0/4034799617 learned_addr learned my addr 172.21.15.31:0/4034799617 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:29.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55b93046f890 con 0x55b930301350 2024-09-21T06:24:29.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9305a8d40 con 0x55b9302d8490 2024-09-21T06:24:29.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b93060e290 con 0x55b9302e0670 2024-09-21T06:24:29.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b930301350 0x55b9306bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.729+0000 7fabceda2740 10 client.0 _ll_register_callbacks cb 0x55b93060cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.730+0000 7fabc259c640 1 -- 172.21.15.31:0/4034799617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302e0670 msgr2=0x55b9306b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.730+0000 7fabc259c640 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302e0670 0x55b9306b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.730+0000 7fabc259c640 1 -- 172.21.15.31:0/4034799617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b930301350 msgr2=0x55b9306bfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.730+0000 7fabc259c640 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b930301350 0x55b9306bfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.730+0000 7fabc259c640 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9306c0120 con 0x55b9302d8490 2024-09-21T06:24:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.731+0000 7fabc259c640 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b3f30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000c00 tx=0x7fabb4031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f1491a3e27a5d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.731+0000 7faba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.731+0000 7faba9ffb640 1 -- 172.21.15.31:0/4034799617 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4049070 con 0x55b9302d8490 2024-09-21T06:24:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.731+0000 7faba9ffb640 1 -- 172.21.15.31:0/4034799617 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fabb40ccd40 con 0x55b9302d8490 2024-09-21T06:24:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.731+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb40cdfe0 con 0x55b9302d8490 2024-09-21T06:24:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.732+0000 7faba9ffb640 1 -- 172.21.15.31:0/4034799617 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fabb40d2050 con 0x55b9302d8490 2024-09-21T06:24:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.732+0000 7fabceda2740 10 client.27857 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.732+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b9306c2a20 con 0x55b9302d8490 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7faba9ffb640 1 -- 172.21.15.31:0/4034799617 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb40cdfe0 con 0x55b9302d8490 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7faba9ffb640 1 -- 172.21.15.31:0/4034799617 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fabb407a070 con 0x55b9302d8490 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7fabceda2740 10 client.27857 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7fabceda2740 10 client.27857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7fabceda2740 20 client.27857 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7fabceda2740 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9302bfcd0 0x55b9306c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7fabceda2740 4 client.27857 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:29.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.733+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b930620db0 con 0x55b9302bfcd0 2024-09-21T06:24:29.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.734+0000 7fabc1d9b640 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9302bfcd0 0x55b9306c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.735+0000 7fabc1d9b640 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9302bfcd0 0x55b9306c5d80 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e5f1d355399653 server_cookie=263cb0741bd8920 in_seq=0 out_seq=0 2024-09-21T06:24:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.735+0000 7faba9ffb640 10 client.27857 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.736+0000 7faba9ffb640 1 -- 172.21.15.31:0/4034799617 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b930620db0 con 0x55b9302bfcd0 2024-09-21T06:24:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.736+0000 7faba9ffb640 10 client.27857 handle_command_reply: tid=0 2024-09-21T06:24:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.736+0000 7fabceda2740 1 client.27857 shutdown 2024-09-21T06:24:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.737+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 msgr2=0x55b9306b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.737+0000 7fabceda2740 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b3f30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000c00 tx=0x7fabb4031f40 comp rx=0 tx=0).stop 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.738+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 shutdown_connections 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b9302bfcd0 0x55b9306c5d80 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b930301350 0x55b9306bfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302e0670 0x55b9306b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7faba9ffb640 0 client.27857 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 --2- 172.21.15.31:0/4034799617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b3f30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 >> 172.21.15.31:0/4034799617 conn(0x55b9302f5740 msgr2=0x55b9306ab610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:29.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 shutdown_connections 2024-09-21T06:24:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 1 -- 172.21.15.31:0/4034799617 wait complete. 2024-09-21T06:24:29.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.739+0000 7fabceda2740 20 client.27857 trim_cache size 0 max 16384 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabceda2740 1 Processor -- start 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabceda2740 1 -- start start 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabceda2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 0x55b9306bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabceda2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabc259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 0x55b9306bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabc259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 0x55b9306bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52200/0 (socket says 172.21.15.31:52200) 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.779+0000 7fabceda2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53652/0 (socket says 172.21.15.31:53652) 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc259c640 1 -- 172.21.15.31:0/2292571071 learned_addr learned my addr 172.21.15.31:0/2292571071 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41bc5c0 con 0x55b9302e0670 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41bee20 con 0x55b9302d8490 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41bcc10 con 0x55b9302bfcd0 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc259c640 1 -- 172.21.15.31:0/2292571071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 msgr2=0x55b9306b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc259c640 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc259c640 1 -- 172.21.15.31:0/2292571071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 msgr2=0x55b9306b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc259c640 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.780+0000 7fabc259c640 1 -- 172.21.15.31:0/2292571071 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9306b4ff0 con 0x55b9302bfcd0 2024-09-21T06:24:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.781+0000 7fabc259c640 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 0x55b9306bfae0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009850 tx=0x7fabb403b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b4ab839c00a302 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.781+0000 7fababfff640 1 -- 172.21.15.31:0/2292571071 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4049020 con 0x55b9302bfcd0 2024-09-21T06:24:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.781+0000 7fababfff640 1 -- 172.21.15.31:0/2292571071 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb40318b0 con 0x55b9302bfcd0 2024-09-21T06:24:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.781+0000 7fababfff640 1 -- 172.21.15.31:0/2292571071 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb40e8080 con 0x55b9302bfcd0 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.783+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 msgr2=0x55b9306bfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.783+0000 7fabceda2740 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 0x55b9306bfae0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009850 tx=0x7fabb403b790 comp rx=0 tx=0).stop 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.783+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 shutdown_connections 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.783+0000 7fabceda2740 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.784+0000 7fabceda2740 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9302d8490 0x55b9306b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.784+0000 7fabceda2740 1 --2- 172.21.15.31:0/2292571071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302bfcd0 0x55b9306bfae0 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009850 tx=0x7fabb403b790 comp rx=0 tx=0).stop 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.784+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 >> 172.21.15.31:0/2292571071 conn(0x55b9302f5740 msgr2=0x55b9306aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.784+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 shutdown_connections 2024-09-21T06:24:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.784+0000 7fabceda2740 1 -- 172.21.15.31:0/2292571071 wait complete. 2024-09-21T06:24:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.784+0000 7fabceda2740 1 -- start start 2024-09-21T06:24:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 Processor -- start 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41bd2e0 con 0x55b9302e0670 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41be750 con 0x55b930301350 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41bd9b0 con 0x55b9302d8490 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabceda2740 10 client.0 _ll_register_callbacks cb 0x7fabb800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabc259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabc259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52214/0 (socket says 172.21.15.31:52214) 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.785+0000 7fabc259c640 1 -- 172.21.15.31:0/1006590314 learned_addr learned my addr 172.21.15.31:0/1006590314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc259c640 1 -- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 msgr2=0x55b9306b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc259c640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc259c640 1 -- 172.21.15.31:0/1006590314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 msgr2=0x55b9306b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc259c640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc259c640 1 -- 172.21.15.31:0/1006590314 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9307407b0 con 0x55b9302d8490 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc2d9d640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabc259c640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306b7bc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fabb40d3620 tx=0x7fabb40d3880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75857c4947314a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7faba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7faba9ffb640 1 -- 172.21.15.31:0/1006590314 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb40e8070 con 0x55b9302d8490 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b93061c860 con 0x55b9302d8490 2024-09-21T06:24:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.786+0000 7faba9ffb640 1 -- 172.21.15.31:0/1006590314 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fabb40c9230 con 0x55b9302d8490 2024-09-21T06:24:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7faba9ffb640 1 -- 172.21.15.31:0/1006590314 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fabb40cdfe0 con 0x55b9302d8490 2024-09-21T06:24:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 10 client.27898 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b9307407b0 con 0x55b9302d8490 2024-09-21T06:24:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7faba9ffb640 1 -- 172.21.15.31:0/1006590314 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b93061c860 con 0x55b9302d8490 2024-09-21T06:24:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7faba9ffb640 1 -- 172.21.15.31:0/1006590314 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fabb407a070 con 0x55b9302d8490 2024-09-21T06:24:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 10 client.27898 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 10 client.27898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 20 client.27898 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b930295080 0x55b930743e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 4 client.27898 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.787+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b930619c90 con 0x55b930295080 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.788+0000 7fabc1d9b640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b930295080 0x55b930743e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.790+0000 7fabc1d9b640 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b930295080 0x55b930743e20 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6bb277a974fe189 server_cookie=72ae98707e7df07 in_seq=0 out_seq=0 2024-09-21T06:24:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.790+0000 7faba9ffb640 10 client.27898 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:29.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.791+0000 7faba9ffb640 1 -- 172.21.15.31:0/1006590314 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b930619c90 con 0x55b930295080 2024-09-21T06:24:29.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.791+0000 7faba9ffb640 10 client.27898 handle_command_reply: tid=0 2024-09-21T06:24:29.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.791+0000 7fabceda2740 1 client.27898 shutdown 2024-09-21T06:24:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 msgr2=0x55b9306b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306b7bc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fabb40d3620 tx=0x7fabb40d3880 comp rx=0 tx=0).stop 2024-09-21T06:24:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 shutdown_connections 2024-09-21T06:24:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55b930295080 0x55b930743e20 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b930301350 0x55b9306b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b9302e0670 0x55b9306b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 --2- 172.21.15.31:0/1006590314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b9302d8490 0x55b9306b7bc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7faba9ffb640 0 client.27898 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 >> 172.21.15.31:0/1006590314 conn(0x55b9302f5740 msgr2=0x55b9306b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 shutdown_connections 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 1 -- 172.21.15.31:0/1006590314 wait complete. 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.792+0000 7fabceda2740 20 client.27898 trim_cache size 0 max 16384 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2", 2024-09-21T06:24:29.795 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:24:29.796 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabac0b67b0 msgr2=0x7fabac0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabac0b67b0 0x7fabac0b8cc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00a5f0 tx=0x7fabbc009990 comp rx=0 tx=0).stop 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 msgr2=0x7fabc41aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 0x7fabc41aa090 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fabb402aa10 tx=0x7fabb4003cd0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 shutdown_connections 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabc4116640 0x7fabc41aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fabac0b67b0 0x7fabac0b8cc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc410aee0 0x7fabc41aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 --2- 172.21.15.31:0/1368430757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabc410a590 0x7fabc41aa090 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 >> 172.21.15.31:0/1368430757 conn(0x7fabc4070ec0 msgr2=0x7fabc410ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:29.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 shutdown_connections 2024-09-21T06:24:29.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:29.793+0000 7fabc882e640 1 -- 172.21.15.31:0/1368430757 wait complete. 2024-09-21T06:24:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 770.0 s, 2419.4 tps, lat 41.490 ms stddev 52.425 2024-09-21T06:24:30.548 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:24:30.989 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:24:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:31 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:24:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:31 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:31 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:24:31.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:31 smithi110 ceph-mon[34550]: pgmap v722: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 6.9 MiB/s wr, 830 op/s 2024-09-21T06:24:31.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:31 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:24:31.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:31 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:31.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:31 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:24:31.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:31 smithi107 ceph-mon[34380]: pgmap v722: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 6.9 MiB/s wr, 830 op/s 2024-09-21T06:24:31.303 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:31 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:24:31.303 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:31 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:31.303 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:31 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:24:31.303 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:31 smithi031 ceph-mon[31237]: pgmap v722: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 6.9 MiB/s wr, 830 op/s 2024-09-21T06:24:31.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 -- 172.21.15.31:0/2462452358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b201048a0 msgr2=0x7f0b20104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/2462452358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b201048a0 0x7f0b20104cb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009900 tx=0x7f0b1402a410 comp rx=0 tx=0).stop 2024-09-21T06:24:31.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 -- 172.21.15.31:0/2462452358 shutdown_connections 2024-09-21T06:24:31.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/2462452358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b2010de50 0x7f0b20110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/2462452358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b20105280 0x7f0b2010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/2462452358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b201048a0 0x7f0b20104cb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 -- 172.21.15.31:0/2462452358 >> 172.21.15.31:0/2462452358 conn(0x7f0b20079790 msgr2=0x7f0b20079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:31.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 -- 172.21.15.31:0/2462452358 shutdown_connections 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.560+0000 7f0b26b7c640 1 -- 172.21.15.31:0/2462452358 wait complete. 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.561+0000 7f0b26b7c640 1 Processor -- start 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.561+0000 7f0b26b7c640 1 -- start start 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b26b7c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b201048a0 0x7f0b2019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b26b7c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105280 0x7f0b2019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b26b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b2010de50 0x7f0b2019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b26b7c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20113af0 con 0x7f0b2010de50 2024-09-21T06:24:31.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b26b7c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20113970 con 0x7f0b20105280 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b26b7c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20113c70 con 0x7f0b201048a0 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b24918640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b201048a0 0x7f0b2019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b25119640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b2010de50 0x7f0b2019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b24918640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b201048a0 0x7f0b2019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52244/0 (socket says 172.21.15.31:52244) 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b24918640 1 -- 172.21.15.31:0/144465631 learned_addr learned my addr 172.21.15.31:0/144465631 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b1ffff640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105280 0x7f0b2019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b1ffff640 1 -- 172.21.15.31:0/144465631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b201048a0 msgr2=0x7f0b2019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b1ffff640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b201048a0 0x7f0b2019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b1ffff640 1 -- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b2010de50 msgr2=0x7f0b2019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b1ffff640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b2010de50 0x7f0b2019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.562+0000 7f0b1ffff640 1 -- 172.21.15.31:0/144465631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b201a0510 con 0x7f0b20105280 2024-09-21T06:24:31.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.563+0000 7f0b1ffff640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105280 0x7f0b2019ab40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b2006ab20 tx=0x7f0b0c00b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:31.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.563+0000 7f0b1dffb640 1 -- 172.21.15.31:0/144465631 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c015020 con 0x7f0b20105280 2024-09-21T06:24:31.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.563+0000 7f0b1dffb640 1 -- 172.21.15.31:0/144465631 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0c011ce0 con 0x7f0b20105280 2024-09-21T06:24:31.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.563+0000 7f0b1dffb640 1 -- 172.21.15.31:0/144465631 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c010d80 con 0x7f0b20105280 2024-09-21T06:24:31.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.563+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201a0800 con 0x7f0b20105280 2024-09-21T06:24:31.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.563+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a6f60 con 0x7f0b20105280 2024-09-21T06:24:31.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.566+0000 7f0b2d0f0740 1 Processor -- start 2024-09-21T06:24:31.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.567+0000 7f0b2d0f0740 1 -- start start 2024-09-21T06:24:31.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.567+0000 7f0b1dffb640 1 -- 172.21.15.31:0/144465631 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0b0c018020 con 0x7f0b20105280 2024-09-21T06:24:31.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.567+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aa2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.567+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aa2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.567+0000 7f0b1dffb640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b100b6700 0x7f0b100b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aa30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650b9a18e50 con 0x5650b96fb070 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650b9a08280 con 0x5650b96da390 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b1dffb640 1 -- 172.21.15.31:0/144465631 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f0b0c0e6cf0 con 0x7f0b20105280 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650b9a0c090 con 0x5650b96d21b0 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b25119640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aa30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b25119640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aa2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b25119640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aa2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44254/0 (socket says 172.21.15.31:44254) 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b25119640 1 -- 172.21.15.31:0/3951430754 learned_addr learned my addr 172.21.15.31:0/3951430754 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b1ffff640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b100b6700 0x7f0b100b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.568+0000 7f0b24918640 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aa2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b24918640 1 -- 172.21.15.31:0/3951430754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 msgr2=0x5650b9aa30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b24918640 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aa30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b24918640 1 -- 172.21.15.31:0/3951430754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 msgr2=0x5650b9aa2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b24918640 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aa2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b24918640 1 -- 172.21.15.31:0/3951430754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b9766650 con 0x5650b96d21b0 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b1ffff640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b100b6700 0x7f0b100b8c10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0ac850 tx=0x7f0b0c010380 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b24918640 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aa2b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b14004770 tx=0x7f0b14031c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3919e6db429fcbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:31.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.569+0000 7f0b0affd640 1 -- 172.21.15.31:0/3951430754 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14035040 con 0x5650b96d21b0 2024-09-21T06:24:31.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.570+0000 7f0b0affd640 1 -- 172.21.15.31:0/3951430754 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b140316a0 con 0x5650b96d21b0 2024-09-21T06:24:31.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.570+0000 7f0b0affd640 1 -- 172.21.15.31:0/3951430754 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1403fcf0 con 0x5650b96d21b0 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3951430754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 msgr2=0x5650b9aa2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aa2b90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b14004770 tx=0x7f0b14031c50 comp rx=0 tx=0).stop 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3951430754 shutdown_connections 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aa30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aa2b90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3951430754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aa2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3951430754 >> 172.21.15.31:0/3951430754 conn(0x5650b96ef460 msgr2=0x5650b9aa0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.571+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3951430754 shutdown_connections 2024-09-21T06:24:31.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.572+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3951430754 wait complete. 2024-09-21T06:24:31.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.572+0000 7f0b2d0f0740 1 -- start start 2024-09-21T06:24:31.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.572+0000 7f0b2d0f0740 1 Processor -- start 2024-09-21T06:24:31.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aadcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b25119640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aadcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5650b9869700 con 0x5650b96fb070 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650b99a3110 con 0x5650b96da390 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5650b9ab0dd0 con 0x5650b96d21b0 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b25119640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aadcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52258/0 (socket says 172.21.15.31:52258) 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b25119640 1 -- 172.21.15.31:0/3738161547 learned_addr learned my addr 172.21.15.31:0/3738161547 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b2d0f0740 10 client.0 _ll_register_callbacks cb 0x5650b99bc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b24918640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.573+0000 7f0b25119640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b25119640 1 -- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 msgr2=0x5650b9aae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b25119640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b25119640 1 -- 172.21.15.31:0/3738161547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 msgr2=0x5650b9aba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b25119640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b25119640 1 -- 172.21.15.31:0/3738161547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b99d0c40 con 0x5650b96d21b0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b25119640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aadcf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b18025e40 tx=0x7f0b18009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569194aa05295689 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0afffff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0afffff640 1 -- 172.21.15.31:0/3738161547 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18019040 con 0x5650b96d21b0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0afffff640 1 -- 172.21.15.31:0/3738161547 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f0b18003d80 con 0x5650b96d21b0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b140316a0 con 0x5650b96d21b0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.574+0000 7f0afffff640 1 -- 172.21.15.31:0/3738161547 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0b18035050 con 0x5650b96d21b0 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.575+0000 7f0b2d0f0740 10 client.27910 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:31.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.575+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5650b9abcb10 con 0x5650b96d21b0 2024-09-21T06:24:31.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.575+0000 7f0afffff640 1 -- 172.21.15.31:0/3738161547 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b140316a0 con 0x5650b96d21b0 2024-09-21T06:24:31.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.575+0000 7f0afffff640 1 -- 172.21.15.31:0/3738161547 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0b18030510 con 0x5650b96d21b0 2024-09-21T06:24:31.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.576+0000 7f0b2d0f0740 10 client.27910 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:31.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.576+0000 7f0b2d0f0740 10 client.27910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:31.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.576+0000 7f0b2d0f0740 20 client.27910 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:31.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.576+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b96b99f0 0x5650b9abffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.576+0000 7f0b2d0f0740 4 client.27910 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.576+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b9a1ada0 con 0x5650b96b99f0 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.577+0000 7f0b24918640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b96b99f0 0x5650b9abffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.577+0000 7f0b24918640 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b96b99f0 0x5650b9abffe0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7fda8efdcf2aee server_cookie=ed8df6014fefa6be in_seq=0 out_seq=0 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.577+0000 7f0afffff640 10 client.27910 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0afffff640 1 -- 172.21.15.31:0/3738161547 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650b9a1ada0 con 0x5650b96b99f0 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0afffff640 10 client.27910 handle_command_reply: tid=0 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 client.27910 shutdown 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 msgr2=0x5650b9aadcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aadcf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b18025e40 tx=0x7f0b18009f40 comp rx=0 tx=0).stop 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 shutdown_connections 2024-09-21T06:24:31.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b96b99f0 0x5650b9abffe0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96fb070 0x5650b9aba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0afffff640 0 client.27910 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96da390 0x5650b9aae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/3738161547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96d21b0 0x5650b9aadcf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 >> 172.21.15.31:0/3738161547 conn(0x5650b96ef460 msgr2=0x5650b9aa5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 shutdown_connections 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.578+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/3738161547 wait complete. 2024-09-21T06:24:31.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.579+0000 7f0b2d0f0740 20 client.27910 trim_cache size 0 max 16384 2024-09-21T06:24:31.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 Processor -- start 2024-09-21T06:24:31.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 -- start start 2024-09-21T06:24:31.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96b99f0 0x5650b9aae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96d21b0 0x5650b9aaaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96da390 0x5650b9aab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ab740 con 0x5650b96b99f0 2024-09-21T06:24:31.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201adfa0 con 0x5650b96d21b0 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201abd90 con 0x5650b96da390 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b25119640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96b99f0 0x5650b9aae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b24918640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96d21b0 0x5650b9aaaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b25119640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96b99f0 0x5650b9aae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44264/0 (socket says 172.21.15.31:44264) 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.605+0000 7f0b25119640 1 -- 172.21.15.31:0/151945159 learned_addr learned my addr 172.21.15.31:0/151945159 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b25119640 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96da390 0x5650b9aab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b24918640 1 -- 172.21.15.31:0/151945159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96da390 msgr2=0x5650b9aab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b24918640 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96da390 0x5650b9aab420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b24918640 1 -- 172.21.15.31:0/151945159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96b99f0 msgr2=0x5650b9aae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b24918640 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96b99f0 0x5650b9aae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b24918640 1 -- 172.21.15.31:0/151945159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b9aaefb0 con 0x5650b96d21b0 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b25119640 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96da390 0x5650b9aab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:24:31.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b24918640 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96d21b0 0x5650b9aaaee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b14006010 tx=0x7f0b1403f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12c1c3b893602bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:31.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b0a7fc640 1 -- 172.21.15.31:0/151945159 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14035040 con 0x5650b96d21b0 2024-09-21T06:24:31.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b0a7fc640 1 -- 172.21.15.31:0/151945159 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b140318d0 con 0x5650b96d21b0 2024-09-21T06:24:31.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.606+0000 7f0b0a7fc640 1 -- 172.21.15.31:0/151945159 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1403f380 con 0x5650b96d21b0 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.607+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/151945159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96d21b0 msgr2=0x5650b9aaaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.607+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96d21b0 0x5650b9aaaee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b14006010 tx=0x7f0b1403f990 comp rx=0 tx=0).stop 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/151945159 shutdown_connections 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96da390 0x5650b9aab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96d21b0 0x5650b9aaaee0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b14006010 tx=0x7f0b1403f990 comp rx=0 tx=0).stop 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/151945159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96b99f0 0x5650b9aae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/151945159 >> 172.21.15.31:0/151945159 conn(0x5650b96ef460 msgr2=0x5650b9aadaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/151945159 shutdown_connections 2024-09-21T06:24:31.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/151945159 wait complete. 2024-09-21T06:24:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 -- start start 2024-09-21T06:24:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.608+0000 7f0b2d0f0740 1 Processor -- start 2024-09-21T06:24:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 0x5650b9ab2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96da390 0x5650b9ab3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96fb070 0x5650b9b239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ac460 con 0x5650b96da390 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ad8d0 con 0x5650b96b99f0 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201acb30 con 0x5650b96fb070 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b2d0f0740 10 client.0 _ll_register_callbacks cb 0x7f0b18010ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.609+0000 7f0b25119640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 0x5650b9ab2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 0x5650b9ab2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53710/0 (socket says 172.21.15.31:53710) 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 -- 172.21.15.31:0/2290431393 learned_addr learned my addr 172.21.15.31:0/2290431393 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b24918640 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96da390 0x5650b9ab3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 -- 172.21.15.31:0/2290431393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96fb070 msgr2=0x5650b9b239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96fb070 0x5650b9b239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 -- 172.21.15.31:0/2290431393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96da390 msgr2=0x5650b9ab3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96da390 0x5650b9ab3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 -- 172.21.15.31:0/2290431393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b9b24050 con 0x5650b96b99f0 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0b25119640 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 0x5650b9ab2ab0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0b180109d0 tx=0x7f0b18009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477c53e948e45cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0afffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.610+0000 7f0afffff640 1 -- 172.21.15.31:0/2290431393 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18030b00 con 0x5650b96b99f0 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.611+0000 7f0afffff640 1 -- 172.21.15.31:0/2290431393 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f0b18030ca0 con 0x5650b96b99f0 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.611+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b140318d0 con 0x5650b96b99f0 2024-09-21T06:24:31.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.611+0000 7f0afffff640 1 -- 172.21.15.31:0/2290431393 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0b18048050 con 0x5650b96b99f0 2024-09-21T06:24:31.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.615+0000 7f0b2d0f0740 10 client.27881 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:31.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.615+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5650b9b2b430 con 0x5650b96b99f0 2024-09-21T06:24:31.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.624+0000 7f0afffff640 1 -- 172.21.15.31:0/2290431393 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18003b80 con 0x5650b96b99f0 2024-09-21T06:24:31.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.624+0000 7f0afffff640 1 -- 172.21.15.31:0/2290431393 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0b18005520 con 0x5650b96b99f0 2024-09-21T06:24:31.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.626+0000 7f0b2d0f0740 10 client.27881 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:31.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.626+0000 7f0b2d0f0740 10 client.27881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:31.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.626+0000 7f0b2d0f0740 20 client.27881 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:31.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.626+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b968eda0 0x5650b9b2db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:31.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.626+0000 7f0b2d0f0740 4 client.27881 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:24:31.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.626+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0b140318d0 con 0x5650b968eda0 2024-09-21T06:24:31.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.635+0000 7f0b24918640 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b968eda0 0x5650b9b2db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.645+0000 7f0b24918640 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b968eda0 0x5650b9b2db90 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5574b8b76e44a9cc server_cookie=31229a3368e9a316 in_seq=0 out_seq=0 2024-09-21T06:24:31.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.645+0000 7f0afffff640 10 client.27881 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:31.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.647+0000 7f0afffff640 1 -- 172.21.15.31:0/2290431393 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9090 (crc 0 0 0) 0x7f0b140318d0 con 0x5650b968eda0 2024-09-21T06:24:31.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.647+0000 7f0afffff640 10 client.27881 handle_command_reply: tid=0 2024-09-21T06:24:31.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.650+0000 7f0b2d0f0740 1 client.27881 shutdown 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 msgr2=0x5650b9ab2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 0x5650b9ab2ab0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0b180109d0 tx=0x7f0b18009e00 comp rx=0 tx=0).stop 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 shutdown_connections 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5650b968eda0 0x5650b9b2db90 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5650b96fb070 0x5650b9b239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5650b96da390 0x5650b9ab3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 --2- 172.21.15.31:0/2290431393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650b96b99f0 0x5650b9ab2ab0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 >> 172.21.15.31:0/2290431393 conn(0x5650b96ef460 msgr2=0x5650b9aa3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:31.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.657+0000 7f0afffff640 0 client.27881 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:31.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.658+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 shutdown_connections 2024-09-21T06:24:31.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.658+0000 7f0b2d0f0740 1 -- 172.21.15.31:0/2290431393 wait complete. 2024-09-21T06:24:31.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.658+0000 7f0b2d0f0740 20 client.27881 trim_cache size 0 max 16384 2024-09-21T06:24:31.662 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:24:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2", 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:24:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:24:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:24:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:24:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:24:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:24:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:24:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.659+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b100b6700 msgr2=0x7f0b100b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.659+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b100b6700 0x7f0b100b8c10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0ac850 tx=0x7f0b0c010380 comp rx=0 tx=0).stop 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.659+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105280 msgr2=0x7f0b2019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.659+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105280 0x7f0b2019ab40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0b2006ab20 tx=0x7f0b0c00b850 comp rx=0 tx=0).stop 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 shutdown_connections 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f0b100b6700 0x7f0b100b8c10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b2010de50 0x7f0b2019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b20105280 0x7f0b2019ab40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 --2- 172.21.15.31:0/144465631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b201048a0 0x7f0b2019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 >> 172.21.15.31:0/144465631 conn(0x7f0b20079790 msgr2=0x7f0b20107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 shutdown_connections 2024-09-21T06:24:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:24:31.660+0000 7f0b26b7c640 1 -- 172.21.15.31:0/144465631 wait complete. 2024-09-21T06:24:32.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:32 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:32.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:32 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:32.312 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:32 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:24:32.442 INFO:tasks.cephfs.filesystem:scrub status for tag:03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2 - {'path': '/', 'tag': '03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2', 'options': 'recursive,force'} 2024-09-21T06:24:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:33 smithi110 ceph-mon[34550]: pgmap v723: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.5 MiB/s wr, 821 op/s 2024-09-21T06:24:33.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:33 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:24:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:33 smithi107 ceph-mon[34380]: pgmap v723: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.5 MiB/s wr, 821 op/s 2024-09-21T06:24:33.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:33 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:24:33.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:33 smithi031 ceph-mon[31237]: pgmap v723: 97 pgs: 97 active+clean; 4.2 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 6.5 MiB/s wr, 821 op/s 2024-09-21T06:24:33.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:33 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:24:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 775.0 s, 1475.6 tps, lat 67.258 ms stddev 122.988 2024-09-21T06:24:35.278 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:35 smithi110 ceph-mon[34550]: pgmap v724: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-21T06:24:35.279 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:35 smithi031 ceph-mon[31237]: pgmap v724: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-21T06:24:35.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:35 smithi107 ceph-mon[34380]: pgmap v724: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 12 MiB/s wr, 1.28k op/s 2024-09-21T06:24:37.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:37 smithi107 ceph-mon[34380]: pgmap v725: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 16 MiB/s wr, 1.62k op/s 2024-09-21T06:24:37.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:37 smithi031 ceph-mon[31237]: pgmap v725: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 16 MiB/s wr, 1.62k op/s 2024-09-21T06:24:37.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:37 smithi110 ceph-mon[34550]: pgmap v725: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 16 MiB/s wr, 1.62k op/s 2024-09-21T06:24:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 -- 172.21.15.31:0/932794580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 msgr2=0x7fb3d010a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/932794580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d010a7d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009ca0 tx=0x7fb3c002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 -- 172.21.15.31:0/932794580 shutdown_connections 2024-09-21T06:24:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/932794580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3d01103a0 0x7fb3d01107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/932794580 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3d010ad10 0x7fb3d010fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/932794580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d010a7d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.815+0000 7fb3d4eec640 1 -- 172.21.15.31:0/932794580 >> 172.21.15.31:0/932794580 conn(0x7fb3d0071220 msgr2=0x7fb3d0073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.820+0000 7fb3d4eec640 1 -- 172.21.15.31:0/932794580 shutdown_connections 2024-09-21T06:24:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.823+0000 7fb3d4eec640 1 -- 172.21.15.31:0/932794580 wait complete. 2024-09-21T06:24:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.824+0000 7fb3d4eec640 1 Processor -- start 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.824+0000 7fb3d4eec640 1 -- start start 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.825+0000 7fb3d4eec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d0117380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.825+0000 7fb3d4eec640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3d010ad10 0x7fb3d01178c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.825+0000 7fb3d4eec640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3d01103a0 0x7fb3d0117e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.825+0000 7fb3d4eec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0113c90 con 0x7fb3d010a3c0 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.825+0000 7fb3d4eec640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0113b10 con 0x7fb3d010ad10 2024-09-21T06:24:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.825+0000 7fb3d4eec640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d0113e10 con 0x7fb3d01103a0 2024-09-21T06:24:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.826+0000 7fb3ce59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d0117380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.826+0000 7fb3ce59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d0117380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44282/0 (socket says 172.21.15.31:44282) 2024-09-21T06:24:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.826+0000 7fb3ce59c640 1 -- 172.21.15.31:0/3021671372 learned_addr learned my addr 172.21.15.31:0/3021671372 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.826+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3d01103a0 0x7fb3d0117e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.827+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3d010ad10 0x7fb3d01178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.828+0000 7fb3ce59c640 1 -- 172.21.15.31:0/3021671372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3d01103a0 msgr2=0x7fb3d0117e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.828+0000 7fb3ce59c640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3d01103a0 0x7fb3d0117e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.829+0000 7fb3ce59c640 1 -- 172.21.15.31:0/3021671372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3d010ad10 msgr2=0x7fb3d01178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.829+0000 7fb3ce59c640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3d010ad10 0x7fb3d01178c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.829+0000 7fb3ce59c640 1 -- 172.21.15.31:0/3021671372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d011ec40 con 0x7fb3d010a3c0 2024-09-21T06:24:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.830+0000 7fb3ce59c640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d0117380 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000f80 tx=0x7fb3c002f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.831+0000 7fb3c77fe640 1 -- 172.21.15.31:0/3021671372 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c002fad0 con 0x7fb3d010a3c0 2024-09-21T06:24:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.831+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d011eed0 con 0x7fb3d010a3c0 2024-09-21T06:24:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.831+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d011f4a0 con 0x7fb3d010a3c0 2024-09-21T06:24:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.831+0000 7fb3c77fe640 1 -- 172.21.15.31:0/3021671372 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c002d070 con 0x7fb3d010a3c0 2024-09-21T06:24:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.832+0000 7fb3c77fe640 1 -- 172.21.15.31:0/3021671372 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00466e0 con 0x7fb3d010a3c0 2024-09-21T06:24:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.835+0000 7fb3dbe58740 1 Processor -- start 2024-09-21T06:24:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.835+0000 7fb3dbe58740 1 -- start start 2024-09-21T06:24:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.836+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f0997f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.836+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b12f0 0x562f0997fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.836+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f09980430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.836+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562f098f46f0 con 0x562f095b94d0 2024-09-21T06:24:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.836+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562f098e3b20 con 0x562f095da1b0 2024-09-21T06:24:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.836+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f098e7930 con 0x562f095b12f0 2024-09-21T06:24:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.837+0000 7fb3ced9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f0997f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.837+0000 7fb3ced9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f0997f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53740/0 (socket says 172.21.15.31:53740) 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.837+0000 7fb3ced9d640 1 -- 172.21.15.31:0/3646385893 learned_addr learned my addr 172.21.15.31:0/3646385893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.838+0000 7fb3ced9d640 1 -- 172.21.15.31:0/3646385893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b12f0 msgr2=0x562f0997fef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.838+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b12f0 0x562f0997fef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.838+0000 7fb3ced9d640 1 -- 172.21.15.31:0/3646385893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 msgr2=0x562f09980430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.838+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f09980430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.838+0000 7fb3c77fe640 1 -- 172.21.15.31:0/3021671372 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb3c002fc70 con 0x7fb3d010a3c0 2024-09-21T06:24:37.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.838+0000 7fb3ced9d640 1 -- 172.21.15.31:0/3646385893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f098a9530 con 0x562f095da1b0 2024-09-21T06:24:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.839+0000 7fb3c77fe640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb3bc0b6880 0x7fb3bc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.840+0000 7fb3c77fe640 1 -- 172.21.15.31:0/3021671372 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fb3c0108eb0 con 0x7fb3d010a3c0 2024-09-21T06:24:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.840+0000 7fb3ce59c640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb3bc0b6880 0x7fb3bc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.840+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f0997f9b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b660 tx=0x7fb3c800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74499e13492d6baa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.841+0000 7fb3c4ff9640 1 -- 172.21.15.31:0/3646385893 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8003bd0 con 0x562f095da1b0 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.841+0000 7fb3c4ff9640 1 -- 172.21.15.31:0/3646385893 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c8015070 con 0x562f095da1b0 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.841+0000 7fb3c4ff9640 1 -- 172.21.15.31:0/3646385893 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c80106a0 con 0x562f095da1b0 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 -- 172.21.15.31:0/3646385893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 msgr2=0x562f0997f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f0997f9b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800b660 tx=0x7fb3c800bb20 comp rx=0 tx=0).stop 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3ce59c640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb3bc0b6880 0x7fb3bc0b8d90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x562f0997e630 tx=0x7fb3c0039ef0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 -- 172.21.15.31:0/3646385893 shutdown_connections 2024-09-21T06:24:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f09980430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b12f0 0x562f0997fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/3646385893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f0997f9b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.842+0000 7fb3dbe58740 1 -- 172.21.15.31:0/3646385893 >> 172.21.15.31:0/3646385893 conn(0x562f095ce300 msgr2=0x562f0997d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.843+0000 7fb3dbe58740 1 -- 172.21.15.31:0/3646385893 shutdown_connections 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.843+0000 7fb3dbe58740 1 -- 172.21.15.31:0/3646385893 wait complete. 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.843+0000 7fb3dbe58740 1 -- start start 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.844+0000 7fb3dbe58740 1 Processor -- start 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.844+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b12f0 0x562f0998b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.844+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f0998b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.844+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x562f0981bab0 con 0x562f095b94d0 2024-09-21T06:24:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562f09889730 con 0x562f095b12f0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562f0998e180 con 0x562f095da1b0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f0998b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f0998b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44298/0 (socket says 172.21.15.31:44298) 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3c7fff640 1 -- 172.21.15.31:0/415772436 learned_addr learned my addr 172.21.15.31:0/415772436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3dbe58740 10 client.0 _ll_register_callbacks cb 0x562f098dd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b12f0 0x562f0998b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.845+0000 7fb3c7fff640 1 -- 172.21.15.31:0/415772436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 msgr2=0x562f09997500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3c7fff640 1 -- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b12f0 msgr2=0x562f0998b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b12f0 0x562f0998b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3c7fff640 1 -- 172.21.15.31:0/415772436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f09645790 con 0x562f095b94d0 2024-09-21T06:24:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:24:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f0998b720 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800aeb0 tx=0x7fb3b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471b8d77275c18ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.846+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/415772436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8015070 con 0x562f095b94d0 2024-09-21T06:24:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.847+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/415772436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fb3b80115f0 con 0x562f095b94d0 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.847+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c8015070 con 0x562f095b94d0 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.847+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/415772436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb3b801f050 con 0x562f095b94d0 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.847+0000 7fb3dbe58740 10 client.18096 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.847+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562f096be620 con 0x562f095b94d0 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/415772436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c8015070 con 0x562f095b94d0 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/415772436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb3b801aa10 con 0x562f095b94d0 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3dbe58740 10 client.18096 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3dbe58740 10 client.18096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3dbe58740 20 client.18096 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f09598d80 0x562f0999d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3dbe58740 4 client.18096 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.848+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f098f6640 con 0x562f09598d80 2024-09-21T06:24:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.849+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f09598d80 0x562f0999d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.849+0000 7fb3b5ffb640 10 client.18096 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.849+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f09598d80 0x562f0999d4e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b4169e59a1fb2e server_cookie=7697f1f708bf184a in_seq=0 out_seq=0 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.850+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/415772436 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f098f6640 con 0x562f09598d80 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.850+0000 7fb3b5ffb640 10 client.18096 handle_command_reply: tid=0 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.850+0000 7fb3dbe58740 1 client.18096 shutdown 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 msgr2=0x562f0998b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f0998b720 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800aeb0 tx=0x7fb3b800c520 comp rx=0 tx=0).stop 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 shutdown_connections 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f09598d80 0x562f0999d4e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3b5ffb640 0 client.18096 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b94d0 0x562f0998b720 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/415772436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b12f0 0x562f0998b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 >> 172.21.15.31:0/415772436 conn(0x562f095ce300 msgr2=0x562f099809e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.851+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 shutdown_connections 2024-09-21T06:24:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.852+0000 7fb3dbe58740 1 -- 172.21.15.31:0/415772436 wait complete. 2024-09-21T06:24:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.852+0000 7fb3dbe58740 20 client.18096 trim_cache size 0 max 16384 2024-09-21T06:24:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 Processor -- start 2024-09-21T06:24:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 -- start start 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 0x562f0998cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b000c8e0 con 0x562f095b12f0 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b000f140 con 0x562f095b94d0 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3dbe58740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b000cf30 con 0x562f095da1b0 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3ced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 0x562f0998cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 0x562f0998cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53760/0 (socket says 172.21.15.31:53760) 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 -- 172.21.15.31:0/1900779269 learned_addr learned my addr 172.21.15.31:0/1900779269 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.882 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 -- 172.21.15.31:0/1900779269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 msgr2=0x562f09997400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 -- 172.21.15.31:0/1900779269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 msgr2=0x562f0998b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.879+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.880+0000 7fb3c7fff640 1 -- 172.21.15.31:0/1900779269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f09979080 con 0x562f095b94d0 2024-09-21T06:24:37.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.880+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 0x562f0998cdd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8006b20 tx=0x7fb3b80046a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0676648473e2121 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.880+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1900779269 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b802b020 con 0x562f095b94d0 2024-09-21T06:24:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.880+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1900779269 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b801aaf0 con 0x562f095b94d0 2024-09-21T06:24:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.880+0000 7fb3b7fff640 1 -- 172.21.15.31:0/1900779269 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8015040 con 0x562f095b94d0 2024-09-21T06:24:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.881+0000 7fb3dbe58740 1 -- 172.21.15.31:0/1900779269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 msgr2=0x562f0998cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.881+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 0x562f0998cdd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8006b20 tx=0x7fb3b80046a0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 -- 172.21.15.31:0/1900779269 shutdown_connections 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095da1b0 0x562f09997400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095b94d0 0x562f0998cdd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/1900779269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 -- 172.21.15.31:0/1900779269 >> 172.21.15.31:0/1900779269 conn(0x562f095ce300 msgr2=0x562f0998c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 -- 172.21.15.31:0/1900779269 shutdown_connections 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 -- 172.21.15.31:0/1900779269 wait complete. 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 -- start start 2024-09-21T06:24:37.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.882+0000 7fb3dbe58740 1 Processor -- start 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b94d0 0x562f0998eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44314/0 (socket says 172.21.15.31:44314) 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f09a05e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 -- 172.21.15.31:0/469825209 learned_addr learned my addr 172.21.15.31:0/469825209 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b000d600 con 0x562f095b12f0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b000ea70 con 0x562f095da1b0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b000dcd0 con 0x562f095b94d0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b94d0 0x562f0998eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f09a05e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3dbe58740 10 client.0 _ll_register_callbacks cb 0x7fb3b8006e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 -- 172.21.15.31:0/469825209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b94d0 msgr2=0x562f0998eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b94d0 0x562f0998eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 -- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 msgr2=0x562f09a05e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.883+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f09a05e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.884+0000 7fb3ced9d640 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f09a063d0 con 0x562f095b12f0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.884+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b801aaf0 con 0x562f095b12f0 2024-09-21T06:24:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.884+0000 7fb3ced9d640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998e860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80045c0 tx=0x7fb3c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f688c579c09a5249 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.884+0000 7fb3b5ffb640 10 client.18102 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.884+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/469825209 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8011440 con 0x562f095b12f0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/469825209 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fb3c80115e0 con 0x562f095b12f0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/469825209 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb3c8011840 con 0x562f095b12f0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 10 client.18102 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({fsmap=21}) -- 0x562f09a07350 con 0x562f095b12f0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/469825209 <== mon.0 v2:172.21.15.31:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb3c801a470 con 0x562f095b12f0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 10 client.18102 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 10 client.18102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 20 client.18102 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f0956e1c0 0x562f09a097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 4 client.18102 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.885+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f098f20f0 con 0x562f0956e1c0 2024-09-21T06:24:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.886+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f0956e1c0 0x562f09a097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.886+0000 7fb3b5ffb640 10 client.18102 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.886+0000 7fb3c7fff640 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f0956e1c0 0x562f09a097e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed98e535eb891fe server_cookie=56615a4cc97403 in_seq=0 out_seq=0 2024-09-21T06:24:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3b5ffb640 1 -- 172.21.15.31:0/469825209 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f098f20f0 con 0x562f0956e1c0 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3b5ffb640 10 client.18102 handle_command_reply: tid=0 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 client.18102 shutdown 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 msgr2=0x562f0998e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998e860 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80045c0 tx=0x7fb3c8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 shutdown_connections 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x562f0956e1c0 0x562f09a097e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f095da1b0 0x562f09a05e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562f095b94d0 0x562f0998eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3b5ffb640 0 client.18102 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 --2- 172.21.15.31:0/469825209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562f095b12f0 0x562f0998e860 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 >> 172.21.15.31:0/469825209 conn(0x562f095ce300 msgr2=0x562f09985200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.914+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 shutdown_connections 2024-09-21T06:24:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3dbe58740 1 -- 172.21.15.31:0/469825209 wait complete. 2024-09-21T06:24:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3dbe58740 20 client.18102 trim_cache size 0 max 16384 2024-09-21T06:24:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:24:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:24:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:24:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb3bc0b6880 msgr2=0x7fb3bc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb3bc0b6880 0x7fb3bc0b8d90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x562f0997e630 tx=0x7fb3c0039ef0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 msgr2=0x7fb3d0117380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d0117380 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000f80 tx=0x7fb3c002f480 comp rx=0 tx=0).stop 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 shutdown_connections 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3d01103a0 0x7fb3d0117e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3d010ad10 0x7fb3d01178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb3bc0b6880 0x7fb3bc0b8d90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 --2- 172.21.15.31:0/3021671372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3d010a3c0 0x7fb3d0117380 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 >> 172.21.15.31:0/3021671372 conn(0x7fb3d0071220 msgr2=0x7fb3d010cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 shutdown_connections 2024-09-21T06:24:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:37.915+0000 7fb3d4eec640 1 -- 172.21.15.31:0/3021671372 wait complete. 2024-09-21T06:24:39.289 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:39 smithi107 ceph-mon[34380]: pgmap v726: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-09-21T06:24:39.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:39 smithi031 ceph-mon[31237]: pgmap v726: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-09-21T06:24:39.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:39 smithi110 ceph-mon[34550]: pgmap v726: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 16 MiB/s wr, 1.60k op/s 2024-09-21T06:24:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 780.0 s, 2382.6 tps, lat 42.037 ms stddev 49.640 2024-09-21T06:24:41.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:41 smithi031 ceph-mon[31237]: pgmap v727: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 17 MiB/s wr, 1.69k op/s 2024-09-21T06:24:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:41 smithi110 ceph-mon[34550]: pgmap v727: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 17 MiB/s wr, 1.69k op/s 2024-09-21T06:24:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:41 smithi107 ceph-mon[34380]: pgmap v727: 97 pgs: 97 active+clean; 4.3 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 17 MiB/s wr, 1.69k op/s 2024-09-21T06:24:43.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:43 smithi031 ceph-mon[31237]: pgmap v728: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-09-21T06:24:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:43 smithi110 ceph-mon[34550]: pgmap v728: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-09-21T06:24:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:43 smithi107 ceph-mon[34380]: pgmap v728: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 17 MiB/s wr, 1.67k op/s 2024-09-21T06:24:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 785.0 s, 1840.0 tps, lat 54.552 ms stddev 67.492 2024-09-21T06:24:45.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:45 smithi031 ceph-mon[31237]: pgmap v729: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 17 MiB/s wr, 1.66k op/s 2024-09-21T06:24:45.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:45 smithi110 ceph-mon[34550]: pgmap v729: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 17 MiB/s wr, 1.66k op/s 2024-09-21T06:24:45.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:45 smithi107 ceph-mon[34380]: pgmap v729: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 17 MiB/s wr, 1.66k op/s 2024-09-21T06:24:47.330 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:47 smithi110 ceph-mon[34550]: pgmap v730: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 1.19k op/s 2024-09-21T06:24:47.341 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:47 smithi107 ceph-mon[34380]: pgmap v730: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 1.19k op/s 2024-09-21T06:24:47.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:47 smithi031 ceph-mon[31237]: pgmap v730: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 12 MiB/s wr, 1.19k op/s 2024-09-21T06:24:49.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:49 smithi031 ceph-mon[31237]: pgmap v731: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 710 op/s 2024-09-21T06:24:49.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:49 smithi110 ceph-mon[34550]: pgmap v731: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 710 op/s 2024-09-21T06:24:49.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:49 smithi107 ceph-mon[34380]: pgmap v731: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.1 MiB/s wr, 710 op/s 2024-09-21T06:24:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 790.0 s, 1614.6 tps, lat 61.558 ms stddev 65.572 2024-09-21T06:24:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3188453931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf58110350 msgr2=0x7fdf58110b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3188453931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf58110350 0x7fdf58110b70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009900 tx=0x7fdf4802a440 comp rx=0 tx=0).stop 2024-09-21T06:24:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3188453931 shutdown_connections 2024-09-21T06:24:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3188453931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf58110350 0x7fdf58110b70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3188453931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5810fa00 0x7fdf5810fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3188453931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf5810a660 0x7fdf5810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.057+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3188453931 >> 172.21.15.31:0/3188453931 conn(0x7fdf58070dd0 msgr2=0x7fdf580711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.063+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3188453931 shutdown_connections 2024-09-21T06:24:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.065+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3188453931 wait complete. 2024-09-21T06:24:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.066+0000 7fdf5e8a9640 1 Processor -- start 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 -- start start 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5810a660 0x7fdf581201d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5810fa00 0x7fdf581193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 0x7fdf58119d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58122940 con 0x7fdf58119910 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581227c0 con 0x7fdf5810fa00 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.067+0000 7fdf5e8a9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58122ac0 con 0x7fdf5810a660 2024-09-21T06:24:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.069+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5810a660 0x7fdf581201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.069+0000 7fdf5ce46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 0x7fdf58119d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.069+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5810a660 0x7fdf581201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:58076/0 (socket says 172.21.15.31:58076) 2024-09-21T06:24:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.069+0000 7fdf5ce46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 0x7fdf58119d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60188/0 (socket says 172.21.15.31:60188) 2024-09-21T06:24:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.069+0000 7fdf57fff640 1 -- 172.21.15.31:0/3897369201 learned_addr learned my addr 172.21.15.31:0/3897369201 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.071+0000 7fdf577fe640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5810fa00 0x7fdf581193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.071+0000 7fdf5ce46640 1 -- 172.21.15.31:0/3897369201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5810a660 msgr2=0x7fdf581201d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.071+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5810a660 0x7fdf581201d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.071+0000 7fdf5ce46640 1 -- 172.21.15.31:0/3897369201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5810fa00 msgr2=0x7fdf581193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.071+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5810fa00 0x7fdf581193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.071+0000 7fdf5ce46640 1 -- 172.21.15.31:0/3897369201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5811ed70 con 0x7fdf58119910 2024-09-21T06:24:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.075+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 0x7fdf58119d20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdf48034040 tx=0x7fdf480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.084+0000 7fdf557fa640 1 -- 172.21.15.31:0/3897369201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48040070 con 0x7fdf58119910 2024-09-21T06:24:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.084+0000 7fdf557fa640 1 -- 172.21.15.31:0/3897369201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf4803a3f0 con 0x7fdf58119910 2024-09-21T06:24:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.084+0000 7fdf557fa640 1 -- 172.21.15.31:0/3897369201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48035040 con 0x7fdf58119910 2024-09-21T06:24:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.085+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5811f000 con 0x7fdf58119910 2024-09-21T06:24:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.085+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5811f4a0 con 0x7fdf58119910 2024-09-21T06:24:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.089+0000 7fdf557fa640 1 -- 172.21.15.31:0/3897369201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdf48043070 con 0x7fdf58119910 2024-09-21T06:24:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf557fa640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf400b6730 0x7fdf400b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 Processor -- start 2024-09-21T06:24:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 -- start start 2024-09-21T06:24:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf557fa640 1 -- 172.21.15.31:0/3897369201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fdf48108950 con 0x7fdf58119910 2024-09-21T06:24:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b3cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b3d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556a34ab16f0 con 0x556a347971b0 2024-09-21T06:24:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556a34aa0b20 con 0x556a347764d0 2024-09-21T06:24:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf6581a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556a34aa4930 con 0x556a3476e2f0 2024-09-21T06:24:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.091+0000 7fdf57fff640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf400b6730 0x7fdf400b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.092+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.092+0000 7fdf5ce46640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b3d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.092+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b3c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60198/0 (socket says 172.21.15.31:60198) 2024-09-21T06:24:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.092+0000 7fdf57fff640 1 -- 172.21.15.31:0/1497552524 learned_addr learned my addr 172.21.15.31:0/1497552524 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf577fe640 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b3cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf57fff640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf400b6730 0x7fdf400b8c40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdf500116c0 tx=0x7fdf50011ac0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf5ce46640 1 -- 172.21.15.31:0/1497552524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 msgr2=0x556a34b3cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b3cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf5ce46640 1 -- 172.21.15.31:0/1497552524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 msgr2=0x556a34b3c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b3c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf5ce46640 1 -- 172.21.15.31:0/1497552524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a34a66530 con 0x556a347764d0 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf57fff640 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b3d430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdf48106020 tx=0x7fdf48048000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2e8897735bc32d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf3e7fc640 1 -- 172.21.15.31:0/1497552524 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4804a070 con 0x556a347764d0 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.093+0000 7fdf3e7fc640 1 -- 172.21.15.31:0/1497552524 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480c8e50 con 0x556a347764d0 2024-09-21T06:24:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.094+0000 7fdf3e7fc640 1 -- 172.21.15.31:0/1497552524 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf480d3230 con 0x556a347764d0 2024-09-21T06:24:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.094+0000 7fdf6581a740 1 -- 172.21.15.31:0/1497552524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 msgr2=0x556a34b3d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.094+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b3d430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdf48106020 tx=0x7fdf48048000 comp rx=0 tx=0).stop 2024-09-21T06:24:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 -- 172.21.15.31:0/1497552524 shutdown_connections 2024-09-21T06:24:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b3d430 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b3cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.098 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1497552524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 -- 172.21.15.31:0/1497552524 >> 172.21.15.31:0/1497552524 conn(0x556a3478b300 msgr2=0x556a34b3a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 -- 172.21.15.31:0/1497552524 shutdown_connections 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 -- 172.21.15.31:0/1497552524 wait complete. 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 -- start start 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.095+0000 7fdf6581a740 1 Processor -- start 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a3476e2f0 0x556a34b48110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b48650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347971b0 0x556a34b54150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x556a349d8ab0 con 0x556a3476e2f0 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556a34a46730 con 0x556a347764d0 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556a34b4aef0 con 0x556a347971b0 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf6581a740 10 client.0 _ll_register_callbacks cb 0x556a34a9a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:51.099 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b48650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a3476e2f0 0x556a34b48110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b48650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56988/0 (socket says 172.21.15.31:56988) 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf577fe640 1 -- 172.21.15.31:0/1766371279 learned_addr learned my addr 172.21.15.31:0/1766371279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.096+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347971b0 0x556a34b54150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf57fff640 1 -- 172.21.15.31:0/1766371279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347971b0 msgr2=0x556a34b54150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf57fff640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347971b0 0x556a34b54150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf57fff640 1 -- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 msgr2=0x556a34b48650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf57fff640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b48650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf57fff640 1 -- 172.21.15.31:0/1766371279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a34802790 con 0x556a3476e2f0 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf577fe640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b48650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf57fff640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a3476e2f0 0x556a34b48110 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e30 tx=0x7fdf50007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5755f9b841cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf377fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf377fe640 1 -- 172.21.15.31:0/1766371279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50025070 con 0x556a3476e2f0 2024-09-21T06:24:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf377fe640 1 -- 172.21.15.31:0/1766371279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fdf50021660 con 0x556a3476e2f0 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf480c8e50 con 0x556a3476e2f0 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf377fe640 1 -- 172.21.15.31:0/1766371279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdf50031050 con 0x556a3476e2f0 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf6581a740 10 client.18111 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.097+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556a3487b620 con 0x556a3476e2f0 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf377fe640 1 -- 172.21.15.31:0/1766371279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480c8e50 con 0x556a3476e2f0 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf377fe640 1 -- 172.21.15.31:0/1766371279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdf500207e0 con 0x556a3476e2f0 2024-09-21T06:24:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf6581a740 10 client.18111 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf6581a740 10 client.18111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf6581a740 20 client.18111 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a34755d80 0x556a34b59ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf6581a740 4 client.18111 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:24:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.100+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a34ab3640 con 0x556a34755d80 2024-09-21T06:24:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.101+0000 7fdf577fe640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a34755d80 0x556a34b59ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.101+0000 7fdf377fe640 10 client.18111 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.101+0000 7fdf577fe640 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a34755d80 0x556a34b59ed0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6167bdeabe2d9171 server_cookie=da9418a77770479d in_seq=0 out_seq=0 2024-09-21T06:24:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.102+0000 7fdf377fe640 1 -- 172.21.15.31:0/1766371279 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a34ab3640 con 0x556a34755d80 2024-09-21T06:24:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.102+0000 7fdf377fe640 10 client.18111 handle_command_reply: tid=0 2024-09-21T06:24:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.102+0000 7fdf6581a740 1 client.18111 shutdown 2024-09-21T06:24:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a3476e2f0 msgr2=0x556a34b48110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a3476e2f0 0x556a34b48110 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e30 tx=0x7fdf50007040 comp rx=0 tx=0).stop 2024-09-21T06:24:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 shutdown_connections 2024-09-21T06:24:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a34755d80 0x556a34b59ed0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347971b0 0x556a34b54150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b48650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 --2- 172.21.15.31:0/1766371279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a3476e2f0 0x556a34b48110 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 >> 172.21.15.31:0/1766371279 conn(0x556a3478b300 msgr2=0x556a34b3d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf377fe640 0 client.18111 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 shutdown_connections 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 1 -- 172.21.15.31:0/1766371279 wait complete. 2024-09-21T06:24:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.103+0000 7fdf6581a740 20 client.18111 trim_cache size 0 max 16384 2024-09-21T06:24:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.123+0000 7fdf6581a740 1 Processor -- start 2024-09-21T06:24:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.123+0000 7fdf6581a740 1 -- start start 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b49d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b54010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b39eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf6581a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581c4f00 con 0x556a347971b0 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf6581a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581c7760 con 0x556a347764d0 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf6581a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581c5550 con 0x556a3476e2f0 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b54010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b54010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:56996/0 (socket says 172.21.15.31:56996) 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf577fe640 1 -- 172.21.15.31:0/157173017 learned_addr learned my addr 172.21.15.31:0/157173017 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf57fff640 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b49d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf5ce46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b39eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf57fff640 1 -- 172.21.15.31:0/157173017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 msgr2=0x556a34b54010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf57fff640 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b54010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf57fff640 1 -- 172.21.15.31:0/157173017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 msgr2=0x556a34b39eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf57fff640 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b39eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.124+0000 7fdf57fff640 1 -- 172.21.15.31:0/157173017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a34b465f0 con 0x556a3476e2f0 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.125+0000 7fdf57fff640 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b49d40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf500070b0 tx=0x7fdf50004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72fee4c1acf22c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.125+0000 7fdf3dffb640 1 -- 172.21.15.31:0/157173017 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50025040 con 0x556a3476e2f0 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.125+0000 7fdf3dffb640 1 -- 172.21.15.31:0/157173017 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf50021660 con 0x556a3476e2f0 2024-09-21T06:24:51.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.125+0000 7fdf3dffb640 1 -- 172.21.15.31:0/157173017 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5002a6a0 con 0x556a3476e2f0 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 -- 172.21.15.31:0/157173017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 msgr2=0x556a34b49d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b49d40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdf500070b0 tx=0x7fdf50004b50 comp rx=0 tx=0).stop 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 -- 172.21.15.31:0/157173017 shutdown_connections 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34b39eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a347764d0 0x556a34b54010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 --2- 172.21.15.31:0/157173017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a3476e2f0 0x556a34b49d40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 -- 172.21.15.31:0/157173017 >> 172.21.15.31:0/157173017 conn(0x556a3478b300 msgr2=0x556a34b49630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 -- 172.21.15.31:0/157173017 shutdown_connections 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 -- 172.21.15.31:0/157173017 wait complete. 2024-09-21T06:24:51.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.126+0000 7fdf6581a740 1 -- start start 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 Processor -- start 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 0x556a34b4b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347764d0 0x556a34b4bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34bc28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581c5c20 con 0x556a347971b0 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581c7090 con 0x556a3476e2f0 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581c62f0 con 0x556a347764d0 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf6581a740 10 client.0 _ll_register_callbacks cb 0x7fdf50014bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:24:51.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 0x556a34b4b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf5ce46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34bc28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf57fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 0x556a34b4b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:57012/0 (socket says 172.21.15.31:57012) 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf57fff640 1 -- 172.21.15.31:0/3424371445 learned_addr learned my addr 172.21.15.31:0/3424371445 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf577fe640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347764d0 0x556a34b4bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.127+0000 7fdf5ce46640 1 -- 172.21.15.31:0/3424371445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347764d0 msgr2=0x556a34b4bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347764d0 0x556a34b4bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf5ce46640 1 -- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 msgr2=0x556a34b4b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 0x556a34b4b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf5ce46640 1 -- 172.21.15.31:0/3424371445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a34bc2de0 con 0x556a347971b0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf57fff640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 0x556a34b4b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf5ce46640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34bc28a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdf480cad20 tx=0x7fdf4803a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c912034a56e73020 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf377fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf377fe640 1 -- 172.21.15.31:0/3424371445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf480f4070 con 0x556a347971b0 2024-09-21T06:24:51.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf377fe640 1 -- 172.21.15.31:0/3424371445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fdf4804dcc0 con 0x556a347971b0 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf50021660 con 0x556a347971b0 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf377fe640 1 -- 172.21.15.31:0/3424371445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdf4804a040 con 0x556a347971b0 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf6581a740 10 client.18117 fetch_fsmap learned FSMap version 21 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.128+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556a34bc3fd0 con 0x556a347971b0 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.129+0000 7fdf377fe640 1 -- 172.21.15.31:0/3424371445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf50021660 con 0x556a347971b0 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.129+0000 7fdf377fe640 1 -- 172.21.15.31:0/3424371445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdf481062d0 con 0x556a347971b0 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.130+0000 7fdf6581a740 10 client.18117 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.130+0000 7fdf6581a740 10 client.18117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.130+0000 7fdf6581a740 20 client.18117 populate_metadata read hostname 'smithi031' 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.130+0000 7fdf6581a740 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a3472b1c0 0x556a34bc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.130+0000 7fdf6581a740 4 client.18117 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:24:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.130+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a34aaf0f0 con 0x556a3472b1c0 2024-09-21T06:24:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.132+0000 7fdf57fff640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a3472b1c0 0x556a34bc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:24:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.133+0000 7fdf377fe640 10 client.18117 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.133+0000 7fdf57fff640 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a3472b1c0 0x556a34bc6580 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b635be02abc73df6 server_cookie=4095099def01a8de in_seq=0 out_seq=0 2024-09-21T06:24:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.164+0000 7fdf377fe640 1 -- 172.21.15.31:0/3424371445 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a34aaf0f0 con 0x556a3472b1c0 2024-09-21T06:24:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.164+0000 7fdf377fe640 10 client.18117 handle_command_reply: tid=0 2024-09-21T06:24:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.164+0000 7fdf6581a740 1 client.18117 shutdown 2024-09-21T06:24:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 msgr2=0x556a34bc28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34bc28a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdf480cad20 tx=0x7fdf4803a7b0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 shutdown_connections 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x556a3472b1c0 0x556a34bc6580 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556a347971b0 0x556a34bc28a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556a347764d0 0x556a34b4bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf377fe640 0 client.18117 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 --2- 172.21.15.31:0/3424371445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556a3476e2f0 0x556a34b4b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 >> 172.21.15.31:0/3424371445 conn(0x556a3478b300 msgr2=0x556a34b495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 shutdown_connections 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 1 -- 172.21.15.31:0/3424371445 wait complete. 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.165+0000 7fdf6581a740 20 client.18117 trim_cache size 0 max 16384 2024-09-21T06:24:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:24:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:24:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:24:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:24:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf400b6730 msgr2=0x7fdf400b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf400b6730 0x7fdf400b8c40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdf500116c0 tx=0x7fdf50011ac0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 msgr2=0x7fdf58119d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 0x7fdf58119d20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdf48034040 tx=0x7fdf480045e0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 shutdown_connections 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdf400b6730 0x7fdf400b8c40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf58119910 0x7fdf58119d20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf5810fa00 0x7fdf581193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 --2- 172.21.15.31:0/3897369201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf5810a660 0x7fdf581201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 >> 172.21.15.31:0/3897369201 conn(0x7fdf58070dd0 msgr2=0x7fdf5810df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 shutdown_connections 2024-09-21T06:24:51.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:24:51.166+0000 7fdf5e8a9640 1 -- 172.21.15.31:0/3897369201 wait complete. 2024-09-21T06:24:51.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:51 smithi031 ceph-mon[31237]: pgmap v732: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-09-21T06:24:51.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:51 smithi110 ceph-mon[34550]: pgmap v732: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-09-21T06:24:51.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:51 smithi107 ceph-mon[34380]: pgmap v732: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-09-21T06:24:53.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:53 smithi031 ceph-mon[31237]: pgmap v733: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.1 MiB/s wr, 627 op/s 2024-09-21T06:24:53.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:53 smithi110 ceph-mon[34550]: pgmap v733: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.1 MiB/s wr, 627 op/s 2024-09-21T06:24:53.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:53 smithi107 ceph-mon[34380]: pgmap v733: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.1 MiB/s wr, 627 op/s 2024-09-21T06:24:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 795.0 s, 1661.6 tps, lat 60.646 ms stddev 68.647 2024-09-21T06:24:55.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:55 smithi031 ceph-mon[31237]: pgmap v734: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-09-21T06:24:55.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:55 smithi110 ceph-mon[34550]: pgmap v734: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-09-21T06:24:55.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:55 smithi107 ceph-mon[34380]: pgmap v734: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.2 MiB/s wr, 653 op/s 2024-09-21T06:24:57.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:57 smithi031 ceph-mon[31237]: pgmap v735: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-09-21T06:24:57.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:24:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:57 smithi110 ceph-mon[34550]: pgmap v735: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-09-21T06:24:57.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:24:57.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:57 smithi107 ceph-mon[34380]: pgmap v735: 97 pgs: 97 active+clean; 4.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-09-21T06:24:57.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:24:58.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:24:58.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:24:58.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:24:58.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:24:58.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:24:58.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:24:58.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:24:58.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:24:58.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:24:59.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:24:59 smithi110 ceph-mon[34550]: pgmap v736: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 631 op/s 2024-09-21T06:24:59.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:24:59 smithi107 ceph-mon[34380]: pgmap v736: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 631 op/s 2024-09-21T06:24:59.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:24:59 smithi031 ceph-mon[31237]: pgmap v736: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.9 MiB/s wr, 631 op/s 2024-09-21T06:25:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 800.0 s, 2143.4 tps, lat 46.685 ms stddev 56.980 2024-09-21T06:25:01.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:01 smithi110 ceph-mon[34550]: pgmap v737: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.9 MiB/s wr, 724 op/s 2024-09-21T06:25:01.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:01 smithi107 ceph-mon[34380]: pgmap v737: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.9 MiB/s wr, 724 op/s 2024-09-21T06:25:01.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:01 smithi031 ceph-mon[31237]: pgmap v737: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.9 MiB/s wr, 724 op/s 2024-09-21T06:25:02.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:25:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:03.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 -- 172.21.15.31:0/1766705921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648105db0 msgr2=0x7f66481061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 --2- 172.21.15.31:0/1766705921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648105db0 0x7f66481061c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f663802a410 comp rx=0 tx=0).stop 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 -- 172.21.15.31:0/1766705921 shutdown_connections 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 --2- 172.21.15.31:0/1766705921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f664810c040 0x7f6648112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 --2- 172.21.15.31:0/1766705921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f664810b6f0 0x7f664810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 --2- 172.21.15.31:0/1766705921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648105db0 0x7f66481061c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 -- 172.21.15.31:0/1766705921 >> 172.21.15.31:0/1766705921 conn(0x7f66481017c0 msgr2=0x7f6648103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 -- 172.21.15.31:0/1766705921 shutdown_connections 2024-09-21T06:25:03.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.295+0000 7f664c840640 1 -- 172.21.15.31:0/1766705921 wait complete. 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.296+0000 7f664c840640 1 Processor -- start 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.296+0000 7f664c840640 1 -- start start 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664c840640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648105db0 0x7f664819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664c840640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 0x7f664819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664c840640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f664810c040 0x7f66481a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664c840640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6648115d00 con 0x7f664810b6f0 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664c840640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6648115b80 con 0x7f6648105db0 2024-09-21T06:25:03.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664c840640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6648115e80 con 0x7f664810c040 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f6645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 0x7f664819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648105db0 0x7f664819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f6645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 0x7f664819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37352/0 (socket says 172.21.15.31:37352) 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f6645d9b640 1 -- 172.21.15.31:0/750372404 learned_addr learned my addr 172.21.15.31:0/750372404 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f664810c040 0x7f66481a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664659c640 1 -- 172.21.15.31:0/750372404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f664810c040 msgr2=0x7f66481a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664659c640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f664810c040 0x7f66481a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664659c640 1 -- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 msgr2=0x7f664819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664659c640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 0x7f664819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f664659c640 1 -- 172.21.15.31:0/750372404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66481a26c0 con 0x7f6648105db0 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.297+0000 7f6645d9b640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 0x7f664819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.298+0000 7f664659c640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648105db0 0x7f664819c750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f663802a930 tx=0x7f6638031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.298+0000 7f66377fe640 1 -- 172.21.15.31:0/750372404 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663803f030 con 0x7f6648105db0 2024-09-21T06:25:03.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.298+0000 7f66377fe640 1 -- 172.21.15.31:0/750372404 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6638004490 con 0x7f6648105db0 2024-09-21T06:25:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.298+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66481a2950 con 0x7f6648105db0 2024-09-21T06:25:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.298+0000 7f66377fe640 1 -- 172.21.15.31:0/750372404 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638035070 con 0x7f6648105db0 2024-09-21T06:25:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.298+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66481a9110 con 0x7f6648105db0 2024-09-21T06:25:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.300+0000 7f6652db4740 1 Processor -- start 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 -- start start 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f66377fe640 1 -- 172.21.15.31:0/750372404 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6638042030 con 0x7f6648105db0 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc8e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bbc85ae50 con 0x559bbc51c390 2024-09-21T06:25:03.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559bbc84a280 con 0x559bbc5141b0 2024-09-21T06:25:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6652db4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559bbc84e090 con 0x559bbc53d070 2024-09-21T06:25:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f664659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6645d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc8e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8e4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60024/0 (socket says 172.21.15.31:60024) 2024-09-21T06:25:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.301+0000 7f6646d9d640 1 -- 172.21.15.31:0/3682191081 learned_addr learned my addr 172.21.15.31:0/3682191081 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6646d9d640 1 -- 172.21.15.31:0/3682191081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 msgr2=0x559bbc8e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6646d9d640 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8e4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6646d9d640 1 -- 172.21.15.31:0/3682191081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 msgr2=0x559bbc8e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6646d9d640 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc8e50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6646d9d640 1 -- 172.21.15.31:0/3682191081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbc5a8650 con 0x559bbc53d070 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6645d9b640 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc8e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6646d9d640 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8e4650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6640009f50 tx=0x7f6640011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd50cfc853b4654 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f66377fe640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66300b6680 0x7f66300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6634ff9640 1 -- 172.21.15.31:0/3682191081 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640004d20 con 0x559bbc53d070 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6634ff9640 1 -- 172.21.15.31:0/3682191081 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6640004050 con 0x559bbc53d070 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f6634ff9640 1 -- 172.21.15.31:0/3682191081 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640005260 con 0x559bbc53d070 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f664659c640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66300b6680 0x7f66300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.302+0000 7f66377fe640 1 -- 172.21.15.31:0/750372404 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f6638107980 con 0x7f6648105db0 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 -- 172.21.15.31:0/3682191081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 msgr2=0x559bbc8e4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f664659c640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66300b6680 0x7f66300b8b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6638105190 tx=0x7f663810d530 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8e4650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6640009f50 tx=0x7f6640011810 comp rx=0 tx=0).stop 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 -- 172.21.15.31:0/3682191081 shutdown_connections 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc8e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 --2- 172.21.15.31:0/3682191081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8e4650 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 -- 172.21.15.31:0/3682191081 >> 172.21.15.31:0/3682191081 conn(0x559bbc531460 msgr2=0x559bbc8e2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 -- 172.21.15.31:0/3682191081 shutdown_connections 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.303+0000 7f6652db4740 1 -- 172.21.15.31:0/3682191081 wait complete. 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 -- start start 2024-09-21T06:25:03.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 Processor -- start 2024-09-21T06:25:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 0x559bbc8efc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8f0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559bbc6ab700 con 0x559bbc5141b0 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559bbc7e5110 con 0x559bbc51c390 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559bbc8f2e30 con 0x559bbc53d070 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 0x559bbc8efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 0x559bbc8efc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37372/0 (socket says 172.21.15.31:37372) 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6646d9d640 1 -- 172.21.15.31:0/4028393131 learned_addr learned my addr 172.21.15.31:0/4028393131 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.304+0000 7f6652db4740 10 client.0 _ll_register_callbacks cb 0x559bbc7fe990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6645d9b640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 -- 172.21.15.31:0/4028393131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 msgr2=0x559bbc8f0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8f0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 -- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 msgr2=0x559bbc8f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 -- 172.21.15.31:0/4028393131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbc812c40 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6646d9d640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 0x559bbc8efc70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x559bbc8f0c50 tx=0x7f66400096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d461e78b7f1ced server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f662dffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f662dffb640 1 -- 172.21.15.31:0/4028393131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664002c030 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f662dffb640 1 -- 172.21.15.31:0/4028393131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f6640025670 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.305+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6640004050 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f662dffb640 1 -- 172.21.15.31:0/4028393131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6640036050 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 10 client.18129 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559bbc8feae0 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f662dffb640 1 -- 172.21.15.31:0/4028393131 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6640004050 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f662dffb640 1 -- 172.21.15.31:0/4028393131 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f66400287c0 con 0x559bbc5141b0 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 10 client.18129 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 20 client.18129 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:03.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc4fb9f0 0x559bbc901fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 4 client.18129 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bbc85cda0 con 0x559bbc4fb9f0 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.306+0000 7f6645d9b640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc4fb9f0 0x559bbc901fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.307+0000 7f662dffb640 10 client.18129 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.307+0000 7f6645d9b640 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc4fb9f0 0x559bbc901fb0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3738d5770baa5557 server_cookie=829d45b0bc21c80a in_seq=0 out_seq=0 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.307+0000 7f662dffb640 1 -- 172.21.15.31:0/4028393131 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559bbc85cda0 con 0x559bbc4fb9f0 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f662dffb640 10 client.18129 handle_command_reply: tid=0 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 client.18129 shutdown 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 msgr2=0x559bbc8efc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 0x559bbc8efc70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x559bbc8f0c50 tx=0x7f66400096c0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 shutdown_connections 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc4fb9f0 0x559bbc901fb0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f662dffb640 0 client.18129 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc53d070 0x559bbc8f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 --2- 172.21.15.31:0/4028393131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc5141b0 0x559bbc8efc70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 >> 172.21.15.31:0/4028393131 conn(0x559bbc531460 msgr2=0x559bbc8e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 shutdown_connections 2024-09-21T06:25:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 1 -- 172.21.15.31:0/4028393131 wait complete. 2024-09-21T06:25:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.308+0000 7f6652db4740 20 client.18129 trim_cache size 0 max 16384 2024-09-21T06:25:03.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 Processor -- start 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 -- start start 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 0x559bbc8e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66481afa50 con 0x559bbc4fb9f0 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66481b22b0 con 0x559bbc51c390 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6652db4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66481b00a0 con 0x559bbc5141b0 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.326+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 0x559bbc8e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 0x559bbc8e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60042/0 (socket says 172.21.15.31:60042) 2024-09-21T06:25:03.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37388/0 (socket says 172.21.15.31:37388) 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 -- 172.21.15.31:0/3482188005 learned_addr learned my addr 172.21.15.31:0/3482188005 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6646d9d640 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 -- 172.21.15.31:0/3482188005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 msgr2=0x559bbc8e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8e20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 -- 172.21.15.31:0/3482188005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc4fb9f0 msgr2=0x559bbc8f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 -- 172.21.15.31:0/3482188005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbc8fc050 con 0x559bbc5141b0 2024-09-21T06:25:03.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6646d9d640 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f6645d9b640 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 0x559bbc8e1b70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f663c012ab0 tx=0x7f663c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3716597ddc42143a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f662ffff640 1 -- 172.21.15.31:0/3482188005 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c004030 con 0x559bbc5141b0 2024-09-21T06:25:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f662ffff640 1 -- 172.21.15.31:0/3482188005 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c00fe30 con 0x559bbc5141b0 2024-09-21T06:25:03.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.327+0000 7f662ffff640 1 -- 172.21.15.31:0/3482188005 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c005450 con 0x559bbc5141b0 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 -- 172.21.15.31:0/3482188005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 msgr2=0x559bbc8e1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 0x559bbc8e1b70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f663c012ab0 tx=0x7f663c005f90 comp rx=0 tx=0).stop 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 -- 172.21.15.31:0/3482188005 shutdown_connections 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc51c390 0x559bbc8e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc5141b0 0x559bbc8e1b70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 --2- 172.21.15.31:0/3482188005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 -- 172.21.15.31:0/3482188005 >> 172.21.15.31:0/3482188005 conn(0x559bbc531460 msgr2=0x559bbc8efa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 -- 172.21.15.31:0/3482188005 shutdown_connections 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.328+0000 7f6652db4740 1 -- 172.21.15.31:0/3482188005 wait complete. 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 -- start start 2024-09-21T06:25:03.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 Processor -- start 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc96b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66481b0770 con 0x559bbc51c390 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66481b1be0 con 0x559bbc5141b0 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66481b0e40 con 0x559bbc4fb9f0 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6652db4740 10 client.0 _ll_register_callbacks cb 0x7f663810d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6646d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60052/0 (socket says 172.21.15.31:60052) 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6646d9d640 1 -- 172.21.15.31:0/2421556553 learned_addr learned my addr 172.21.15.31:0/2421556553 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6646d9d640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc96b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.329+0000 7f6645d9b640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6646d9d640 1 -- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 msgr2=0x559bbc8f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6646d9d640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6646d9d640 1 -- 172.21.15.31:0/2421556553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 msgr2=0x559bbc96b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6646d9d640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc96b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6646d9d640 1 -- 172.21.15.31:0/2421556553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbc8f5010 con 0x559bbc4fb9f0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6646d9d640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f42e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6640009690 tx=0x7f6640003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678d32d92fd77030 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f662dffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f662dffb640 1 -- 172.21.15.31:0/2421556553 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664002c040 con 0x559bbc4fb9f0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f662dffb640 1 -- 172.21.15.31:0/2421556553 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f6640004510 con 0x559bbc4fb9f0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c00fe30 con 0x559bbc4fb9f0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f662dffb640 1 -- 172.21.15.31:0/2421556553 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f664004d050 con 0x559bbc4fb9f0 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6652db4740 10 client.27955 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:03.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.330+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559bbc96cce0 con 0x559bbc4fb9f0 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f662dffb640 1 -- 172.21.15.31:0/2421556553 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f663c00fe30 con 0x559bbc4fb9f0 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f662dffb640 1 -- 172.21.15.31:0/2421556553 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6640025800 con 0x559bbc4fb9f0 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6652db4740 10 client.27955 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6652db4740 10 client.27955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6652db4740 20 client.27955 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6652db4740 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc53d070 0x559bbc96f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6652db4740 4 client.27955 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bbc858850 con 0x559bbc53d070 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6645d9b640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc53d070 0x559bbc96f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f662dffb640 10 client.27955 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:03.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.331+0000 7f6645d9b640 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc53d070 0x559bbc96f2f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf844e22b0492d0 server_cookie=d882670f5072278e in_seq=0 out_seq=0 2024-09-21T06:25:03.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f662dffb640 1 -- 172.21.15.31:0/2421556553 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8894 (crc 0 0 0) 0x559bbc858850 con 0x559bbc53d070 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f662dffb640 10 client.27955 handle_command_reply: tid=0 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 client.27955 shutdown 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 msgr2=0x559bbc8f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f42e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6640009690 tx=0x7f6640003bf0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 shutdown_connections 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559bbc53d070 0x559bbc96f2f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559bbc51c390 0x559bbc96b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559bbc5141b0 0x559bbc8f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f662dffb640 0 client.27955 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 --2- 172.21.15.31:0/2421556553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559bbc4fb9f0 0x559bbc8f42e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 >> 172.21.15.31:0/2421556553 conn(0x559bbc531460 msgr2=0x559bbc8fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 shutdown_connections 2024-09-21T06:25:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 1 -- 172.21.15.31:0/2421556553 wait complete. 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.332+0000 7f6652db4740 20 client.27955 trim_cache size 0 max 16384 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:25:03.340 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.342 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.343 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.344 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:25:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:25:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:25:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:25:03.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66300b6680 msgr2=0x7f66300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66300b6680 0x7f66300b8b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6638105190 tx=0x7f663810d530 comp rx=0 tx=0).stop 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648105db0 msgr2=0x7f664819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648105db0 0x7f664819c750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f663802a930 tx=0x7f6638031a20 comp rx=0 tx=0).stop 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 shutdown_connections 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f664810c040 0x7f66481a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66300b6680 0x7f66300b8b90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f664810b6f0 0x7f664819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 --2- 172.21.15.31:0/750372404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6648105db0 0x7f664819c750 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 >> 172.21.15.31:0/750372404 conn(0x7f66481017c0 msgr2=0x7f6648109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 shutdown_connections 2024-09-21T06:25:03.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:03.333+0000 7f664c840640 1 -- 172.21.15.31:0/750372404 wait complete. 2024-09-21T06:25:03.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:03 smithi110 ceph-mon[34550]: pgmap v738: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-21T06:25:03.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:03 smithi107 ceph-mon[34380]: pgmap v738: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-21T06:25:03.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:03 smithi031 ceph-mon[31237]: pgmap v738: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.0 MiB/s wr, 656 op/s 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.280+0000 7fe25532b640 1 -- 172.21.15.31:0/1691261029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 msgr2=0x7fe250102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.280+0000 7fe25532b640 1 --2- 172.21.15.31:0/1691261029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe250102dc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe240009900 tx=0x7fe24002a410 comp rx=0 tx=0).stop 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 -- 172.21.15.31:0/1691261029 shutdown_connections 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 --2- 172.21.15.31:0/1691261029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe250108c30 0x7fe25010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 --2- 172.21.15.31:0/1691261029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2501082e0 0x7fe2501086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 --2- 172.21.15.31:0/1691261029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe250102dc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 -- 172.21.15.31:0/1691261029 >> 172.21.15.31:0/1691261029 conn(0x7fe25007a740 msgr2=0x7fe25007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 -- 172.21.15.31:0/1691261029 shutdown_connections 2024-09-21T06:25:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 -- 172.21.15.31:0/1691261029 wait complete. 2024-09-21T06:25:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.281+0000 7fe25532b640 1 Processor -- start 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 -- start start 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe25019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2501082e0 0x7fe25019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe250108c30 0x7fe25019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501128f0 con 0x7fe2501029b0 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe250112770 con 0x7fe2501082e0 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe25532b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe250112a70 con 0x7fe250108c30 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe24effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe25019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe24e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2501082e0 0x7fe25019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe24effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe25019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37410/0 (socket says 172.21.15.31:37410) 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe24effd640 1 -- 172.21.15.31:0/905786905 learned_addr learned my addr 172.21.15.31:0/905786905 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.282+0000 7fe24f7fe640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe250108c30 0x7fe25019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24e7fc640 1 -- 172.21.15.31:0/905786905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe250108c30 msgr2=0x7fe25019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe250108c30 0x7fe25019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24e7fc640 1 -- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 msgr2=0x7fe25019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe25019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24e7fc640 1 -- 172.21.15.31:0/905786905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2501a6580 con 0x7fe2501082e0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24effd640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe25019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2501082e0 0x7fe25019b800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe25006c9c0 tx=0x7fe23800da70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe247fff640 1 -- 172.21.15.31:0/905786905 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238017070 con 0x7fe2501082e0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2501a6870 con 0x7fe2501082e0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe247fff640 1 -- 172.21.15.31:0/905786905 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380047a0 con 0x7fe2501082e0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe247fff640 1 -- 172.21.15.31:0/905786905 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238009a70 con 0x7fe2501082e0 2024-09-21T06:25:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.283+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501a6dc0 con 0x7fe2501082e0 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 Processor -- start 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 -- start start 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 0x5619b78aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 0x5619b78aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5619b781f6f0 con 0x5619b74dc2f0 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5619b780eb20 con 0x5619b75051b0 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.286+0000 7fe25c29a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5619b7812930 con 0x5619b74e44d0 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 0x5619b78aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 0x5619b78aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 0x5619b78aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47930/0 (socket says 172.21.15.31:47930) 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 0x5619b78aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37426/0 (socket says 172.21.15.31:37426) 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24f7fe640 1 -- 172.21.15.31:0/2347793823 learned_addr learned my addr 172.21.15.31:0/2347793823 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe247fff640 1 -- 172.21.15.31:0/905786905 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe238009c10 con 0x7fe2501082e0 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 -- 172.21.15.31:0/2347793823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 msgr2=0x5619b78ab430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ab430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 -- 172.21.15.31:0/2347793823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 msgr2=0x5619b78aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 0x5619b78aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 -- 172.21.15.31:0/2347793823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b77d4530 con 0x5619b74dc2f0 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24f7fe640 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 0x5619b78aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe24effd640 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 0x5619b78aaef0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe240004770 tx=0x7fe2400314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89d74186fefe9bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.287+0000 7fe2457fa640 1 -- 172.21.15.31:0/2347793823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240035040 con 0x5619b74dc2f0 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.288+0000 7fe2457fa640 1 -- 172.21.15.31:0/2347793823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240031890 con 0x5619b74dc2f0 2024-09-21T06:25:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.288+0000 7fe2457fa640 1 -- 172.21.15.31:0/2347793823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2400049c0 con 0x5619b74dc2f0 2024-09-21T06:25:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.288+0000 7fe247fff640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe23c0b6680 0x7fe23c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.288+0000 7fe24f7fe640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe23c0b6680 0x7fe23c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.288+0000 7fe247fff640 1 -- 172.21.15.31:0/905786905 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fe2380e6f80 con 0x7fe2501082e0 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 -- 172.21.15.31:0/2347793823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 msgr2=0x5619b78aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 0x5619b78aaef0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe240004770 tx=0x7fe2400314a0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 -- 172.21.15.31:0/2347793823 shutdown_connections 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74dc2f0 0x5619b78aaef0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 --2- 172.21.15.31:0/2347793823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b75051b0 0x5619b78aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 -- 172.21.15.31:0/2347793823 >> 172.21.15.31:0/2347793823 conn(0x5619b74f9300 msgr2=0x5619b78a88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe24f7fe640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe23c0b6680 0x7fe23c0b8b90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe2480127c0 tx=0x7fe248009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 -- 172.21.15.31:0/2347793823 shutdown_connections 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 -- 172.21.15.31:0/2347793823 wait complete. 2024-09-21T06:25:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 -- start start 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.289+0000 7fe25c29a740 1 Processor -- start 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 0x5619b78b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b75051b0 0x5619b78c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5619b7746ab0 con 0x5619b74e44d0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5619b77b4730 con 0x5619b74dc2f0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5619b78b90d0 con 0x5619b75051b0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe25c29a740 10 client.0 _ll_register_callbacks cb 0x5619b78086b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 0x5619b78b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b75051b0 0x5619b78c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 0x5619b78b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37442/0 (socket says 172.21.15.31:37442) 2024-09-21T06:25:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47936/0 (socket says 172.21.15.31:47936) 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b75051b0 0x5619b78c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60088/0 (socket says 172.21.15.31:60088) 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24e7fc640 1 -- 172.21.15.31:0/1736472134 learned_addr learned my addr 172.21.15.31:0/1736472134 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 -- 172.21.15.31:0/1736472134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b75051b0 msgr2=0x5619b78c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b75051b0 0x5619b78c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 -- 172.21.15.31:0/1736472134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 msgr2=0x5619b78b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 0x5619b78b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 -- 172.21.15.31:0/1736472134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b7570790 con 0x5619b74dc2f0 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 0x5619b78b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe24effd640 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b60c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe240005940 tx=0x7fe240031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7080528a2f58012 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.290+0000 7fe2367fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe2367fc640 1 -- 172.21.15.31:0/1736472134 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24004e070 con 0x5619b74dc2f0 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe2367fc640 1 -- 172.21.15.31:0/1736472134 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fe2400035f0 con 0x5619b74dc2f0 2024-09-21T06:25:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe240031890 con 0x5619b74dc2f0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe2367fc640 1 -- 172.21.15.31:0/1736472134 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe24005a050 con 0x5619b74dc2f0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 10 client.27941 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5619b75e9620 con 0x5619b74dc2f0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe2367fc640 1 -- 172.21.15.31:0/1736472134 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240031890 con 0x5619b74dc2f0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe2367fc640 1 -- 172.21.15.31:0/1736472134 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe240046720 con 0x5619b74dc2f0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 10 client.27941 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 10 client.27941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 20 client.27941 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74c3d80 0x5619b78c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.291+0000 7fe25c29a740 4 client.27941 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.292+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619b7821640 con 0x5619b74c3d80 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.292+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74c3d80 0x5619b78c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.292+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74c3d80 0x5619b78c7fd0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bde731923341295 server_cookie=e70b22777bdedc2c in_seq=0 out_seq=0 2024-09-21T06:25:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.292+0000 7fe2367fc640 10 client.27941 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe2367fc640 1 -- 172.21.15.31:0/1736472134 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5619b7821640 con 0x5619b74c3d80 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe2367fc640 10 client.27941 handle_command_reply: tid=0 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 client.27941 shutdown 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 msgr2=0x5619b78b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b60c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe240005940 tx=0x7fe240031400 comp rx=0 tx=0).stop 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 shutdown_connections 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74c3d80 0x5619b78c7fd0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b75051b0 0x5619b78c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe2367fc640 0 client.27941 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b74e44d0 0x5619b78b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1736472134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b60c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 >> 172.21.15.31:0/1736472134 conn(0x5619b74f9300 msgr2=0x5619b78ab9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 shutdown_connections 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 1 -- 172.21.15.31:0/1736472134 wait complete. 2024-09-21T06:25:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.293+0000 7fe25c29a740 20 client.27941 trim_cache size 0 max 16384 2024-09-21T06:25:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.309+0000 7fe25c29a740 1 Processor -- start 2024-09-21T06:25:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.309+0000 7fe25c29a740 1 -- start start 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b78b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe25c29a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501ad710 con 0x5619b75051b0 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe25c29a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501aff70 con 0x5619b74dc2f0 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe25c29a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501add60 con 0x5619b74e44d0 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b78b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b78b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37448/0 (socket says 172.21.15.31:37448) 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24f7fe640 1 -- 172.21.15.31:0/1059394515 learned_addr learned my addr 172.21.15.31:0/1059394515 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24effd640 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24effd640 1 -- 172.21.15.31:0/1059394515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 msgr2=0x5619b78ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24effd640 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24effd640 1 -- 172.21.15.31:0/1059394515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 msgr2=0x5619b78b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.310+0000 7fe24effd640 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b78b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.311+0000 7fe24effd640 1 -- 172.21.15.31:0/1059394515 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b78acce0 con 0x5619b74dc2f0 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.311+0000 7fe24effd640 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b4f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe240006010 tx=0x7fe2400314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef291386a6e13fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.311+0000 7fe244ff9640 1 -- 172.21.15.31:0/1059394515 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24004e070 con 0x5619b74dc2f0 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.311+0000 7fe244ff9640 1 -- 172.21.15.31:0/1059394515 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240046900 con 0x5619b74dc2f0 2024-09-21T06:25:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.311+0000 7fe244ff9640 1 -- 172.21.15.31:0/1059394515 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240035040 con 0x5619b74dc2f0 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 -- 172.21.15.31:0/1059394515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 msgr2=0x5619b78b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b4f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe240006010 tx=0x7fe2400314a0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 -- 172.21.15.31:0/1059394515 shutdown_connections 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b78b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 --2- 172.21.15.31:0/1059394515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78b4f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 -- 172.21.15.31:0/1059394515 >> 172.21.15.31:0/1059394515 conn(0x5619b74f9300 msgr2=0x5619b78b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 -- 172.21.15.31:0/1059394515 shutdown_connections 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.312+0000 7fe25c29a740 1 -- 172.21.15.31:0/1059394515 wait complete. 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 -- start start 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 Processor -- start 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b7930750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501ae430 con 0x5619b75051b0 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501af8a0 con 0x5619b74dc2f0 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2501aeb00 con 0x5619b74e44d0 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe25c29a740 10 client.0 _ll_register_callbacks cb 0x7fe240040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b7930750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe24e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.313+0000 7fe24f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b7930750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37462/0 (socket says 172.21.15.31:37462) 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24f7fe640 1 -- 172.21.15.31:0/3374577273 learned_addr learned my addr 172.21.15.31:0/3374577273 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24effd640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24e7fc640 1 -- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 msgr2=0x5619b78ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24e7fc640 1 -- 172.21.15.31:0/3374577273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 msgr2=0x5619b7930750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b7930750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24e7fc640 1 -- 172.21.15.31:0/3374577273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b78cb150 con 0x5619b74e44d0 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24effd640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe24e7fc640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ca880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe2380a93b0 tx=0x7fe23800bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e28cd6638e74421 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe2367fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe2367fc640 1 -- 172.21.15.31:0/3374577273 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238098070 con 0x5619b74e44d0 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe2367fc640 1 -- 172.21.15.31:0/3374577273 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7fe2380e4960 con 0x5619b74e44d0 2024-09-21T06:25:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe240046900 con 0x5619b74e44d0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe2367fc640 1 -- 172.21.15.31:0/3374577273 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe238020040 con 0x5619b74e44d0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe25c29a740 10 client.27970 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.314+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5619b7931d80 con 0x5619b74e44d0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe2367fc640 1 -- 172.21.15.31:0/3374577273 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe240046900 con 0x5619b74e44d0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe2367fc640 1 -- 172.21.15.31:0/3374577273 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe2380a9050 con 0x5619b74e44d0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe25c29a740 10 client.27970 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe25c29a740 10 client.27970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe25c29a740 20 client.27970 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe25c29a740 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74991c0 0x5619b79344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe25c29a740 4 client.27970 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619b781d0f0 con 0x5619b74991c0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe24effd640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74991c0 0x5619b79344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.315+0000 7fe2367fc640 10 client.27970 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.316+0000 7fe24effd640 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74991c0 0x5619b79344d0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0bd9ffebfb2d142 server_cookie=98b1bbd716848b8 in_seq=0 out_seq=0 2024-09-21T06:25:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe2367fc640 1 -- 172.21.15.31:0/3374577273 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619b781d0f0 con 0x5619b74991c0 2024-09-21T06:25:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe2367fc640 10 client.27970 handle_command_reply: tid=0 2024-09-21T06:25:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 client.27970 shutdown 2024-09-21T06:25:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 msgr2=0x5619b78ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ca880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe2380a93b0 tx=0x7fe23800bf80 comp rx=0 tx=0).stop 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 shutdown_connections 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5619b74991c0 0x5619b79344d0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5619b75051b0 0x5619b7930750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5619b74e44d0 0x5619b78ca880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe2367fc640 0 client.27970 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 --2- 172.21.15.31:0/3374577273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619b74dc2f0 0x5619b78ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 >> 172.21.15.31:0/3374577273 conn(0x5619b74f9300 msgr2=0x5619b78ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 shutdown_connections 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 1 -- 172.21.15.31:0/3374577273 wait complete. 2024-09-21T06:25:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.342+0000 7fe25c29a740 20 client.27970 trim_cache size 0 max 16384 2024-09-21T06:25:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:25:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:25:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:25:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:25:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe23c0b6680 msgr2=0x7fe23c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe23c0b6680 0x7fe23c0b8b90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe2480127c0 tx=0x7fe248009500 comp rx=0 tx=0).stop 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2501082e0 msgr2=0x7fe25019b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2501082e0 0x7fe25019b800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe25006c9c0 tx=0x7fe23800da70 comp rx=0 tx=0).stop 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 shutdown_connections 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe250108c30 0x7fe25019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2501082e0 0x7fe25019b800 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe23c0b6680 0x7fe23c0b8b90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 --2- 172.21.15.31:0/905786905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2501029b0 0x7fe25019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 >> 172.21.15.31:0/905786905 conn(0x7fe25007a740 msgr2=0x7fe250106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 shutdown_connections 2024-09-21T06:25:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:04.343+0000 7fe25532b640 1 -- 172.21.15.31:0/905786905 wait complete. 2024-09-21T06:25:04.475 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:25:04.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:25:04.764 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 805.0 s, 508.4 tps, lat 183.356 ms stddev 208.949 2024-09-21T06:25:05.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 -- 172.21.15.31:0/2695886711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c801062d0 msgr2=0x7f4c801066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 --2- 172.21.15.31:0/2695886711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c801062d0 0x7f4c801066e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009930 tx=0x7f4c7402a440 comp rx=0 tx=0).stop 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 -- 172.21.15.31:0/2695886711 shutdown_connections 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 --2- 172.21.15.31:0/2695886711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c80106c20 0x7f4c801106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 --2- 172.21.15.31:0/2695886711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c801062d0 0x7f4c801066e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 --2- 172.21.15.31:0/2695886711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80102080 0x7f4c80102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 -- 172.21.15.31:0/2695886711 >> 172.21.15.31:0/2695886711 conn(0x7f4c80079f90 msgr2=0x7f4c8007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 -- 172.21.15.31:0/2695886711 shutdown_connections 2024-09-21T06:25:05.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.251+0000 7f4c85212640 1 -- 172.21.15.31:0/2695886711 wait complete. 2024-09-21T06:25:05.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 Processor -- start 2024-09-21T06:25:05.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 -- start start 2024-09-21T06:25:05.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80102080 0x7f4c8019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:05.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c801062d0 0x7f4c8019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:05.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 0x7f4c8019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80113b40 con 0x7f4c80106c20 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801139c0 con 0x7f4c801062d0 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c85212640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80113cc0 con 0x7f4c80102080 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c7f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 0x7f4c8019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c7ed9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80102080 0x7f4c8019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c7e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c801062d0 0x7f4c8019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c7f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 0x7f4c8019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37492/0 (socket says 172.21.15.31:37492) 2024-09-21T06:25:05.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.253+0000 7f4c7f59e640 1 -- 172.21.15.31:0/3387067427 learned_addr learned my addr 172.21.15.31:0/3387067427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7ed9d640 1 -- 172.21.15.31:0/3387067427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c801062d0 msgr2=0x7f4c8019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7ed9d640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c801062d0 0x7f4c8019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7ed9d640 1 -- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 msgr2=0x7f4c8019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7ed9d640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 0x7f4c8019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7ed9d640 1 -- 172.21.15.31:0/3387067427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c801a0440 con 0x7f4c80102080 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7e59c640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c801062d0 0x7f4c8019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7f59e640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 0x7f4c8019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c7ed9d640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80102080 0x7f4c8019a4d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000d6a0 tx=0x7f4c7000db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70017070 con 0x7f4c80102080 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c801a0730 con 0x7f4c80102080 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c70009cd0 con 0x7f4c80102080 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.254+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70003b50 con 0x7f4c80102080 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.255+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c801a6e90 con 0x7f4c80102080 2024-09-21T06:25:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.256+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c80102490 con 0x7f4c80102080 2024-09-21T06:25:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.258+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4c700042b0 con 0x7f4c80102080 2024-09-21T06:25:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.259+0000 7f4c67fff640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c680b6680 0x7f4c680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.259+0000 7f4c7e59c640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c680b6680 0x7f4c680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.260+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8499+0+0 (secure 0 0 0) 0x7f4c700e7060 con 0x7f4c80102080 2024-09-21T06:25:05.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.260+0000 7f4c7e59c640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c680b6680 0x7f4c680b8b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009230 tx=0x7f4c74005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:05.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.262+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c700b19a0 con 0x7f4c80102080 2024-09-21T06:25:05.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.505+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c8019b600 con 0x7f4c80102080 2024-09-21T06:25:05.507 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:05 smithi031 ceph-mon[31237]: pgmap v739: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 701 op/s 2024-09-21T06:25:05.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.506+0000 7f4c67fff640 1 -- 172.21.15.31:0/3387067427 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f4c700b68e0 con 0x7f4c80102080 2024-09-21T06:25:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:25:05.508 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:25:05.508 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c680b6680 msgr2=0x7f4c680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c680b6680 0x7f4c680b8b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009230 tx=0x7f4c74005850 comp rx=0 tx=0).stop 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80102080 msgr2=0x7f4c8019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80102080 0x7f4c8019a4d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000d6a0 tx=0x7f4c7000db60 comp rx=0 tx=0).stop 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 shutdown_connections 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f4c680b6680 0x7f4c680b8b90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c80106c20 0x7f4c8019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c801062d0 0x7f4c8019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 --2- 172.21.15.31:0/3387067427 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c80102080 0x7f4c8019a4d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:05.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 >> 172.21.15.31:0/3387067427 conn(0x7f4c80079f90 msgr2=0x7f4c80107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:05.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 shutdown_connections 2024-09-21T06:25:05.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:05.508+0000 7f4c85212640 1 -- 172.21.15.31:0/3387067427 wait complete. 2024-09-21T06:25:05.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:05 smithi110 ceph-mon[34550]: pgmap v739: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 701 op/s 2024-09-21T06:25:05.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:05 smithi107 ceph-mon[34380]: pgmap v739: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.6 MiB/s wr, 701 op/s 2024-09-21T06:25:06.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:25:06.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:06 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3387067427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:06.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:06 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3387067427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:06.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:06 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3387067427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:06.731 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:07.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.376+0000 7f002a693640 1 -- 172.21.15.31:0/3345174952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410ee60 msgr2=0x7f002410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.376+0000 7f002a693640 1 --2- 172.21.15.31:0/3345174952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410ee60 0x7f002410f270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f001c009ca0 tx=0x7f001c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.377+0000 7f002a693640 1 -- 172.21.15.31:0/3345174952 shutdown_connections 2024-09-21T06:25:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.377+0000 7f002a693640 1 --2- 172.21.15.31:0/3345174952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410a900 0x7f002410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.377+0000 7f002a693640 1 --2- 172.21.15.31:0/3345174952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410f7b0 0x7f002410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.377+0000 7f002a693640 1 --2- 172.21.15.31:0/3345174952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410ee60 0x7f002410f270 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.377+0000 7f002a693640 1 -- 172.21.15.31:0/3345174952 >> 172.21.15.31:0/3345174952 conn(0x7f00240712f0 msgr2=0x7f0024073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.379+0000 7f002a693640 1 -- 172.21.15.31:0/3345174952 shutdown_connections 2024-09-21T06:25:07.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.379+0000 7f002a693640 1 -- 172.21.15.31:0/3345174952 wait complete. 2024-09-21T06:25:07.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.380+0000 7f002a693640 1 Processor -- start 2024-09-21T06:25:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.380+0000 7f002a693640 1 -- start start 2024-09-21T06:25:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f002a693640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 0x7f00241179d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f002a693640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 0x7f0024117f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f002a693640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410f7b0 0x7f00241aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f002a693640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f00241140d0 con 0x7f002410f7b0 2024-09-21T06:25:07.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f002a693640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0024113f50 con 0x7f002410a900 2024-09-21T06:25:07.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f002a693640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f002411a2e0 con 0x7f002410ee60 2024-09-21T06:25:07.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f0023fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 0x7f00241179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.381+0000 7f00237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 0x7f0024117f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.382+0000 7f00237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 0x7f0024117f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60146/0 (socket says 172.21.15.31:60146) 2024-09-21T06:25:07.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.382+0000 7f0023fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 0x7f00241179d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47982/0 (socket says 172.21.15.31:47982) 2024-09-21T06:25:07.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.382+0000 7f0023fff640 1 -- 172.21.15.31:0/2299171113 learned_addr learned my addr 172.21.15.31:0/2299171113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f00237fe640 1 -- 172.21.15.31:0/2299171113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 msgr2=0x7f00241179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f00237fe640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 0x7f00241179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f00237fe640 1 -- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410f7b0 msgr2=0x7f00241aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f00237fe640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410f7b0 0x7f00241aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f00237fe640 1 -- 172.21.15.31:0/2299171113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00241af340 con 0x7f002410ee60 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f0023fff640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 0x7f00241179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.383+0000 7f00237fe640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 0x7f0024117f70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f000c00e9c0 tx=0x7f000c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.385+0000 7f00217fa640 1 -- 172.21.15.31:0/2299171113 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c00cd80 con 0x7f002410ee60 2024-09-21T06:25:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.385+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00241af630 con 0x7f002410ee60 2024-09-21T06:25:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.385+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002411e6f0 con 0x7f002410ee60 2024-09-21T06:25:07.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.385+0000 7f00217fa640 1 -- 172.21.15.31:0/2299171113 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c0035f0 con 0x7f002410ee60 2024-09-21T06:25:07.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.385+0000 7f00217fa640 1 -- 172.21.15.31:0/2299171113 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c010690 con 0x7f002410ee60 2024-09-21T06:25:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.389+0000 7f00217fa640 1 -- 172.21.15.31:0/2299171113 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f000c004020 con 0x7f002410ee60 2024-09-21T06:25:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.390+0000 7f00217fa640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00100b6800 0x7f00100b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.390+0000 7f0023fff640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00100b6800 0x7f00100b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.390+0000 7f00217fa640 1 -- 172.21.15.31:0/2299171113 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f000c0e77d0 con 0x7f002410ee60 2024-09-21T06:25:07.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.391+0000 7f0023fff640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00100b6800 0x7f00100b8d10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f001c000f80 tx=0x7f001c030f00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.395+0000 7f0030c07740 1 Processor -- start 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.395+0000 7f0030c07740 1 -- start start 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0030c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 0x55d5219582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0030c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d521958800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0030c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521958d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0030c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d5218ceea0 con 0x55d5215881b0 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0030c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d5218be2d0 con 0x55d5215b1070 2024-09-21T06:25:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0030c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d5218c20e0 con 0x55d521590390 2024-09-21T06:25:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0023fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 0x55d5219582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0023fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 0x55d5219582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47992/0 (socket says 172.21.15.31:47992) 2024-09-21T06:25:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.396+0000 7f0023fff640 1 -- 172.21.15.31:0/355635125 learned_addr learned my addr 172.21.15.31:0/355635125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.397+0000 7f0028c30640 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521958d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.397+0000 7f0023fff640 1 -- 172.21.15.31:0/355635125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 msgr2=0x55d521958d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.397+0000 7f0023fff640 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521958d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.397+0000 7f0023fff640 1 -- 172.21.15.31:0/355635125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 msgr2=0x55d521958800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.397+0000 7f0023fff640 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d521958800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.397+0000 7f0023fff640 1 -- 172.21.15.31:0/355635125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d52161c650 con 0x55d5215b1070 2024-09-21T06:25:07.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.399+0000 7f0023fff640 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 0x55d5219582c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f001c009dc0 tx=0x7f001c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f6544d3e57ec323 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.399+0000 7f001a7fc640 1 -- 172.21.15.31:0/355635125 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c045660 con 0x55d5215b1070 2024-09-21T06:25:07.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.399+0000 7f001a7fc640 1 -- 172.21.15.31:0/355635125 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f001c045cb0 con 0x55d5215b1070 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.400+0000 7f0030c07740 1 -- 172.21.15.31:0/355635125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 msgr2=0x55d5219582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.400+0000 7f0030c07740 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 0x55d5219582c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f001c009dc0 tx=0x7f001c03f000 comp rx=0 tx=0).stop 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.400+0000 7f0030c07740 1 -- 172.21.15.31:0/355635125 shutdown_connections 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.400+0000 7f0030c07740 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521958d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.401+0000 7f0030c07740 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d521958800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.401+0000 7f0030c07740 1 --2- 172.21.15.31:0/355635125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5215b1070 0x55d5219582c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.401+0000 7f0030c07740 1 -- 172.21.15.31:0/355635125 >> 172.21.15.31:0/355635125 conn(0x55d5215a5460 msgr2=0x55d5219561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.401+0000 7f0030c07740 1 -- 172.21.15.31:0/355635125 shutdown_connections 2024-09-21T06:25:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.401+0000 7f0030c07740 1 -- 172.21.15.31:0/355635125 wait complete. 2024-09-21T06:25:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.401+0000 7f0030c07740 1 -- start start 2024-09-21T06:25:07.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.402+0000 7f0030c07740 1 Processor -- start 2024-09-21T06:25:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d5219636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521963c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521964170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55d52171f700 con 0x55d5215881b0 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d521859160 con 0x55d52156f9f0 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d521966910 con 0x55d521590390 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0030c07740 10 client.0 _ll_register_callbacks cb 0x55d5218729e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f00237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521963c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f00237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521963c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60164/0 (socket says 172.21.15.31:60164) 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f00237fe640 1 -- 172.21.15.31:0/926573291 learned_addr learned my addr 172.21.15.31:0/926573291 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:07.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.403+0000 7f0023fff640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d5219636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f0028c30640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521964170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f00237fe640 1 -- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 msgr2=0x55d521964170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f00237fe640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521964170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f00237fe640 1 -- 172.21.15.31:0/926573291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 msgr2=0x55d5219636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f00237fe640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d5219636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f00237fe640 1 -- 172.21.15.31:0/926573291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d521886c90 con 0x55d521590390 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.404+0000 7f0028c30640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521964170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.405+0000 7f00237fe640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521963c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f000c0e4f70 tx=0x7f000c0ace40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db7850f40252e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.405+0000 7f00037fe640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:07.408 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:07 smithi031 ceph-mon[31237]: pgmap v740: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 586 op/s 2024-09-21T06:25:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.408+0000 7f00037fe640 1 -- 172.21.15.31:0/926573291 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0b7030 con 0x55d521590390 2024-09-21T06:25:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.408+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5218d0df0 con 0x55d521590390 2024-09-21T06:25:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.408+0000 7f00037fe640 1 -- 172.21.15.31:0/926573291 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f000c0ac870 con 0x55d521590390 2024-09-21T06:25:07.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.408+0000 7f00037fe640 1 -- 172.21.15.31:0/926573291 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f000c0aca90 con 0x55d521590390 2024-09-21T06:25:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.414+0000 7f0030c07740 10 client.27988 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:07.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.414+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d5219724e0 con 0x55d521590390 2024-09-21T06:25:07.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.415+0000 7f00037fe640 1 -- 172.21.15.31:0/926573291 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d5218d0df0 con 0x55d521590390 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.415+0000 7f00037fe640 1 -- 172.21.15.31:0/926573291 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f000c0b3280 con 0x55d521590390 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.415+0000 7f0030c07740 10 client.27988 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.415+0000 7f0030c07740 10 client.27988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.415+0000 7f0030c07740 20 client.27988 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.415+0000 7f0030c07740 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d521544da0 0x55d521975e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.416+0000 7f0030c07740 4 client.27988 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.416+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5218cc8a0 con 0x55d521544da0 2024-09-21T06:25:07.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.417+0000 7f0023fff640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d521544da0 0x55d521975e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.417+0000 7f00037fe640 10 client.27988 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:07.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.417+0000 7f0023fff640 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d521544da0 0x55d521975e40 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62b69eea7c1d9cc9 server_cookie=6996329de1533039 in_seq=0 out_seq=0 2024-09-21T06:25:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f00037fe640 1 -- 172.21.15.31:0/926573291 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5218cc8a0 con 0x55d521544da0 2024-09-21T06:25:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f00037fe640 10 client.27988 handle_command_reply: tid=0 2024-09-21T06:25:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 client.27988 shutdown 2024-09-21T06:25:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 msgr2=0x55d521963c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521963c30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f000c0e4f70 tx=0x7f000c0ace40 comp rx=0 tx=0).stop 2024-09-21T06:25:07.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 shutdown_connections 2024-09-21T06:25:07.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d521544da0 0x55d521975e40 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521964170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d521590390 0x55d521963c30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 --2- 172.21.15.31:0/926573291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5215881b0 0x55d5219636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.418+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 >> 172.21.15.31:0/926573291 conn(0x55d5215a5460 msgr2=0x55d5219592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.419+0000 7f00037fe640 0 client.27988 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:07.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.419+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 shutdown_connections 2024-09-21T06:25:07.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.419+0000 7f0030c07740 1 -- 172.21.15.31:0/926573291 wait complete. 2024-09-21T06:25:07.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.419+0000 7f0030c07740 20 client.27988 trim_cache size 0 max 16384 2024-09-21T06:25:07.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.445+0000 7f0030c07740 1 Processor -- start 2024-09-21T06:25:07.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.445+0000 7f0030c07740 1 -- start start 2024-09-21T06:25:07.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0030c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 0x55d521962930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0030c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d521961160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0030c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52195ed00 0x55d521959a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0030c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f000400c970 con 0x55d52156f9f0 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0030c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f000400f1d0 con 0x55d52195ed00 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0030c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f000400cfc0 con 0x55d5215881b0 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0023fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 0x55d521962930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0023fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 0x55d521962930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37530/0 (socket says 172.21.15.31:37530) 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f0023fff640 1 -- 172.21.15.31:0/3231732619 learned_addr learned my addr 172.21.15.31:0/3231732619 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.446+0000 7f00237fe640 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d521961160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f0028c30640 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52195ed00 0x55d521959a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f00237fe640 1 -- 172.21.15.31:0/3231732619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52195ed00 msgr2=0x55d521959a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f00237fe640 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52195ed00 0x55d521959a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f00237fe640 1 -- 172.21.15.31:0/3231732619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 msgr2=0x55d521962930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f00237fe640 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 0x55d521962930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f00237fe640 1 -- 172.21.15.31:0/3231732619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5215b12a0 con 0x55d5215881b0 2024-09-21T06:25:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f0023fff640 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 0x55d521962930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f00237fe640 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d521961160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f000c0aa880 tx=0x7f000c0acf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a42aa61147fb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f0019ffb640 1 -- 172.21.15.31:0/3231732619 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0cc070 con 0x55d5215881b0 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.447+0000 7f0019ffb640 1 -- 172.21.15.31:0/3231732619 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c0b7030 con 0x55d5215881b0 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.448+0000 7f0019ffb640 1 -- 172.21.15.31:0/3231732619 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0b6730 con 0x55d5215881b0 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 -- 172.21.15.31:0/3231732619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 msgr2=0x55d521961160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d521961160 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f000c0aa880 tx=0x7f000c0acf60 comp rx=0 tx=0).stop 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 -- 172.21.15.31:0/3231732619 shutdown_connections 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52195ed00 0x55d521959a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d521961160 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 --2- 172.21.15.31:0/3231732619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52156f9f0 0x55d521962930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 -- 172.21.15.31:0/3231732619 >> 172.21.15.31:0/3231732619 conn(0x55d5215a5460 msgr2=0x55d5219592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 -- 172.21.15.31:0/3231732619 shutdown_connections 2024-09-21T06:25:07.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.449+0000 7f0030c07740 1 -- 172.21.15.31:0/3231732619 wait complete. 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.450+0000 7f0030c07740 1 -- start start 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.450+0000 7f0030c07740 1 Processor -- start 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521983f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d5219844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52195ed00 0x55d5219db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f000400d690 con 0x55d52195ed00 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f000400eb00 con 0x55d52156f9f0 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f000400dd60 con 0x55d5215881b0 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0030c07740 10 client.0 _ll_register_callbacks cb 0x7f000c0a9c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f00237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d5219844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f00237fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d5219844f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60182/0 (socket says 172.21.15.31:60182) 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f00237fe640 1 -- 172.21.15.31:0/2895571329 learned_addr learned my addr 172.21.15.31:0/2895571329 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.451+0000 7f0023fff640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521983f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f0023fff640 1 -- 172.21.15.31:0/2895571329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 msgr2=0x55d5219844f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f0023fff640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d5219844f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f0023fff640 1 -- 172.21.15.31:0/2895571329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52195ed00 msgr2=0x55d5219db820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f0023fff640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52195ed00 0x55d5219db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f0023fff640 1 -- 172.21.15.31:0/2895571329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5219dbe90 con 0x55d52156f9f0 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f00237fe640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d5219844f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.452+0000 7f0023fff640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521983f30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f001c02aea0 tx=0x7f001c030800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8fe58ec610a28ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f00037fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f00037fe640 1 -- 172.21.15.31:0/2895571329 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001c03c5d0 con 0x55d52156f9f0 2024-09-21T06:25:07.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f00037fe640 1 -- 172.21.15.31:0/2895571329 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f001c03c770 con 0x55d52156f9f0 2024-09-21T06:25:07.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5218c9cd0 con 0x55d52156f9f0 2024-09-21T06:25:07.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f00037fe640 1 -- 172.21.15.31:0/2895571329 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f001c03c970 con 0x55d52156f9f0 2024-09-21T06:25:07.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f0030c07740 10 client.27968 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:07.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d5219e3190 con 0x55d52156f9f0 2024-09-21T06:25:07.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.453+0000 7f00037fe640 1 -- 172.21.15.31:0/2895571329 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f001c068070 con 0x55d52156f9f0 2024-09-21T06:25:07.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f00037fe640 1 -- 172.21.15.31:0/2895571329 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f001c04e070 con 0x55d52156f9f0 2024-09-21T06:25:07.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f0030c07740 10 client.27968 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f0030c07740 10 client.27968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f0030c07740 20 client.27968 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f0030c07740 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d5219e35c0 0x55d5219e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f0030c07740 4 client.27968 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5218c9cd0 con 0x55d5219e35c0 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.454+0000 7f00237fe640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d5219e35c0 0x55d5219e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.455+0000 7f00037fe640 10 client.27968 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.455+0000 7f00237fe640 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d5219e35c0 0x55d5219e59e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f247ebdad8e743 server_cookie=5ad869540075d210 in_seq=0 out_seq=0 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.455+0000 7f00037fe640 1 -- 172.21.15.31:0/2895571329 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5218c9cd0 con 0x55d5219e35c0 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.455+0000 7f00037fe640 10 client.27968 handle_command_reply: tid=0 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.455+0000 7f0030c07740 1 client.27968 shutdown 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.455+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 msgr2=0x55d521983f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521983f30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f001c02aea0 tx=0x7f001c030800 comp rx=0 tx=0).stop 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 shutdown_connections 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55d5219e35c0 0x55d5219e59e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d52195ed00 0x55d5219db820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f00037fe640 0 client.27968 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5215881b0 0x55d5219844f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 --2- 172.21.15.31:0/2895571329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d52156f9f0 0x55d521983f30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 >> 172.21.15.31:0/2895571329 conn(0x55d5215a5460 msgr2=0x55d521964ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 shutdown_connections 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 1 -- 172.21.15.31:0/2895571329 wait complete. 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.456+0000 7f0030c07740 20 client.27968 trim_cache size 0 max 16384 2024-09-21T06:25:07.460 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:25:07.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.457+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00100b6800 msgr2=0x7f00100b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.457+0000 7f002a693640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00100b6800 0x7f00100b8d10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f001c000f80 tx=0x7f001c030f00 comp rx=0 tx=0).stop 2024-09-21T06:25:07.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.457+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 msgr2=0x7f0024117f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:07.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.457+0000 7f002a693640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 0x7f0024117f70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f000c00e9c0 tx=0x7f000c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 shutdown_connections 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f00100b6800 0x7f00100b8d10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f002410f7b0 0x7f00241aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f002410ee60 0x7f0024117f70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 --2- 172.21.15.31:0/2299171113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f002410a900 0x7f00241179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 >> 172.21.15.31:0/2299171113 conn(0x7f00240712f0 msgr2=0x7f002410cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 shutdown_connections 2024-09-21T06:25:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:07.458+0000 7f002a693640 1 -- 172.21.15.31:0/2299171113 wait complete. 2024-09-21T06:25:07.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:07 smithi110 ceph-mon[34550]: pgmap v740: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 586 op/s 2024-09-21T06:25:07.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:07 smithi107 ceph-mon[34380]: pgmap v740: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.1 MiB/s wr, 586 op/s 2024-09-21T06:25:08.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:08 smithi031 ceph-mon[31237]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T06:25:08.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:08 smithi110 ceph-mon[34550]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T06:25:08.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:08 smithi107 ceph-mon[34380]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T06:25:09.260 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:25:09.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:25:09.526 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:09.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:09 smithi110 ceph-mon[34550]: pgmap v742: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-09-21T06:25:09.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:09 smithi107 ceph-mon[34380]: pgmap v742: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-09-21T06:25:09.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:09 smithi031 ceph-mon[31237]: pgmap v742: 97 pgs: 97 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-09-21T06:25:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 -- 172.21.15.31:0/246077974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 msgr2=0x7f2954110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 --2- 172.21.15.31:0/246077974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 0x7f2954110330 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f294c009ca0 tx=0x7f294c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 -- 172.21.15.31:0/246077974 shutdown_connections 2024-09-21T06:25:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 --2- 172.21.15.31:0/246077974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 0x7f2954110330 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 --2- 172.21.15.31:0/246077974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 --2- 172.21.15.31:0/246077974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 0x7f2954104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 -- 172.21.15.31:0/246077974 >> 172.21.15.31:0/246077974 conn(0x7f29540797b0 msgr2=0x7f2954079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 -- 172.21.15.31:0/246077974 shutdown_connections 2024-09-21T06:25:10.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.033+0000 7f2958f79640 1 -- 172.21.15.31:0/246077974 wait complete. 2024-09-21T06:25:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 Processor -- start 2024-09-21T06:25:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 -- start start 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 0x7f295419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 0x7f295419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2954113ac0 con 0x7f295410de60 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2954113940 con 0x7f2954105380 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.034+0000 7f2958f79640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2954113c40 con 0x7f2954104a30 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f295259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 0x7f295419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f2952d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 0x7f295419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43584/0 (socket says 172.21.15.31:43584) 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f295259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 0x7f295419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52398/0 (socket says 172.21.15.31:52398) 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 -- 172.21.15.31:0/4160153176 learned_addr learned my addr 172.21.15.31:0/4160153176 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 -- 172.21.15.31:0/4160153176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 msgr2=0x7f295419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 0x7f295419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 -- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 msgr2=0x7f295419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 0x7f295419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 -- 172.21.15.31:0/4160153176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29541a03e0 con 0x7f2954105380 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294bfff640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295419aa10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.035+0000 7f294b7fe640 1 -- 172.21.15.31:0/4160153176 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c00ccc0 con 0x7f2954105380 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.036+0000 7f294b7fe640 1 -- 172.21.15.31:0/4160153176 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293c0108e0 con 0x7f2954105380 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.036+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29541a06d0 con 0x7f2954105380 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.036+0000 7f294b7fe640 1 -- 172.21.15.31:0/4160153176 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c018860 con 0x7f2954105380 2024-09-21T06:25:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.036+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2954102650 con 0x7f2954105380 2024-09-21T06:25:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.038+0000 7f294b7fe640 1 -- 172.21.15.31:0/4160153176 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f293c010430 con 0x7f2954105380 2024-09-21T06:25:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.038+0000 7f295f4ed740 1 Processor -- start 2024-09-21T06:25:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 -- start start 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d927430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d927970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d927eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55658d89de60 con 0x55658d55f670 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55658d88d290 con 0x55658d580350 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295f4ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55658d8910a0 con 0x55658d557490 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d927970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f2952d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d927430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f294bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d927eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f2952d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d927430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43588/0 (socket says 172.21.15.31:43588) 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f2952d9d640 1 -- 172.21.15.31:0/1186631765 learned_addr learned my addr 172.21.15.31:0/1186631765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f295259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d927970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52414/0 (socket says 172.21.15.31:52414) 2024-09-21T06:25:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.039+0000 7f2952d9d640 1 -- 172.21.15.31:0/1186631765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 msgr2=0x55658d927970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2952d9d640 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d927970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2952d9d640 1 -- 172.21.15.31:0/1186631765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 msgr2=0x55658d927eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2952d9d640 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d927eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2952d9d640 1 -- 172.21.15.31:0/1186631765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658d5eb930 con 0x55658d580350 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f295259c640 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d927970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f294b7fe640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f29400b6680 0x7f29400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f295259c640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f29400b6680 0x7f29400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2952d9d640 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d927430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f294c02f410 tx=0x7f294c02f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0bcf94b2a6c0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f294b7fe640 1 -- 172.21.15.31:0/4160153176 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f293c0e7a30 con 0x7f2954105380 2024-09-21T06:25:10.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2948ff9640 1 -- 172.21.15.31:0/1186631765 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c031cf0 con 0x55658d580350 2024-09-21T06:25:10.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2948ff9640 1 -- 172.21.15.31:0/1186631765 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294c036030 con 0x55658d580350 2024-09-21T06:25:10.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.040+0000 7f2948ff9640 1 -- 172.21.15.31:0/1186631765 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c004280 con 0x55658d580350 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 -- 172.21.15.31:0/1186631765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 msgr2=0x55658d927430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d927430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f294c02f410 tx=0x7f294c02f440 comp rx=0 tx=0).stop 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295259c640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f29400b6680 0x7f29400b8b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55658d9260b0 tx=0x7f2944005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 -- 172.21.15.31:0/1186631765 shutdown_connections 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d927eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d927970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 --2- 172.21.15.31:0/1186631765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d927430 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 -- 172.21.15.31:0/1186631765 >> 172.21.15.31:0/1186631765 conn(0x55658d574740 msgr2=0x55658d925320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 -- 172.21.15.31:0/1186631765 shutdown_connections 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.041+0000 7f295f4ed740 1 -- 172.21.15.31:0/1186631765 wait complete. 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 -- start start 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 Processor -- start 2024-09-21T06:25:10.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d933010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d55f670 0x55658d933570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d93ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55658d6ee890 con 0x55658d557490 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55658d827d40 con 0x55658d580350 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55658d935a30 con 0x55658d55f670 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295f4ed740 10 client.0 _ll_register_callbacks cb 0x55658d88bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d933010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d933010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45206/0 (socket says 172.21.15.31:45206) 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f295259c640 1 -- 172.21.15.31:0/3406025085 learned_addr learned my addr 172.21.15.31:0/3406025085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f2952d9d640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d93ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.042+0000 7f294bfff640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d55f670 0x55658d933570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f294bfff640 1 -- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 msgr2=0x55658d93ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f294bfff640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d93ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f294bfff640 1 -- 172.21.15.31:0/3406025085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 msgr2=0x55658d933010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f294bfff640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d933010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f294bfff640 1 -- 172.21.15.31:0/3406025085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658d93f310 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f295259c640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d933010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f294bfff640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d55f670 0x55658d933570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f293c0e5110 tx=0x7f293c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dd40a08006e54be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f2939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f2939ffb640 1 -- 172.21.15.31:0/3406025085 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0a9760 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c036030 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f2939ffb640 1 -- 172.21.15.31:0/3406025085 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f293c0ac590 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f2939ffb640 1 -- 172.21.15.31:0/3406025085 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f293c0ad030 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f295f4ed740 10 client.28006 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.043+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55658d941c40 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f2939ffb640 1 -- 172.21.15.31:0/3406025085 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294c036030 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f2939ffb640 1 -- 172.21.15.31:0/3406025085 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f293c0ac790 con 0x55658d55f670 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295f4ed740 10 client.28006 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:10.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295f4ed740 10 client.28006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295f4ed740 20 client.28006 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295f4ed740 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d53ecd0 0x55658d9450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295f4ed740 4 client.28006 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658d89fdb0 con 0x55658d53ecd0 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.044+0000 7f295259c640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d53ecd0 0x55658d9450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.045+0000 7f2939ffb640 10 client.28006 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.045+0000 7f295259c640 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d53ecd0 0x55658d9450d0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9b0f83b1c67cfc server_cookie=3955bcf8f27cc916 in_seq=0 out_seq=0 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.045+0000 7f2939ffb640 1 -- 172.21.15.31:0/3406025085 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55658d89fdb0 con 0x55658d53ecd0 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.045+0000 7f2939ffb640 10 client.28006 handle_command_reply: tid=0 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.045+0000 7f295f4ed740 1 client.28006 shutdown 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d55f670 msgr2=0x55658d933570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d55f670 0x55658d933570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f293c0e5110 tx=0x7f293c009740 comp rx=0 tx=0).stop 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 shutdown_connections 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d53ecd0 0x55658d9450d0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d580350 0x55658d93ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d55f670 0x55658d933570 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f2939ffb640 0 client.28006 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 --2- 172.21.15.31:0/3406025085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d933010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 >> 172.21.15.31:0/3406025085 conn(0x55658d574740 msgr2=0x55658d92a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 shutdown_connections 2024-09-21T06:25:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 1 -- 172.21.15.31:0/3406025085 wait complete. 2024-09-21T06:25:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.046+0000 7f295f4ed740 20 client.28006 trim_cache size 0 max 16384 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 Processor -- start 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 -- start start 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d53ecd0 0x55658d93ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d9326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d934a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f29541afa00 con 0x55658d55f670 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29541b2240 con 0x55658d53ecd0 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f295f4ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29541b0050 con 0x55658d557490 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f2952d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d934a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.064+0000 7f2952d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d934a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45218/0 (socket says 172.21.15.31:45218) 2024-09-21T06:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 -- 172.21.15.31:0/905616626 learned_addr learned my addr 172.21.15.31:0/905616626 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f295259c640 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d53ecd0 0x55658d93ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f294bfff640 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d9326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 -- 172.21.15.31:0/905616626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 msgr2=0x55658d9326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d9326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 -- 172.21.15.31:0/905616626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d53ecd0 msgr2=0x55658d93ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d53ecd0 0x55658d93ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 -- 172.21.15.31:0/905616626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658d92e6a0 con 0x55658d55f670 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f295259c640 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d53ecd0 0x55658d93ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f2952d9d640 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d934a40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f294c003d40 tx=0x7f294c003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65f339aaef0a830 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f293bfff640 1 -- 172.21.15.31:0/905616626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c04a040 con 0x55658d55f670 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f293bfff640 1 -- 172.21.15.31:0/905616626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294c056070 con 0x55658d55f670 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.065+0000 7f293bfff640 1 -- 172.21.15.31:0/905616626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294c02f8c0 con 0x55658d55f670 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 -- 172.21.15.31:0/905616626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 msgr2=0x55658d934a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d934a40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f294c003d40 tx=0x7f294c003e10 comp rx=0 tx=0).stop 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 -- 172.21.15.31:0/905616626 shutdown_connections 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d55f670 0x55658d934a40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d557490 0x55658d9326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 --2- 172.21.15.31:0/905616626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d53ecd0 0x55658d93ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 -- 172.21.15.31:0/905616626 >> 172.21.15.31:0/905616626 conn(0x55658d574740 msgr2=0x55658d934330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 -- 172.21.15.31:0/905616626 shutdown_connections 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.066+0000 7f295f4ed740 1 -- 172.21.15.31:0/905616626 wait complete. 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 -- start start 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 Processor -- start 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d53ecd0 0x55658d93ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d93b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d55f670 0x55658d935c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f29541b0700 con 0x55658d557490 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29541b1b70 con 0x55658d55f670 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f294bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d93b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f295f4ed740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29541b0dd0 con 0x55658d53ecd0 2024-09-21T06:25:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.067+0000 7f294bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d93b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45232/0 (socket says 172.21.15.31:45232) 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 -- 172.21.15.31:0/2019229435 learned_addr learned my addr 172.21.15.31:0/2019229435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f2952d9d640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d55f670 0x55658d935c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f295f4ed740 10 client.0 _ll_register_callbacks cb 0x7f293c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f295259c640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d53ecd0 0x55658d93ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 -- 172.21.15.31:0/2019229435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d53ecd0 msgr2=0x55658d93ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d53ecd0 0x55658d93ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 -- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d55f670 msgr2=0x55658d935c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d55f670 0x55658d935c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 -- 172.21.15.31:0/2019229435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658d9362b0 con 0x55658d557490 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f2952d9d640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d55f670 0x55658d935c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f294bfff640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d93b3d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f293c0a9e90 tx=0x7f293c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4090ceffc3c9dd2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f2939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:25:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f2939ffb640 1 -- 172.21.15.31:0/2019229435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0ad210 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f2939ffb640 1 -- 172.21.15.31:0/2019229435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f293c0aed40 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f294c056070 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f2939ffb640 1 -- 172.21.15.31:0/2019229435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f293c098020 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f295f4ed740 10 client.18186 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.068+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55658d9b03d0 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f2939ffb640 1 -- 172.21.15.31:0/2019229435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294c056070 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f2939ffb640 1 -- 172.21.15.31:0/2019229435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f293c0b3480 con 0x55658d557490 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295f4ed740 10 client.18186 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295f4ed740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295f4ed740 20 client.18186 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295f4ed740 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d580350 0x55658d9b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295f4ed740 4 client.18186 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:25:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55658d89b860 con 0x55658d580350 2024-09-21T06:25:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.069+0000 7f295259c640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d580350 0x55658d9b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.070+0000 7f295259c640 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d580350 0x55658d9b2b90 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616f394f147baa06 server_cookie=36adbba4b53e6815 in_seq=0 out_seq=0 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.070+0000 7f2939ffb640 10 client.18186 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f2939ffb640 1 -- 172.21.15.31:0/2019229435 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55658d89b860 con 0x55658d580350 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f2939ffb640 10 client.18186 handle_command_reply: tid=0 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f295f4ed740 1 client.18186 shutdown 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 msgr2=0x55658d93b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f295f4ed740 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d93b3d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f293c0a9e90 tx=0x7f293c0096a0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 shutdown_connections 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.071+0000 7f295f4ed740 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55658d580350 0x55658d9b2b90 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55658d55f670 0x55658d935c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f2939ffb640 0 client.18186 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55658d557490 0x55658d93b3d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 1 --2- 172.21.15.31:0/2019229435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55658d53ecd0 0x55658d93ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 >> 172.21.15.31:0/2019229435 conn(0x55658d574740 msgr2=0x55658d92a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 shutdown_connections 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 1 -- 172.21.15.31:0/2019229435 wait complete. 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f295f4ed740 20 client.18186 trim_cache size 0 max 16384 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526", 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:25:10.077 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:25:10.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f29400b6680 msgr2=0x7f29400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f2958f79640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f29400b6680 0x7f29400b8b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55658d9260b0 tx=0x7f2944005f90 comp rx=0 tx=0).stop 2024-09-21T06:25:10.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 msgr2=0x7f295419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:10.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.072+0000 7f2958f79640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295419aa10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f293c00e9f0 tx=0x7f293c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 shutdown_connections 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f29400b6680 0x7f29400b8b90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295410de60 0x7f295419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2954105380 0x7f295419aa10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 --2- 172.21.15.31:0/4160153176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2954104a30 0x7f295419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 >> 172.21.15.31:0/4160153176 conn(0x7f29540797b0 msgr2=0x7f2954107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 shutdown_connections 2024-09-21T06:25:10.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:10.073+0000 7f2958f79640 1 -- 172.21.15.31:0/4160153176 wait complete. 2024-09-21T06:25:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 810.0 s, 55.0 tps, lat 477.709 ms stddev 404.426 2024-09-21T06:25:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:10 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:25:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:10 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:10.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:10 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:25:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:10 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:25:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:10 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:10.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:10 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:25:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:10 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:25:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:10 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:10.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:10 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:25:11.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:25:11.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:11 smithi031 ceph-mon[31237]: pgmap v743: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 4.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-09-21T06:25:11.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:11 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:11.416 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:11 smithi110 ceph-mon[34550]: pgmap v743: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 4.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-09-21T06:25:11.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:11 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:11 smithi107 ceph-mon[34380]: pgmap v743: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 4.5 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2024-09-21T06:25:11.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:11 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:11.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 -- 172.21.15.31:0/2339771391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c102ae0 msgr2=0x7ff33c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 --2- 172.21.15.31:0/2339771391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c102ae0 0x7ff33c102ef0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009900 tx=0x7ff32c02a410 comp rx=0 tx=0).stop 2024-09-21T06:25:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 -- 172.21.15.31:0/2339771391 shutdown_connections 2024-09-21T06:25:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 --2- 172.21.15.31:0/2339771391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 0x7ff33c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 --2- 172.21.15.31:0/2339771391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c1083d0 0x7ff33c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 --2- 172.21.15.31:0/2339771391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c102ae0 0x7ff33c102ef0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 -- 172.21.15.31:0/2339771391 >> 172.21.15.31:0/2339771391 conn(0x7ff33c079760 msgr2=0x7ff33c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 -- 172.21.15.31:0/2339771391 shutdown_connections 2024-09-21T06:25:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.922+0000 7ff34198d640 1 -- 172.21.15.31:0/2339771391 wait complete. 2024-09-21T06:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 Processor -- start 2024-09-21T06:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 -- start start 2024-09-21T06:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 0x7ff33c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 0x7ff33c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 0x7ff33c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112990 con 0x7ff33c1083d0 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112810 con 0x7ff33c102ae0 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff34198d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112b10 con 0x7ff33c108d20 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff33affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 0x7ff33c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 0x7ff33c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff33affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 0x7ff33c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43634/0 (socket says 172.21.15.31:43634) 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.923+0000 7ff333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 0x7ff33c199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45262/0 (socket says 172.21.15.31:45262) 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 -- 172.21.15.31:0/3156884320 learned_addr learned my addr 172.21.15.31:0/3156884320 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:11.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 0x7ff33c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 -- 172.21.15.31:0/3156884320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 msgr2=0x7ff33c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 0x7ff33c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 -- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 msgr2=0x7ff33c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 0x7ff33c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 -- 172.21.15.31:0/3156884320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c19f2d0 con 0x7ff33c102ae0 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 0x7ff33c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff333fff640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 0x7ff33c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff33affd640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 0x7ff33c199360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005940 tx=0x7ff32c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff34098b640 1 -- 172.21.15.31:0/3156884320 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c03f030 con 0x7ff33c102ae0 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff34098b640 1 -- 172.21.15.31:0/3156884320 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c0044c0 con 0x7ff33c102ae0 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff34098b640 1 -- 172.21.15.31:0/3156884320 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c035070 con 0x7ff33c102ae0 2024-09-21T06:25:11.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c19f560 con 0x7ff33c102ae0 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.924+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a5d20 con 0x7ff33c102ae0 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 Processor -- start 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 -- start start 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a00cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561fa1976e50 con 0x561fa16301b0 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561fa1966280 con 0x561fa1659070 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.927+0000 7ff347f01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561fa196a090 con 0x561fa1638390 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a00270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a00270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43638/0 (socket says 172.21.15.31:43638) 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff34098b640 1 -- 172.21.15.31:0/3156884320 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff32c042030 con 0x7ff33c102ae0 2024-09-21T06:25:11.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 -- 172.21.15.31:0/3676572164 learned_addr learned my addr 172.21.15.31:0/3676572164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff333fff640 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a00cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33affd640 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 -- 172.21.15.31:0/3676572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 msgr2=0x561fa1a00cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a00cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 -- 172.21.15.31:0/3676572164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 msgr2=0x561fa1a007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 -- 172.21.15.31:0/3676572164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa16c4650 con 0x561fa1659070 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a00270 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff334009fd0 tx=0x7ff3340108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c15fe891fb75ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.928+0000 7ff331ffb640 1 -- 172.21.15.31:0/3676572164 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334011760 con 0x561fa1659070 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.929+0000 7ff331ffb640 1 -- 172.21.15.31:0/3676572164 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff334011db0 con 0x561fa1659070 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.929+0000 7ff331ffb640 1 -- 172.21.15.31:0/3676572164 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33400e3c0 con 0x561fa1659070 2024-09-21T06:25:11.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.929+0000 7ff34098b640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3280b6680 0x7ff3280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.929+0000 7ff33affd640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3280b6680 0x7ff3280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.929+0000 7ff34098b640 1 -- 172.21.15.31:0/3156884320 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7ff32c107b80 con 0x7ff33c102ae0 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 -- 172.21.15.31:0/3676572164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 msgr2=0x561fa1a00270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff33affd640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3280b6680 0x7ff3280b8b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x561fa19feef0 tx=0x7ff32c02a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a00270 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff334009fd0 tx=0x7ff3340108e0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 -- 172.21.15.31:0/3676572164 shutdown_connections 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a00cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 --2- 172.21.15.31:0/3676572164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a00270 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 -- 172.21.15.31:0/3676572164 >> 172.21.15.31:0/3676572164 conn(0x561fa164d460 msgr2=0x561fa19fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 -- 172.21.15.31:0/3676572164 shutdown_connections 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 -- 172.21.15.31:0/3676572164 wait complete. 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.930+0000 7ff347f01740 1 -- start start 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 Processor -- start 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a0bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x561fa17c7700 con 0x561fa1638390 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561fa1901110 con 0x561fa16301b0 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561fa1a10c60 con 0x561fa1659070 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff347f01740 10 client.0 _ll_register_callbacks cb 0x561fa191a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a0bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a0bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45288/0 (socket says 172.21.15.31:45288) 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.931+0000 7ff333fff640 1 -- 172.21.15.31:0/3285841665 learned_addr learned my addr 172.21.15.31:0/3285841665 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff333fff640 1 -- 172.21.15.31:0/3285841665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 msgr2=0x561fa1a19f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff333fff640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a19f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff333fff640 1 -- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 msgr2=0x561fa1a0b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff333fff640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff333fff640 1 -- 172.21.15.31:0/3285841665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa192ec40 con 0x561fa1638390 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff333fff640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a0bf10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff324012d00 tx=0x7ff32400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9073742055ce929c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff31affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff31affd640 1 -- 172.21.15.31:0/3285841665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324009d90 con 0x561fa1638390 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff334011db0 con 0x561fa1638390 2024-09-21T06:25:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.932+0000 7ff31affd640 1 -- 172.21.15.31:0/3285841665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7ff324012110 con 0x561fa1638390 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff31affd640 1 -- 172.21.15.31:0/3285841665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff324004050 con 0x561fa1638390 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff347f01740 10 client.18198 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fa1a1ca30 con 0x561fa1638390 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff31affd640 1 -- 172.21.15.31:0/3285841665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff334011db0 con 0x561fa1638390 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff31affd640 1 -- 172.21.15.31:0/3285841665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff324010430 con 0x561fa1638390 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff347f01740 10 client.18198 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff347f01740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff347f01740 20 client.18198 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.933+0000 7ff347f01740 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa16179f0 0x561fa1a1ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.934+0000 7ff347f01740 4 client.18198 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.934+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fa1978da0 con 0x561fa16179f0 2024-09-21T06:25:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.934+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa16179f0 0x561fa1a1ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.934+0000 7ff31affd640 10 client.18198 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.934+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa16179f0 0x561fa1a1ff40 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06131c818946d83 server_cookie=a0525aa67c33d79f in_seq=0 out_seq=0 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff31affd640 1 -- 172.21.15.31:0/3285841665 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fa1978da0 con 0x561fa16179f0 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff31affd640 10 client.18198 handle_command_reply: tid=0 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 client.18198 shutdown 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 msgr2=0x561fa1a0bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a0bf10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff324012d00 tx=0x7ff32400f790 comp rx=0 tx=0).stop 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 shutdown_connections 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa16179f0 0x561fa1a1ff40 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a0bf10 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 --2- 172.21.15.31:0/3285841665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff31affd640 0 client.18198 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 >> 172.21.15.31:0/3285841665 conn(0x561fa164d460 msgr2=0x561fa1a03450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 shutdown_connections 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.935+0000 7ff347f01740 1 -- 172.21.15.31:0/3285841665 wait complete. 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.936+0000 7ff347f01740 20 client.18198 trim_cache size 0 max 16384 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 Processor -- start 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 -- start start 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a03370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ab590 con 0x561fa1638390 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1addf0 con 0x561fa16301b0 2024-09-21T06:25:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff347f01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1abbe0 con 0x561fa1659070 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.956+0000 7ff333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a03370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff33b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a03370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45298/0 (socket says 172.21.15.31:45298) 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 -- 172.21.15.31:0/3500633715 learned_addr learned my addr 172.21.15.31:0/3500633715 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 -- 172.21.15.31:0/3500633715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 msgr2=0x561fa1a038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 -- 172.21.15.31:0/3500633715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 msgr2=0x561fa1a0be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 -- 172.21.15.31:0/3500633715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fa1a0e500 con 0x561fa1638390 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff333fff640 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a03370 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff324012870 tx=0x7ff324009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe68b91f211d9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff3317fa640 1 -- 172.21.15.31:0/3500633715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324028070 con 0x561fa1638390 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff3317fa640 1 -- 172.21.15.31:0/3500633715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3240121d0 con 0x561fa1638390 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.957+0000 7ff3317fa640 1 -- 172.21.15.31:0/3500633715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324003870 con 0x561fa1638390 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 -- 172.21.15.31:0/3500633715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 msgr2=0x561fa1a03370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a03370 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff324012870 tx=0x7ff324009fa0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 -- 172.21.15.31:0/3500633715 shutdown_connections 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1659070 0x561fa1a038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa1638390 0x561fa1a03370 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 --2- 172.21.15.31:0/3500633715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa16301b0 0x561fa1a0be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 -- 172.21.15.31:0/3500633715 >> 172.21.15.31:0/3500633715 conn(0x561fa164d460 msgr2=0x561fa1a0b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 -- 172.21.15.31:0/3500633715 shutdown_connections 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.958+0000 7ff347f01740 1 -- 172.21.15.31:0/3500633715 wait complete. 2024-09-21T06:25:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 -- start start 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 Processor -- start 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a12a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ac2b0 con 0x561fa16301b0 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ad720 con 0x561fa1659070 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ac980 con 0x561fa1638390 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.959+0000 7ff347f01740 10 client.0 _ll_register_callbacks cb 0x7ff324009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a11f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45302/0 (socket says 172.21.15.31:45302) 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 -- 172.21.15.31:0/186723344 learned_addr learned my addr 172.21.15.31:0/186723344 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff333fff640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a12a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 -- 172.21.15.31:0/186723344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 msgr2=0x561fa1a124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 -- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 msgr2=0x561fa1a12a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a12a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 -- 172.21.15.31:0/186723344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fa1a89a50 con 0x561fa16301b0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff33b7fe640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a11f80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff33401f010 tx=0x7ff33401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be45edaac8d804cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff31affd640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff31affd640 1 -- 172.21.15.31:0/186723344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334026040 con 0x561fa16301b0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff31affd640 1 -- 172.21.15.31:0/186723344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7ff334011760 con 0x561fa16301b0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.960+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3240121d0 con 0x561fa16301b0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff31affd640 1 -- 172.21.15.31:0/186723344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff334044050 con 0x561fa16301b0 2024-09-21T06:25:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 10 client.18204 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561fa1a8ad40 con 0x561fa16301b0 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff31affd640 1 -- 172.21.15.31:0/186723344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3240121d0 con 0x561fa16301b0 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff31affd640 1 -- 172.21.15.31:0/186723344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff33403f030 con 0x561fa16301b0 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 10 client.18204 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 20 client.18204 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa15ecda0 0x561fa1a8d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 4 client.18204 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fa1974850 con 0x561fa15ecda0 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.961+0000 7ff333fff640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa15ecda0 0x561fa1a8d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff31affd640 10 client.18204 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff333fff640 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa15ecda0 0x561fa1a8d2a0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de75c8f093e2dd58 server_cookie=4362e454156e4800 in_seq=0 out_seq=0 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff31affd640 1 -- 172.21.15.31:0/186723344 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9374 (crc 0 0 0) 0x561fa1974850 con 0x561fa15ecda0 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff31affd640 10 client.18204 handle_command_reply: tid=0 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff347f01740 1 client.18204 shutdown 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 msgr2=0x561fa1a11f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff347f01740 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a11f80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff33401f010 tx=0x7ff33401f040 comp rx=0 tx=0).stop 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.962+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 shutdown_connections 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x561fa15ecda0 0x561fa1a8d2a0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561fa1659070 0x561fa1a12a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff31affd640 0 client.18204 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fa1638390 0x561fa1a124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 --2- 172.21.15.31:0/186723344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561fa16301b0 0x561fa1a11f80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 >> 172.21.15.31:0/186723344 conn(0x561fa164d460 msgr2=0x561fa1a012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 shutdown_connections 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 1 -- 172.21.15.31:0/186723344 wait complete. 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff347f01740 20 client.18204 trim_cache size 0 max 16384 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:25:11.998 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526", 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:10.069920+0000", 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:11.999 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.001 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:25:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3280b6680 msgr2=0x7ff3280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3280b6680 0x7ff3280b8b90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x561fa19feef0 tx=0x7ff32c02a410 comp rx=0 tx=0).stop 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 msgr2=0x7ff33c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 0x7ff33c199360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005940 tx=0x7ff32c0040c0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 shutdown_connections 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff33c108d20 0x7ff33c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff3280b6680 0x7ff3280b8b90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff33c1083d0 0x7ff33c199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 --2- 172.21.15.31:0/3156884320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff33c102ae0 0x7ff33c199360 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.963+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 >> 172.21.15.31:0/3156884320 conn(0x7ff33c079760 msgr2=0x7ff33c101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.964+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 shutdown_connections 2024-09-21T06:25:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:11.964+0000 7ff34198d640 1 -- 172.21.15.31:0/3156884320 wait complete. 2024-09-21T06:25:12.605 INFO:tasks.cephfs.filesystem:scrub status for tag:ccf4fa15-85d4-4e94-aab2-fbf0d30d0526 - {'path': '/', 'tag': 'ccf4fa15-85d4-4e94-aab2-fbf0d30d0526', 'options': 'recursive,force'} 2024-09-21T06:25:13.458 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:13 smithi031 ceph-mon[31237]: pgmap v744: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2024-09-21T06:25:13.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:13 smithi110 ceph-mon[34550]: pgmap v744: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2024-09-21T06:25:13.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:13 smithi107 ceph-mon[34380]: pgmap v744: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 4.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 21 MiB/s wr, 1.70k op/s 2024-09-21T06:25:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 815.0 s, 596.5 tps, lat 301.266 ms stddev 1033.357 2024-09-21T06:25:15.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:15 smithi031 ceph-mon[31237]: pgmap v745: 97 pgs: 33 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-09-21T06:25:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:15 smithi110 ceph-mon[34550]: pgmap v745: 97 pgs: 33 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-09-21T06:25:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:15 smithi107 ceph-mon[34380]: pgmap v745: 97 pgs: 33 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 43 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 20 MiB/s wr, 1.65k op/s 2024-09-21T06:25:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 -- 172.21.15.31:0/372877213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4109d50 msgr2=0x7f49f4110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 --2- 172.21.15.31:0/372877213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4109d50 0x7f49f4110580 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009ca0 tx=0x7f49ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 -- 172.21.15.31:0/372877213 shutdown_connections 2024-09-21T06:25:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 --2- 172.21.15.31:0/372877213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4109d50 0x7f49f4110580 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 --2- 172.21.15.31:0/372877213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49f4109400 0x7f49f4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 --2- 172.21.15.31:0/372877213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4103ad0 0x7f49f4103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 -- 172.21.15.31:0/372877213 >> 172.21.15.31:0/372877213 conn(0x7f49f40ff510 msgr2=0x7f49f4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 -- 172.21.15.31:0/372877213 shutdown_connections 2024-09-21T06:25:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.482+0000 7f49fa20e640 1 -- 172.21.15.31:0/372877213 wait complete. 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 Processor -- start 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 -- start start 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 0x7f49f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4109400 0x7f49f4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49f4109d50 0x7f49f4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4113a10 con 0x7f49f4103ad0 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4113890 con 0x7f49f4109d50 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49fa20e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4113b90 con 0x7f49f4109400 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 0x7f49f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 0x7f49f4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45316/0 (socket says 172.21.15.31:45316) 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.483+0000 7f49f37fe640 1 -- 172.21.15.31:0/2387997116 learned_addr learned my addr 172.21.15.31:0/2387997116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4109400 0x7f49f4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f3fff640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49f4109d50 0x7f49f4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f37fe640 1 -- 172.21.15.31:0/2387997116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4109400 msgr2=0x7f49f4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4109400 0x7f49f4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f37fe640 1 -- 172.21.15.31:0/2387997116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49f4109d50 msgr2=0x7f49f4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49f4109d50 0x7f49f4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f37fe640 1 -- 172.21.15.31:0/2387997116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f41a8640 con 0x7f49f4103ad0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4109400 0x7f49f4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 0x7f49f4075710 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9f0 tx=0x7f49e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f0ff9640 1 -- 172.21.15.31:0/2387997116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400ccc0 con 0x7f49f4103ad0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f0ff9640 1 -- 172.21.15.31:0/2387997116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e400ce60 con 0x7f49f4103ad0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f41a8930 con 0x7f49f4103ad0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49f0ff9640 1 -- 172.21.15.31:0/2387997116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4010430 con 0x7f49f4103ad0 2024-09-21T06:25:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.484+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41a8e30 con 0x7f49f4103ad0 2024-09-21T06:25:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.489+0000 7f4a0125a740 1 Processor -- start 2024-09-21T06:25:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.489+0000 7f49f0ff9640 1 -- 172.21.15.31:0/2387997116 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f49e4010610 con 0x7f49f4103ad0 2024-09-21T06:25:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.489+0000 7f4a0125a740 1 -- start start 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814ae2f0 0x555a81885700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a81885c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f4a0125a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555a817f16f0 con 0x555a814ae2f0 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f4a0125a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555a817e0b20 con 0x555a814d71b0 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f4a0125a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555a817e4930 con 0x555a814b64d0 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43690/0 (socket says 172.21.15.31:43690) 2024-09-21T06:25:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f49f3fff640 1 -- 172.21.15.31:0/2817824856 learned_addr learned my addr 172.21.15.31:0/2817824856 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814ae2f0 0x555a81885700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a81885c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.490+0000 7f49f0ff9640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f49dc0b6700 0x7f49dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f37fe640 1 -- 172.21.15.31:0/2817824856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 msgr2=0x555a81885c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a81885c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f37fe640 1 -- 172.21.15.31:0/2817824856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 msgr2=0x555a818851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818851c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f37fe640 1 -- 172.21.15.31:0/2817824856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a817a6530 con 0x555a814ae2f0 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f3fff640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a81885c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f3fff640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f49dc0b6700 0x7f49dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f0ff9640 1 -- 172.21.15.31:0/2387997116 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f49e40e7bd0 con 0x7f49f4103ad0 2024-09-21T06:25:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814ae2f0 0x555a81885700 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f49e40e52b0 tx=0x7f49e40a9a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfc95683779c332 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49d9ffb640 1 -- 172.21.15.31:0/2817824856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40ad780 con 0x555a814ae2f0 2024-09-21T06:25:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49f3fff640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f49dc0b6700 0x7f49dc0b8c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f49ec035000 tx=0x7f49ec0095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.491+0000 7f49d9ffb640 1 -- 172.21.15.31:0/2817824856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40ad4c0 con 0x555a814ae2f0 2024-09-21T06:25:17.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.492+0000 7f49d9ffb640 1 -- 172.21.15.31:0/2817824856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40ad120 con 0x555a814ae2f0 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.492+0000 7f4a0125a740 1 -- 172.21.15.31:0/2817824856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814ae2f0 msgr2=0x555a81885700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.492+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814ae2f0 0x555a81885700 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f49e40e52b0 tx=0x7f49e40a9a10 comp rx=0 tx=0).stop 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 -- 172.21.15.31:0/2817824856 shutdown_connections 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a81885c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814ae2f0 0x555a81885700 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2817824856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 -- 172.21.15.31:0/2817824856 >> 172.21.15.31:0/2817824856 conn(0x555a814cb300 msgr2=0x555a818830b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 -- 172.21.15.31:0/2817824856 shutdown_connections 2024-09-21T06:25:17.498 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 -- 172.21.15.31:0/2817824856 wait complete. 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 -- start start 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.493+0000 7f4a0125a740 1 Processor -- start 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a818907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a81890d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555a81718ab0 con 0x555a814b64d0 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555a81786730 con 0x555a814d71b0 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a818907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a818907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52536/0 (socket says 172.21.15.31:52536) 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f49f37fe640 1 -- 172.21.15.31:0/3641793961 learned_addr learned my addr 172.21.15.31:0/3641793961 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a81890d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555a81893880 con 0x555a814ae2f0 2024-09-21T06:25:17.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f49f3fff640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.494+0000 7f4a0125a740 10 client.0 _ll_register_callbacks cb 0x555a817da6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49f37fe640 1 -- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 msgr2=0x555a818912b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49f37fe640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818912b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49f37fe640 1 -- 172.21.15.31:0/3641793961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 msgr2=0x555a81890d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49f37fe640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a81890d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49f37fe640 1 -- 172.21.15.31:0/3641793961 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a81542790 con 0x555a814ae2f0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49f37fe640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a818907f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f49e40ad6b0 tx=0x7f49e40e57d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb0eb9577b1d1d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49d2ffd640 1 -- 172.21.15.31:0/3641793961 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40ad300 con 0x555a814ae2f0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49d2ffd640 1 -- 172.21.15.31:0/3641793961 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f49e4059530 con 0x555a814ae2f0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e40ad4c0 con 0x555a814ae2f0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49d2ffd640 1 -- 172.21.15.31:0/3641793961 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f49e40ad780 con 0x555a814ae2f0 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f4a0125a740 10 client.28042 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555a815bb620 con 0x555a814ae2f0 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.495+0000 7f49d2ffd640 1 -- 172.21.15.31:0/3641793961 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40ad4c0 con 0x555a814ae2f0 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f49d2ffd640 1 -- 172.21.15.31:0/3641793961 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f49e40aece0 con 0x555a814ae2f0 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f4a0125a740 10 client.28042 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f4a0125a740 10 client.28042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f4a0125a740 20 client.28042 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f4a0125a740 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a81495d80 0x555a818a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f4a0125a740 4 client.28042 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a817f3640 con 0x555a81495d80 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a81495d80 0x555a818a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a81495d80 0x555a818a26a0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999cdc1cabfce07f server_cookie=bca2328306be718c in_seq=0 out_seq=0 2024-09-21T06:25:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.496+0000 7f49d2ffd640 10 client.28042 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f49d2ffd640 1 -- 172.21.15.31:0/3641793961 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a817f3640 con 0x555a81495d80 2024-09-21T06:25:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f49d2ffd640 10 client.28042 handle_command_reply: tid=0 2024-09-21T06:25:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 client.28042 shutdown 2024-09-21T06:25:17.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 msgr2=0x555a818907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a818907f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f49e40ad6b0 tx=0x7f49e40e57d0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 shutdown_connections 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a81495d80 0x555a818a26a0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818912b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a81890d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f49d2ffd640 0 client.28042 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 --2- 172.21.15.31:0/3641793961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a818907f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 >> 172.21.15.31:0/3641793961 conn(0x555a814cb300 msgr2=0x555a818861f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 shutdown_connections 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.497+0000 7f4a0125a740 1 -- 172.21.15.31:0/3641793961 wait complete. 2024-09-21T06:25:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.498+0000 7f4a0125a740 20 client.28042 trim_cache size 0 max 16384 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 Processor -- start 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 -- start start 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a81883010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a818a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ab530 con 0x555a814b64d0 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41add70 con 0x555a814d71b0 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f4a0125a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41abb80 con 0x555a814ae2f0 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a818a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a81883010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.520 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a818a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45338/0 (socket says 172.21.15.31:45338) 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.517+0000 7f49f2ffd640 1 -- 172.21.15.31:0/2677718803 learned_addr learned my addr 172.21.15.31:0/2677718803 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f37fe640 1 -- 172.21.15.31:0/2677718803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 msgr2=0x555a818a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f37fe640 1 -- 172.21.15.31:0/2677718803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 msgr2=0x555a818a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a818a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f37fe640 1 -- 172.21.15.31:0/2677718803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8188ecd0 con 0x555a814ae2f0 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f3fff640 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a818a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49f37fe640 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a81883010 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f49e40aac20 tx=0x7f49e40e5390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ffd1a69a3c7d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49d97fa640 1 -- 172.21.15.31:0/2677718803 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40ad300 con 0x555a814ae2f0 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49d97fa640 1 -- 172.21.15.31:0/2677718803 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40ad4c0 con 0x555a814ae2f0 2024-09-21T06:25:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.518+0000 7f49d97fa640 1 -- 172.21.15.31:0/2677718803 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40ad120 con 0x555a814ae2f0 2024-09-21T06:25:17.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 -- 172.21.15.31:0/2677718803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 msgr2=0x555a81883010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a81883010 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f49e40aac20 tx=0x7f49e40e5390 comp rx=0 tx=0).stop 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 -- 172.21.15.31:0/2677718803 shutdown_connections 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814d71b0 0x555a818a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814b64d0 0x555a818a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 --2- 172.21.15.31:0/2677718803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814ae2f0 0x555a81883010 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 -- 172.21.15.31:0/2677718803 >> 172.21.15.31:0/2677718803 conn(0x555a814cb300 msgr2=0x555a81882900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 -- 172.21.15.31:0/2677718803 shutdown_connections 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 -- 172.21.15.31:0/2677718803 wait complete. 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.519+0000 7f4a0125a740 1 -- start start 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 Processor -- start 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814ae2f0 0x555a818a5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a818a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814d71b0 0x555a81904e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ac230 con 0x555a814d71b0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ad6a0 con 0x555a814ae2f0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41ac900 con 0x555a814b64d0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f4a0125a740 10 client.0 _ll_register_callbacks cb 0x7f49e40eb020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814d71b0 0x555a81904e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a818a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f49f37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814ae2f0 0x555a818a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f49f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814d71b0 0x555a81904e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45352/0 (socket says 172.21.15.31:45352) 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f49f3fff640 1 -- 172.21.15.31:0/1286936383 learned_addr learned my addr 172.21.15.31:0/1286936383 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.520+0000 7f49f2ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a818a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:52546/0 (socket says 172.21.15.31:52546) 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f2ffd640 1 -- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814ae2f0 msgr2=0x555a818a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814ae2f0 0x555a818a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f2ffd640 1 -- 172.21.15.31:0/1286936383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814d71b0 msgr2=0x555a81904e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814d71b0 0x555a81904e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f2ffd640 1 -- 172.21.15.31:0/1286936383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a819054d0 con 0x555a814b64d0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f37fe640 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814ae2f0 0x555a818a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49f2ffd640 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a818a55f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f49e8004700 tx=0x7f49e8010dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b6c3488cc44d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49d2ffd640 1 -- 172.21.15.31:0/1286936383 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8028070 con 0x555a814b64d0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e40ad4c0 con 0x555a814b64d0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49d2ffd640 1 -- 172.21.15.31:0/1286936383 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8543+0+0 (secure 0 0 0) 0x7f49e8024410 con 0x555a814b64d0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f49d2ffd640 1 -- 172.21.15.31:0/1286936383 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f49e8034050 con 0x555a814b64d0 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f4a0125a740 10 client.28048 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:17.526 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.521+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555a8190c8d0 con 0x555a814b64d0 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.523+0000 7f49d2ffd640 1 -- 172.21.15.31:0/1286936383 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40ad4c0 con 0x555a814b64d0 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.523+0000 7f49d2ffd640 1 -- 172.21.15.31:0/1286936383 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f49e800f660 con 0x555a814b64d0 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f4a0125a740 10 client.28048 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f4a0125a740 10 client.28048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f4a0125a740 20 client.28048 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f4a0125a740 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a8146b1c0 0x555a8190f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f4a0125a740 4 client.28048 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:25:17.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a817ef0f0 con 0x555a8146b1c0 2024-09-21T06:25:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.524+0000 7f49f37fe640 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a8146b1c0 0x555a8190f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.525+0000 7f49d2ffd640 10 client.28048 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:17.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.525+0000 7f49f37fe640 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a8146b1c0 0x555a8190f070 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc79147ef07fb83a server_cookie=ac78dcee340064e6 in_seq=0 out_seq=0 2024-09-21T06:25:17.590 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:17 smithi107 ceph-mon[34380]: pgmap v746: 97 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 57 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-09-21T06:25:17.591 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:17 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:25:17.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:17 smithi031 ceph-mon[31237]: pgmap v746: 97 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 57 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-09-21T06:25:17.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:17 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:25:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.658+0000 7f49d2ffd640 1 -- 172.21.15.31:0/1286936383 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a817ef0f0 con 0x555a8146b1c0 2024-09-21T06:25:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.658+0000 7f49d2ffd640 10 client.28048 handle_command_reply: tid=0 2024-09-21T06:25:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.658+0000 7f4a0125a740 1 client.28048 shutdown 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.658+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 msgr2=0x555a818a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.658+0000 7f4a0125a740 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a818a55f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f49e8004700 tx=0x7f49e8010dd0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 shutdown_connections 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555a8146b1c0 0x555a8190f070 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f49d2ffd640 0 client.28048 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a814d71b0 0x555a81904e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.661 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555a814b64d0 0x555a818a55f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 --2- 172.21.15.31:0/1286936383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555a814ae2f0 0x555a818a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 >> 172.21.15.31:0/1286936383 conn(0x555a814cb300 msgr2=0x555a81878ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.659+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 shutdown_connections 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.660+0000 7f4a0125a740 1 -- 172.21.15.31:0/1286936383 wait complete. 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.660+0000 7f4a0125a740 20 client.28048 trim_cache size 0 max 16384 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:25:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:25:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:25:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:25:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f49dc0b6700 msgr2=0x7f49dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f49dc0b6700 0x7f49dc0b8c10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f49ec035000 tx=0x7f49ec0095c0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 msgr2=0x7f49f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 0x7f49f4075710 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9f0 tx=0x7f49e400edf0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 shutdown_connections 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49f4109d50 0x7f49f4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f4109400 0x7f49f4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f49dc0b6700 0x7f49dc0b8c10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 --2- 172.21.15.31:0/2387997116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49f4103ad0 0x7f49f4075710 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 >> 172.21.15.31:0/2387997116 conn(0x7f49f40ff510 msgr2=0x7f49f4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 shutdown_connections 2024-09-21T06:25:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:17.661+0000 7f49fa20e640 1 -- 172.21.15.31:0/2387997116 wait complete. 2024-09-21T06:25:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:17 smithi110 ceph-mon[34550]: pgmap v746: 97 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 57 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 20 MiB/s wr, 1.64k op/s 2024-09-21T06:25:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:17 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:25:19.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:19 smithi031 ceph-mon[31237]: pgmap v747: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-21T06:25:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:19 smithi110 ceph-mon[34550]: pgmap v747: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-21T06:25:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:19 smithi107 ceph-mon[34380]: pgmap v747: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-21T06:25:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 820.0 s, 1416.8 tps, lat 70.859 ms stddev 82.447 2024-09-21T06:25:21.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:21 smithi031 ceph-mon[31237]: pgmap v748: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-09-21T06:25:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:21 smithi110 ceph-mon[34550]: pgmap v748: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-09-21T06:25:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:21 smithi107 ceph-mon[34380]: pgmap v748: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 17 MiB/s wr, 1.40k op/s 2024-09-21T06:25:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:23 smithi031 ceph-mon[31237]: pgmap v749: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 9.8 MiB/s wr, 905 op/s 2024-09-21T06:25:23.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:23 smithi031 ceph-mon[31237]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T06:25:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:23 smithi110 ceph-mon[34550]: pgmap v749: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 9.8 MiB/s wr, 905 op/s 2024-09-21T06:25:23.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:23 smithi110 ceph-mon[34550]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T06:25:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:23 smithi107 ceph-mon[34380]: pgmap v749: 97 pgs: 97 active+clean; 4.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 9.8 MiB/s wr, 905 op/s 2024-09-21T06:25:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:23 smithi107 ceph-mon[34380]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T06:25:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 825.0 s, 1024.2 tps, lat 97.601 ms stddev 128.198 2024-09-21T06:25:25.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:25 smithi031 ceph-mon[31237]: pgmap v751: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 845 op/s 2024-09-21T06:25:25.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:25 smithi110 ceph-mon[34550]: pgmap v751: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 845 op/s 2024-09-21T06:25:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:25 smithi107 ceph-mon[34380]: pgmap v751: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.7 MiB/s wr, 845 op/s 2024-09-21T06:25:27.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:27 smithi031 ceph-mon[31237]: pgmap v752: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 1.02k op/s 2024-09-21T06:25:27.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:27 smithi110 ceph-mon[34550]: pgmap v752: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 1.02k op/s 2024-09-21T06:25:27.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:27 smithi107 ceph-mon[34380]: pgmap v752: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.2 MiB/s wr, 1.02k op/s 2024-09-21T06:25:29.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:29 smithi031 ceph-mon[31237]: pgmap v753: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-09-21T06:25:29.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:29 smithi110 ceph-mon[34550]: pgmap v753: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-09-21T06:25:29.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:29 smithi107 ceph-mon[34380]: pgmap v753: 97 pgs: 97 active+clean; 4.2 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.32k op/s 2024-09-21T06:25:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 830.0 s, 1353.2 tps, lat 73.079 ms stddev 82.948 2024-09-21T06:25:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 -- 172.21.15.31:0/79151918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 msgr2=0x7fe23010a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 --2- 172.21.15.31:0/79151918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe23010a4f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe220009860 tx=0x7fe22002a340 comp rx=0 tx=0).stop 2024-09-21T06:25:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 -- 172.21.15.31:0/79151918 shutdown_connections 2024-09-21T06:25:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 --2- 172.21.15.31:0/79151918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe230110360 0x7fe230116b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 --2- 172.21.15.31:0/79151918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe23010fa10 0x7fe23010fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 --2- 172.21.15.31:0/79151918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe23010a4f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 -- 172.21.15.31:0/79151918 >> 172.21.15.31:0/79151918 conn(0x7fe230079630 msgr2=0x7fe2301081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 -- 172.21.15.31:0/79151918 shutdown_connections 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.804+0000 7fe23680e640 1 -- 172.21.15.31:0/79151918 wait complete. 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.805+0000 7fe23680e640 1 Processor -- start 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.805+0000 7fe23680e640 1 -- start start 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.806+0000 7fe23680e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe2301a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.806+0000 7fe23680e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 0x7fe2301a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.806+0000 7fe23680e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe230110360 0x7fe2301a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.806+0000 7fe23680e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe23011a010 con 0x7fe230110360 2024-09-21T06:25:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.806+0000 7fe23680e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe230119e90 con 0x7fe23010fa10 2024-09-21T06:25:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.806+0000 7fe23680e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe23011a190 con 0x7fe23010a0e0 2024-09-21T06:25:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 0x7fe2301a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe2301a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 0x7fe2301a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44768/0 (socket says 172.21.15.31:44768) 2024-09-21T06:25:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22f7fe640 1 -- 172.21.15.31:0/2922858769 learned_addr learned my addr 172.21.15.31:0/2922858769 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22ffff640 1 -- 172.21.15.31:0/2922858769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 msgr2=0x7fe2301a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22ffff640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 0x7fe2301a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22ffff640 1 -- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe230110360 msgr2=0x7fe2301a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22ffff640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe230110360 0x7fe2301a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22ffff640 1 -- 172.21.15.31:0/2922858769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2301a6960 con 0x7fe23010a0e0 2024-09-21T06:25:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.807+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 0x7fe2301a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.808+0000 7fe22ffff640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe2301a0a30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe220035040 tx=0x7fe220032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.808+0000 7fe22d7fa640 1 -- 172.21.15.31:0/2922858769 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220040020 con 0x7fe23010a0e0 2024-09-21T06:25:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.808+0000 7fe22d7fa640 1 -- 172.21.15.31:0/2922858769 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe220030e10 con 0x7fe23010a0e0 2024-09-21T06:25:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.808+0000 7fe22d7fa640 1 -- 172.21.15.31:0/2922858769 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220037040 con 0x7fe23010a0e0 2024-09-21T06:25:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.809+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2301a6bf0 con 0x7fe23010a0e0 2024-09-21T06:25:30.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.809+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2301a7090 con 0x7fe23010a0e0 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 Processor -- start 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 -- start start 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ae81b0 0x55c824e8d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824e8e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c824e026f0 con 0x55c824ac74d0 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c824df1b20 con 0x55c824abf2f0 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe23d859740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c824df5930 con 0x55c824ae81b0 2024-09-21T06:25:30.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824e8e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe234dab640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ae81b0 0x55c824e8d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824e8e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56290/0 (socket says 172.21.15.31:56290) 2024-09-21T06:25:30.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.812+0000 7fe22f7fe640 1 -- 172.21.15.31:0/409087625 learned_addr learned my addr 172.21.15.31:0/409087625 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe234dab640 1 -- 172.21.15.31:0/409087625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 msgr2=0x55c824e8def0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe234dab640 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8def0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe234dab640 1 -- 172.21.15.31:0/409087625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 msgr2=0x55c824e8e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe234dab640 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824e8e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe234dab640 1 -- 172.21.15.31:0/409087625 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c824db7530 con 0x55c824ae81b0 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe22d7fa640 1 -- 172.21.15.31:0/2922858769 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe220043020 con 0x7fe23010a0e0 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe234dab640 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ae81b0 0x55c824e8d9b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe228009e10 tx=0x7fe22800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57b2d4c5dcbb2e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe2167fc640 1 -- 172.21.15.31:0/409087625 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228004d20 con 0x55c824ae81b0 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.813+0000 7fe2167fc640 1 -- 172.21.15.31:0/409087625 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe22800ee70 con 0x55c824ae81b0 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe22d7fa640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2180b67d0 0x7fe2180b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe2167fc640 1 -- 172.21.15.31:0/409087625 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228005020 con 0x55c824ae81b0 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe22ffff640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2180b67d0 0x7fe2180b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe22d7fa640 1 -- 172.21.15.31:0/2922858769 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fe220107580 con 0x7fe23010a0e0 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe22ffff640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2180b67d0 0x7fe2180b8ce0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55c824e8c630 tx=0x7fe220030fc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe23d859740 1 -- 172.21.15.31:0/409087625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ae81b0 msgr2=0x55c824e8d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.814+0000 7fe23d859740 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ae81b0 0x55c824e8d9b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe228009e10 tx=0x7fe22800e720 comp rx=0 tx=0).stop 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 -- 172.21.15.31:0/409087625 shutdown_connections 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824e8e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 --2- 172.21.15.31:0/409087625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ae81b0 0x55c824e8d9b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 -- 172.21.15.31:0/409087625 >> 172.21.15.31:0/409087625 conn(0x55c824adc300 msgr2=0x55c824e8b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 -- 172.21.15.31:0/409087625 shutdown_connections 2024-09-21T06:25:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 -- 172.21.15.31:0/409087625 wait complete. 2024-09-21T06:25:30.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.815+0000 7fe23d859740 1 -- start start 2024-09-21T06:25:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 Processor -- start 2024-09-21T06:25:30.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824abf2f0 0x55c824e99000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 0x55c824e99540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ae81b0 0x55c824ea5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c824d29ab0 con 0x55c824ae81b0 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c824d97730 con 0x55c824ac74d0 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c824e9c1d0 con 0x55c824abf2f0 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe23d859740 10 client.0 _ll_register_callbacks cb 0x55c824deb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 0x55c824e99540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 0x55c824e99540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44784/0 (socket says 172.21.15.31:44784) 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe22f7fe640 1 -- 172.21.15.31:0/392807593 learned_addr learned my addr 172.21.15.31:0/392807593 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe234dab640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ae81b0 0x55c824ea5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.816+0000 7fe234dab640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824abf2f0 0x55c824e99000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe22f7fe640 1 -- 172.21.15.31:0/392807593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824abf2f0 msgr2=0x55c824e99000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824abf2f0 0x55c824e99000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe22f7fe640 1 -- 172.21.15.31:0/392807593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ae81b0 msgr2=0x55c824ea5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ae81b0 0x55c824ea5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe22f7fe640 1 -- 172.21.15.31:0/392807593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c824b53790 con 0x55c824ac74d0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 0x55c824e99540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe224013830 tx=0x7fe224013cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dffa5a06da731142 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe20f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe20f7fe640 1 -- 172.21.15.31:0/392807593 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224020060 con 0x55c824ac74d0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe22800ee70 con 0x55c824ac74d0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.817+0000 7fe20f7fe640 1 -- 172.21.15.31:0/392807593 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fe224011e30 con 0x55c824ac74d0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.818+0000 7fe20f7fe640 1 -- 172.21.15.31:0/392807593 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe224024400 con 0x55c824ac74d0 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.818+0000 7fe23d859740 10 client.28028 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.818+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c824bcc620 con 0x55c824ac74d0 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.818+0000 7fe20f7fe640 1 -- 172.21.15.31:0/392807593 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe22800ee70 con 0x55c824ac74d0 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.818+0000 7fe20f7fe640 1 -- 172.21.15.31:0/392807593 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe22401a030 con 0x55c824ac74d0 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.819+0000 7fe23d859740 10 client.28028 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.819+0000 7fe23d859740 10 client.28028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.819+0000 7fe23d859740 20 client.28028 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.819+0000 7fe23d859740 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824aa6d80 0x55c824eab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.819+0000 7fe23d859740 4 client.28028 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.819+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c824e04640 con 0x55c824aa6d80 2024-09-21T06:25:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.820+0000 7fe234dab640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824aa6d80 0x55c824eab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.821+0000 7fe20f7fe640 10 client.28028 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:30.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.821+0000 7fe234dab640 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824aa6d80 0x55c824eab370 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91500569f421db9 server_cookie=da36048437f08c1f in_seq=0 out_seq=0 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe20f7fe640 1 -- 172.21.15.31:0/392807593 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c824e04640 con 0x55c824aa6d80 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe20f7fe640 10 client.28028 handle_command_reply: tid=0 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 client.28028 shutdown 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 msgr2=0x55c824e99540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 0x55c824e99540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe224013830 tx=0x7fe224013cf0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 shutdown_connections 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824aa6d80 0x55c824eab370 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe20f7fe640 0 client.28028 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ae81b0 0x55c824ea5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824ac74d0 0x55c824e99540 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 --2- 172.21.15.31:0/392807593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824abf2f0 0x55c824e99000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 >> 172.21.15.31:0/392807593 conn(0x55c824adc300 msgr2=0x55c824e8e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 shutdown_connections 2024-09-21T06:25:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.822+0000 7fe23d859740 1 -- 172.21.15.31:0/392807593 wait complete. 2024-09-21T06:25:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.823+0000 7fe23d859740 20 client.28028 trim_cache size 0 max 16384 2024-09-21T06:25:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.844+0000 7fe23d859740 1 Processor -- start 2024-09-21T06:25:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.844+0000 7fe23d859740 1 -- start start 2024-09-21T06:25:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.844+0000 7fe23d859740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824aa6d80 0x55c824e8f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.844+0000 7fe23d859740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe23d859740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824eb0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe23d859740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301b3c50 con 0x55c824ac74d0 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe23d859740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301b64b0 con 0x55c824abf2f0 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe23d859740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301b42a0 con 0x55c824aa6d80 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44794/0 (socket says 172.21.15.31:44794) 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 -- 172.21.15.31:0/777880012 learned_addr learned my addr 172.21.15.31:0/777880012 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe234dab640 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824aa6d80 0x55c824e8f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe234dab640 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824eb0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 -- 172.21.15.31:0/777880012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824aa6d80 msgr2=0x55c824e8f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824aa6d80 0x55c824e8f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 -- 172.21.15.31:0/777880012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 msgr2=0x55c824eb0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824eb0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 -- 172.21.15.31:0/777880012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c824e99ae0 con 0x55c824abf2f0 2024-09-21T06:25:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8a5b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe224005ce0 tx=0x7fe224011890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f458bfc30bcc0424 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe215ffb640 1 -- 172.21.15.31:0/777880012 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2240108a0 con 0x55c824abf2f0 2024-09-21T06:25:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe215ffb640 1 -- 172.21.15.31:0/777880012 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe224010a40 con 0x55c824abf2f0 2024-09-21T06:25:30.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.845+0000 7fe215ffb640 1 -- 172.21.15.31:0/777880012 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224024a60 con 0x55c824abf2f0 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 -- 172.21.15.31:0/777880012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 msgr2=0x55c824e8a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8a5b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe224005ce0 tx=0x7fe224011890 comp rx=0 tx=0).stop 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 -- 172.21.15.31:0/777880012 shutdown_connections 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824ac74d0 0x55c824eb0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e8a5b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 --2- 172.21.15.31:0/777880012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824aa6d80 0x55c824e8f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 -- 172.21.15.31:0/777880012 >> 172.21.15.31:0/777880012 conn(0x55c824adc300 msgr2=0x55c824e89ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 -- 172.21.15.31:0/777880012 shutdown_connections 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.846+0000 7fe23d859740 1 -- 172.21.15.31:0/777880012 wait complete. 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.847+0000 7fe23d859740 1 -- start start 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.847+0000 7fe23d859740 1 Processor -- start 2024-09-21T06:25:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824aa6d80 0x55c824e9ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e9f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ac74d0 0x55c824f0ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301b4970 con 0x55c824aa6d80 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301b5de0 con 0x55c824abf2f0 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2301b5040 con 0x55c824ac74d0 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe23d859740 10 client.0 _ll_register_callbacks cb 0x7fe2240098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e9f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe234dab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824aa6d80 0x55c824e9ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e9f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:44798/0 (socket says 172.21.15.31:44798) 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 -- 172.21.15.31:0/2782899178 learned_addr learned my addr 172.21.15.31:0/2782899178 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe234dab640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ac74d0 0x55c824f0ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 -- 172.21.15.31:0/2782899178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ac74d0 msgr2=0x55c824f0ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ac74d0 0x55c824f0ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 -- 172.21.15.31:0/2782899178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824aa6d80 msgr2=0x55c824e9ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824aa6d80 0x55c824e9ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe22f7fe640 1 -- 172.21.15.31:0/2782899178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c824ea7e20 con 0x55c824abf2f0 2024-09-21T06:25:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.848+0000 7fe234dab640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ac74d0 0x55c824f0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.851+0000 7fe22f7fe640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e9f4c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe224010820 tx=0x7fe224010f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e07b0201d5bf88a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.852+0000 7fe20f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.852+0000 7fe20f7fe640 1 -- 172.21.15.31:0/2782899178 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224020040 con 0x55c824abf2f0 2024-09-21T06:25:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.852+0000 7fe20f7fe640 1 -- 172.21.15.31:0/2782899178 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fe224005250 con 0x55c824abf2f0 2024-09-21T06:25:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.852+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe224010a40 con 0x55c824abf2f0 2024-09-21T06:25:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.852+0000 7fe20f7fe640 1 -- 172.21.15.31:0/2782899178 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe224005450 con 0x55c824abf2f0 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 10 client.28034 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c824f15690 con 0x55c824abf2f0 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe20f7fe640 1 -- 172.21.15.31:0/2782899178 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe224010a40 con 0x55c824abf2f0 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe20f7fe640 1 -- 172.21.15.31:0/2782899178 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe22401a070 con 0x55c824abf2f0 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 10 client.28034 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 10 client.28034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 20 client.28034 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824ae81b0 0x55c824f17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 4 client.28034 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c824e000f0 con 0x55c824ae81b0 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.853+0000 7fe234dab640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824ae81b0 0x55c824f17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:30.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.854+0000 7fe20f7fe640 10 client.28034 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.854+0000 7fe234dab640 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824ae81b0 0x55c824f17c00 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7938ec3dab59c883 server_cookie=f6025dd352b6281e in_seq=0 out_seq=0 2024-09-21T06:25:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.871+0000 7fe20f7fe640 1 -- 172.21.15.31:0/2782899178 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c824e000f0 con 0x55c824ae81b0 2024-09-21T06:25:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.872+0000 7fe20f7fe640 10 client.28034 handle_command_reply: tid=0 2024-09-21T06:25:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.873+0000 7fe23d859740 1 client.28034 shutdown 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 msgr2=0x55c824e9f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e9f4c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe224010820 tx=0x7fe224010f50 comp rx=0 tx=0).stop 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 shutdown_connections 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c824ae81b0 0x55c824f17c00 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c824ac74d0 0x55c824f0ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c824abf2f0 0x55c824e9f4c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 --2- 172.21.15.31:0/2782899178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c824aa6d80 0x55c824e9ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe20f7fe640 0 client.28034 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 >> 172.21.15.31:0/2782899178 conn(0x55c824adc300 msgr2=0x55c824e8f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 shutdown_connections 2024-09-21T06:25:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.874+0000 7fe23d859740 1 -- 172.21.15.31:0/2782899178 wait complete. 2024-09-21T06:25:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23d859740 20 client.28034 trim_cache size 0 max 16384 2024-09-21T06:25:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:25:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:25:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:25:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2180b67d0 msgr2=0x7fe2180b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2180b67d0 0x7fe2180b8ce0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55c824e8c630 tx=0x7fe220030fc0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 msgr2=0x7fe2301a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe2301a0a30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe220035040 tx=0x7fe220032d50 comp rx=0 tx=0).stop 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 shutdown_connections 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe2180b67d0 0x7fe2180b8ce0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe230110360 0x7fe2301a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23010fa10 0x7fe2301a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 --2- 172.21.15.31:0/2922858769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe23010a0e0 0x7fe2301a0a30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 >> 172.21.15.31:0/2922858769 conn(0x7fe230079630 msgr2=0x7fe23007aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 shutdown_connections 2024-09-21T06:25:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:30.875+0000 7fe23680e640 1 -- 172.21.15.31:0/2922858769 wait complete. 2024-09-21T06:25:31.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:31 smithi031 ceph-mon[31237]: pgmap v754: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T06:25:31.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:31 smithi110 ceph-mon[34550]: pgmap v754: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T06:25:31.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:31 smithi107 ceph-mon[34380]: pgmap v754: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T06:25:33.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:33 smithi031 ceph-mon[31237]: pgmap v755: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T06:25:33.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:33 smithi110 ceph-mon[34550]: pgmap v755: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T06:25:33.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:33 smithi107 ceph-mon[34380]: pgmap v755: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 1.30k op/s 2024-09-21T06:25:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 835.0 s, 1509.2 tps, lat 66.399 ms stddev 75.855 2024-09-21T06:25:35.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:35 smithi031 ceph-mon[31237]: pgmap v756: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-21T06:25:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:35 smithi110 ceph-mon[34550]: pgmap v756: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-21T06:25:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:35 smithi107 ceph-mon[34380]: pgmap v756: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-21T06:25:37.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:37 smithi031 ceph-mon[31237]: pgmap v757: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 862 op/s 2024-09-21T06:25:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:37 smithi110 ceph-mon[34550]: pgmap v757: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 862 op/s 2024-09-21T06:25:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:37 smithi107 ceph-mon[34380]: pgmap v757: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.7 MiB/s wr, 862 op/s 2024-09-21T06:25:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:39 smithi031 ceph-mon[31237]: pgmap v758: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 800 op/s 2024-09-21T06:25:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:39 smithi110 ceph-mon[34550]: pgmap v758: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 800 op/s 2024-09-21T06:25:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:39 smithi107 ceph-mon[34380]: pgmap v758: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 800 op/s 2024-09-21T06:25:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 840.0 s, 1440.6 tps, lat 69.762 ms stddev 84.551 2024-09-21T06:25:41.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:41 smithi110 ceph-mon[34550]: pgmap v759: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-09-21T06:25:41.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:41 smithi107 ceph-mon[34380]: pgmap v759: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-09-21T06:25:41.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:41 smithi031 ceph-mon[31237]: pgmap v759: 97 pgs: 97 active+clean; 4.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.2 MiB/s wr, 590 op/s 2024-09-21T06:25:42.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:25:43.025 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:43.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:43 smithi031 ceph-mon[31237]: pgmap v760: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-09-21T06:25:43.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 -- 172.21.15.31:0/1404202079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5401094c0 msgr2=0x7fd5401098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 --2- 172.21.15.31:0/1404202079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5401094c0 0x7fd5401098d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd534009860 tx=0x7fd53403b350 comp rx=0 tx=0).stop 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 -- 172.21.15.31:0/1404202079 shutdown_connections 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 --2- 172.21.15.31:0/1404202079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 0x7fd540110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 --2- 172.21.15.31:0/1404202079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5401094c0 0x7fd5401098d0 secure :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd534009860 tx=0x7fd53403b350 comp rx=0 tx=0).stop 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 --2- 172.21.15.31:0/1404202079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd540103b80 0x7fd540103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.628+0000 7fd5451df640 1 -- 172.21.15.31:0/1404202079 >> 172.21.15.31:0/1404202079 conn(0x7fd5400ff610 msgr2=0x7fd540101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.629+0000 7fd5451df640 1 -- 172.21.15.31:0/1404202079 shutdown_connections 2024-09-21T06:25:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.629+0000 7fd5451df640 1 -- 172.21.15.31:0/1404202079 wait complete. 2024-09-21T06:25:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.630+0000 7fd5451df640 1 Processor -- start 2024-09-21T06:25:43.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.630+0000 7fd5451df640 1 -- start start 2024-09-21T06:25:43.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.632+0000 7fd5451df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd540103b80 0x7fd540110540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.632+0000 7fd5451df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 0x7fd54010c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.632+0000 7fd5451df640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54010cd20 0x7fd54010d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.632+0000 7fd53e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 0x7fd54010c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.632+0000 7fd53e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 0x7fd54010c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53294/0 (socket says 172.21.15.31:53294) 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd53e59c640 1 -- 172.21.15.31:0/1617065241 learned_addr learned my addr 172.21.15.31:0/1617065241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd54006f620 con 0x7fd540109e10 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd54006f4a0 con 0x7fd540103b80 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd53f59e640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54010cd20 0x7fd54010d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd54006f7a0 con 0x7fd54010cd20 2024-09-21T06:25:43.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd540103b80 0x7fd540110540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd53f59e640 1 -- 172.21.15.31:0/1617065241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd540103b80 msgr2=0x7fd540110540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd53f59e640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd540103b80 0x7fd540110540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.633+0000 7fd53f59e640 1 -- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 msgr2=0x7fd54010c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.634+0000 7fd53f59e640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 0x7fd54010c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.634+0000 7fd53f59e640 1 -- 172.21.15.31:0/1617065241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54019e2d0 con 0x7fd54010cd20 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.634+0000 7fd53f59e640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54010cd20 0x7fd54010d220 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd53800ef00 tx=0x7fd53800c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.634+0000 7fd527fff640 1 -- 172.21.15.31:0/1617065241 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538004b20 con 0x7fd54010cd20 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.634+0000 7fd527fff640 1 -- 172.21.15.31:0/1617065241 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd538003e20 con 0x7fd54010cd20 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.634+0000 7fd527fff640 1 -- 172.21.15.31:0/1617065241 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53800b570 con 0x7fd54010cd20 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.635+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54019e5c0 con 0x7fd54010cd20 2024-09-21T06:25:43.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.635+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54019ea80 con 0x7fd54010cd20 2024-09-21T06:25:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.638+0000 7fd54b753740 1 Processor -- start 2024-09-21T06:25:43.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.640+0000 7fd527fff640 1 -- 172.21.15.31:0/1617065241 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd53801d020 con 0x7fd54010cd20 2024-09-21T06:25:43.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 -- start start 2024-09-21T06:25:43.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87ca4070 0x55af880490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd527fff640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5280b6780 0x7fd5280b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87c7b1b0 0x55af88049630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af8805ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55af87fc1e50 con 0x55af87ca4070 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55af87fb1280 con 0x55af87c83390 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd54b753740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55af87fb5090 con 0x55af87c7b1b0 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.641+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5280b6780 0x7fd5280b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af8805ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87ca4070 0x55af880490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53e59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87c7b1b0 0x55af88049630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af8805ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47962/0 (socket says 172.21.15.31:47962) 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 -- 172.21.15.31:0/3057144119 learned_addr learned my addr 172.21.15.31:0/3057144119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd527fff640 1 -- 172.21.15.31:0/1617065241 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fd5380e7580 con 0x7fd54010cd20 2024-09-21T06:25:43.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 -- 172.21.15.31:0/3057144119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87c7b1b0 msgr2=0x55af88049630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87c7b1b0 0x55af88049630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 -- 172.21.15.31:0/3057144119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87ca4070 msgr2=0x55af880490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87ca4070 0x55af880490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 -- 172.21.15.31:0/3057144119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af87d0f650 con 0x55af87c83390 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53e59c640 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87c7b1b0 0x55af88049630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5280b6780 0x7fd5280b8c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd5401101d0 tx=0x7fd53000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd53f59e640 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af8805ba00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5380e4bc0 tx=0x7fd5380ad150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d76175952703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.642+0000 7fd5257fa640 1 -- 172.21.15.31:0/3057144119 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53801a070 con 0x55af87c83390 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.643+0000 7fd5257fa640 1 -- 172.21.15.31:0/3057144119 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5380ace90 con 0x55af87c83390 2024-09-21T06:25:43.646 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.643+0000 7fd5257fa640 1 -- 172.21.15.31:0/3057144119 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd538013da0 con 0x55af87c83390 2024-09-21T06:25:43.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 -- 172.21.15.31:0/3057144119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 msgr2=0x55af8805ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af8805ba00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5380e4bc0 tx=0x7fd5380ad150 comp rx=0 tx=0).stop 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 -- 172.21.15.31:0/3057144119 shutdown_connections 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af8805ba00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87c7b1b0 0x55af88049630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 --2- 172.21.15.31:0/3057144119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87ca4070 0x55af880490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 -- 172.21.15.31:0/3057144119 >> 172.21.15.31:0/3057144119 conn(0x55af87c98460 msgr2=0x55af8804b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 -- 172.21.15.31:0/3057144119 shutdown_connections 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.645+0000 7fd54b753740 1 -- 172.21.15.31:0/3057144119 wait complete. 2024-09-21T06:25:43.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.646+0000 7fd54b753740 1 -- start start 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.646+0000 7fd54b753740 1 Processor -- start 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.646+0000 7fd54b753740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af8805fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af880602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af880652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55af87e12700 con 0x55af87c83390 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55af87f4c110 con 0x55af87c7b1b0 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55af8805c500 con 0x55af87ca4070 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af8805fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af8805fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47974/0 (socket says 172.21.15.31:47974) 2024-09-21T06:25:43.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 -- 172.21.15.31:0/1550456264 learned_addr learned my addr 172.21.15.31:0/1550456264 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:43.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af880602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 10 client.0 _ll_register_callbacks cb 0x55af87f65990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 -- 172.21.15.31:0/1550456264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 msgr2=0x55af880652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af880652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 -- 172.21.15.31:0/1550456264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 msgr2=0x55af880602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af880602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 -- 172.21.15.31:0/1550456264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af87f79c40 con 0x55af87c7b1b0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5380ace90 con 0x55af87c7b1b0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af8805fcf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd5300116f0 tx=0x7fd530004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c1393ebe26233b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.647+0000 7fd522ff5640 10 client.28046 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd522ff5640 1 -- 172.21.15.31:0/1550456264 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53001c910 con 0x55af87c7b1b0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd522ff5640 1 -- 172.21.15.31:0/1550456264 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fd53001cab0 con 0x55af87c7b1b0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd522ff5640 1 -- 172.21.15.31:0/1550456264 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd53001ccb0 con 0x55af87c7b1b0 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 10 client.28046 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:43.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55af8806e1d0 con 0x55af87c7b1b0 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd522ff5640 1 -- 172.21.15.31:0/1550456264 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd530027480 con 0x55af87c7b1b0 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 10 client.28046 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 10 client.28046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 20 client.28046 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c629f0 0x55af88071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 4 client.28046 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.648+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af87fc3da0 con 0x55af87c629f0 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.649+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c629f0 0x55af88071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.649+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c629f0 0x55af88071680 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4392c7b5a585e7 server_cookie=f1122eb1fd2aa09 in_seq=0 out_seq=0 2024-09-21T06:25:43.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.649+0000 7fd522ff5640 10 client.28046 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:43.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.650+0000 7fd522ff5640 1 -- 172.21.15.31:0/1550456264 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af87fc3da0 con 0x55af87c629f0 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.650+0000 7fd522ff5640 10 client.28046 handle_command_reply: tid=0 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.650+0000 7fd54b753740 1 client.28046 shutdown 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.650+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 msgr2=0x55af8805fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.650+0000 7fd54b753740 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af8805fcf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd5300116f0 tx=0x7fd530004730 comp rx=0 tx=0).stop 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 shutdown_connections 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd54b753740 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c629f0 0x55af88071680 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd54b753740 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af880652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd54b753740 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af880602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd54b753740 1 --2- 172.21.15.31:0/1550456264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af8805fcf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 >> 172.21.15.31:0/1550456264 conn(0x55af87c98460 msgr2=0x55af8804a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.651+0000 7fd522ff5640 0 client.28046 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.652+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 shutdown_connections 2024-09-21T06:25:43.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.652+0000 7fd54b753740 1 -- 172.21.15.31:0/1550456264 wait complete. 2024-09-21T06:25:43.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.652+0000 7fd54b753740 20 client.28046 trim_cache size 0 max 16384 2024-09-21T06:25:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.679+0000 7fd54b753740 1 Processor -- start 2024-09-21T06:25:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.679+0000 7fd54b753740 1 -- start start 2024-09-21T06:25:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd54b753740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af88044280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd54b753740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af8804b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd54b753740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af88049050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd54b753740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401a3210 con 0x55af87c83390 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd54b753740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401a5a70 con 0x55af87c7b1b0 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd54b753740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401a3860 con 0x55af87ca4070 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af8804b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af88044280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af8804b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53318/0 (socket says 172.21.15.31:53318) 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af88044280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:47986/0 (socket says 172.21.15.31:47986) 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53e59c640 1 -- 172.21.15.31:0/2953639495 learned_addr learned my addr 172.21.15.31:0/2953639495 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53f59e640 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af88049050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 -- 172.21.15.31:0/2953639495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 msgr2=0x55af88049050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af88049050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 -- 172.21.15.31:0/2953639495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 msgr2=0x55af8804b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af8804b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 -- 172.21.15.31:0/2953639495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af8804bec0 con 0x55af87c7b1b0 2024-09-21T06:25:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.680+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af88044280 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5300116f0 tx=0x7fd53001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77a7dc7f7c5cbe56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.681+0000 7fd524ff9640 1 -- 172.21.15.31:0/2953639495 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530040070 con 0x55af87c7b1b0 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.681+0000 7fd524ff9640 1 -- 172.21.15.31:0/2953639495 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5380ace90 con 0x55af87c7b1b0 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.681+0000 7fd524ff9640 1 -- 172.21.15.31:0/2953639495 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5300317c0 con 0x55af87c7b1b0 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 -- 172.21.15.31:0/2953639495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 msgr2=0x55af88044280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af88044280 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5300116f0 tx=0x7fd53001f000 comp rx=0 tx=0).stop 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 -- 172.21.15.31:0/2953639495 shutdown_connections 2024-09-21T06:25:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af88049050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c83390 0x55af8804b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 --2- 172.21.15.31:0/2953639495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c7b1b0 0x55af88044280 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.682+0000 7fd54b753740 1 -- 172.21.15.31:0/2953639495 >> 172.21.15.31:0/2953639495 conn(0x55af87c98460 msgr2=0x55af8805b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.683+0000 7fd54b753740 1 -- 172.21.15.31:0/2953639495 shutdown_connections 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.683+0000 7fd54b753740 1 -- 172.21.15.31:0/2953639495 wait complete. 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.684+0000 7fd54b753740 1 -- start start 2024-09-21T06:25:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.684+0000 7fd54b753740 1 Processor -- start 2024-09-21T06:25:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.685+0000 7fd54b753740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 0x55af88066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.685+0000 7fd54b753740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af88066a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd54b753740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af8806b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd54b753740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401a3f30 con 0x55af87c7b1b0 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd54b753740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401a53a0 con 0x55af87c83390 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd54b753740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd5401a4600 con 0x55af87ca4070 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd54b753740 10 client.0 _ll_register_callbacks cb 0x7fd530010f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd53ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 0x55af88066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd53ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 0x55af88066520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53320/0 (socket says 172.21.15.31:53320) 2024-09-21T06:25:43.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.686+0000 7fd53ed9d640 1 -- 172.21.15.31:0/1479564103 learned_addr learned my addr 172.21.15.31:0/1479564103 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af88066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53e59c640 1 -- 172.21.15.31:0/1479564103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 msgr2=0x55af8806b370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af8806b370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53e59c640 1 -- 172.21.15.31:0/1479564103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 msgr2=0x55af88066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 0x55af88066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53e59c640 1 -- 172.21.15.31:0/1479564103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8806b8b0 con 0x55af87c83390 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.688+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 0x55af88066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.689+0000 7fd53e59c640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af88066a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55af8803a9a0 tx=0x7fd5340050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edaf505d6ad0d978 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.689+0000 7fd522ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.689+0000 7fd522ff5640 1 -- 172.21.15.31:0/1479564103 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534000e60 con 0x55af87c83390 2024-09-21T06:25:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.689+0000 7fd522ff5640 1 -- 172.21.15.31:0/1479564103 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fd534004ab0 con 0x55af87c83390 2024-09-21T06:25:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.690+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5380ace90 con 0x55af87c83390 2024-09-21T06:25:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.690+0000 7fd522ff5640 1 -- 172.21.15.31:0/1479564103 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd534048040 con 0x55af87c83390 2024-09-21T06:25:43.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.691+0000 7fd54b753740 10 client.28052 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:43.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.692+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55af880dc720 con 0x55af87c83390 2024-09-21T06:25:43.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.695+0000 7fd522ff5640 1 -- 172.21.15.31:0/1479564103 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5380ace90 con 0x55af87c83390 2024-09-21T06:25:43.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.695+0000 7fd522ff5640 1 -- 172.21.15.31:0/1479564103 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd53404e040 con 0x55af87c83390 2024-09-21T06:25:43.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.698+0000 7fd54b753740 10 client.28052 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:43.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.698+0000 7fd54b753740 10 client.28052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:43.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.698+0000 7fd54b753740 20 client.28052 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:43.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.699+0000 7fd54b753740 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c37da0 0x55af880dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:43.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.699+0000 7fd54b753740 4 client.28052 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:25:43.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.699+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af87fbf850 con 0x55af87c37da0 2024-09-21T06:25:43.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.701+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c37da0 0x55af880dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:43.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.710+0000 7fd53ed9d640 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c37da0 0x55af880dee50 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd23ec4eae866f1 server_cookie=6bc488ca38d09f2d in_seq=0 out_seq=0 2024-09-21T06:25:43.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.710+0000 7fd522ff5640 10 client.28052 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:43.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.710+0000 7fd522ff5640 1 -- 172.21.15.31:0/1479564103 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9178 (crc 0 0 0) 0x55af87fbf850 con 0x55af87c37da0 2024-09-21T06:25:43.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.710+0000 7fd522ff5640 10 client.28052 handle_command_reply: tid=0 2024-09-21T06:25:43.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.710+0000 7fd54b753740 1 client.28052 shutdown 2024-09-21T06:25:43.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.714+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 msgr2=0x55af88066a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.714+0000 7fd54b753740 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af88066a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55af8803a9a0 tx=0x7fd5340050c0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 shutdown_connections 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd54b753740 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55af87c37da0 0x55af880dee50 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd54b753740 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55af87ca4070 0x55af8806b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd54b753740 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55af87c83390 0x55af88066a60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd54b753740 1 --2- 172.21.15.31:0/1479564103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55af87c7b1b0 0x55af88066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 >> 172.21.15.31:0/1479564103 conn(0x55af87c98460 msgr2=0x55af880599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.723+0000 7fd522ff5640 0 client.28052 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.724+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 shutdown_connections 2024-09-21T06:25:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.724+0000 7fd54b753740 1 -- 172.21.15.31:0/1479564103 wait complete. 2024-09-21T06:25:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.725+0000 7fd54b753740 20 client.28052 trim_cache size 0 max 16384 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:10.069920+0000", 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:25:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:25:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:25:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:25:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:25:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:43.751 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.732+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5280b6780 msgr2=0x7fd5280b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.732+0000 7fd5451df640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5280b6780 0x7fd5280b8c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd5401101d0 tx=0x7fd53000b390 comp rx=0 tx=0).stop 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.732+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54010cd20 msgr2=0x7fd54010d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.732+0000 7fd5451df640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54010cd20 0x7fd54010d220 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd53800ef00 tx=0x7fd53800c740 comp rx=0 tx=0).stop 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 shutdown_connections 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd54010cd20 0x7fd54010d220 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd5280b6780 0x7fd5280b8c90 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd5401101d0 tx=0x7fd53000b390 comp rx=0 tx=0).stop 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd540109e10 0x7fd54010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 --2- 172.21.15.31:0/1617065241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd540103b80 0x7fd540110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 >> 172.21.15.31:0/1617065241 conn(0x7fd5400ff610 msgr2=0x7fd540101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.733+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 shutdown_connections 2024-09-21T06:25:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:43.736+0000 7fd5451df640 1 -- 172.21.15.31:0/1617065241 wait complete. 2024-09-21T06:25:43.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:43 smithi110 ceph-mon[34550]: pgmap v760: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-09-21T06:25:43.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:43 smithi107 ceph-mon[34380]: pgmap v760: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-09-21T06:25:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.020+0000 7fccc8c27640 1 -- 172.21.15.31:0/10661469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc41107b0 msgr2=0x7fccc4116fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.020+0000 7fccc8c27640 1 --2- 172.21.15.31:0/10661469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc41107b0 0x7fccc4116fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009860 tx=0x7fccbc02a340 comp rx=0 tx=0).stop 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 -- 172.21.15.31:0/10661469 shutdown_connections 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 --2- 172.21.15.31:0/10661469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc41107b0 0x7fccc4116fe0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 --2- 172.21.15.31:0/10661469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc410fe60 0x7fccc4110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 --2- 172.21.15.31:0/10661469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc410a660 0x7fccc410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 -- 172.21.15.31:0/10661469 >> 172.21.15.31:0/10661469 conn(0x7fccc4071220 msgr2=0x7fccc4073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 -- 172.21.15.31:0/10661469 shutdown_connections 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.021+0000 7fccc8c27640 1 -- 172.21.15.31:0/10661469 wait complete. 2024-09-21T06:25:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.022+0000 7fccc8c27640 1 Processor -- start 2024-09-21T06:25:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.022+0000 7fccc8c27640 1 -- start start 2024-09-21T06:25:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc8c27640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc410a660 0x7fccc41a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc8c27640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 0x7fccc41a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc8c27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc41107b0 0x7fccc41a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc8c27640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fccc411a010 con 0x7fccc41107b0 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc8c27640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4119e90 con 0x7fccc410a660 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 0x7fccc41a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc1d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 0x7fccc41a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60286/0 (socket says 172.21.15.31:60286) 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.023+0000 7fccc1d9b640 1 -- 172.21.15.31:0/2497016971 learned_addr learned my addr 172.21.15.31:0/2497016971 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fccc411a190 con 0x7fccc410fe60 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc41107b0 0x7fccc41a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc259c640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc410a660 0x7fccc41a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc1d9b640 1 -- 172.21.15.31:0/2497016971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc410a660 msgr2=0x7fccc41a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc410a660 0x7fccc41a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc1d9b640 1 -- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc41107b0 msgr2=0x7fccc41a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc41107b0 0x7fccc41a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.024+0000 7fccc1d9b640 1 -- 172.21.15.31:0/2497016971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc41a6920 con 0x7fccc410fe60 2024-09-21T06:25:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc41107b0 0x7fccc41a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 0x7fccc41a0f90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fccb80058c0 tx=0x7fccb800cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccb37fe640 1 -- 172.21.15.31:0/2497016971 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8009c30 con 0x7fccc410fe60 2024-09-21T06:25:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccb37fe640 1 -- 172.21.15.31:0/2497016971 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb8009dd0 con 0x7fccc410fe60 2024-09-21T06:25:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccb37fe640 1 -- 172.21.15.31:0/2497016971 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8015720 con 0x7fccc410fe60 2024-09-21T06:25:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc41a6c10 con 0x7fccc410fe60 2024-09-21T06:25:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.025+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc41ad3a0 con 0x7fccc410fe60 2024-09-21T06:25:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.028+0000 7fcccfc69740 1 Processor -- start 2024-09-21T06:25:44.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 -- start start 2024-09-21T06:25:44.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccb37fe640 1 -- 172.21.15.31:0/2497016971 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fccb8016ab0 con 0x7fccc410fe60 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b00299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 0x5642b0029ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc634d0 0x5642b002a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5642aff9e6f0 con 0x5642afc841b0 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5642aff8db20 con 0x5642afc5b2f0 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 0x5642b0029ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b00299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 0x5642b0029ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48014/0 (socket says 172.21.15.31:48014) 2024-09-21T06:25:44.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b00299b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53350/0 (socket says 172.21.15.31:53350) 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccc2d9d640 1 -- 172.21.15.31:0/79993687 learned_addr learned my addr 172.21.15.31:0/79993687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fcccfc69740 1 -- 172.21.15.31:0/79993687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5642aff91930 con 0x5642afc634d0 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.030+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc634d0 0x5642b002a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccc259c640 1 -- 172.21.15.31:0/79993687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc634d0 msgr2=0x5642b002a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccc259c640 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc634d0 0x5642b002a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccc259c640 1 -- 172.21.15.31:0/79993687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 msgr2=0x5642b00299b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccc259c640 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b00299b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccc259c640 1 -- 172.21.15.31:0/79993687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642aff53530 con 0x5642afc5b2f0 2024-09-21T06:25:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccc259c640 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 0x5642b0029ef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009fd0 tx=0x7fccb4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e617001447b3ffee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.031+0000 7fccb0ff9640 1 -- 172.21.15.31:0/79993687 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4003cf0 con 0x5642afc5b2f0 2024-09-21T06:25:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fccb0ff9640 1 -- 172.21.15.31:0/79993687 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb401d070 con 0x5642afc5b2f0 2024-09-21T06:25:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fccb0ff9640 1 -- 172.21.15.31:0/79993687 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb40188b0 con 0x5642afc5b2f0 2024-09-21T06:25:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fccb37fe640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fccac0b6680 0x7fccac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fccac0b6680 0x7fccac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fccb37fe640 1 -- 172.21.15.31:0/2497016971 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fccb80e7730 con 0x7fccc410fe60 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fcccfc69740 1 -- 172.21.15.31:0/79993687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 msgr2=0x5642b0029ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.032+0000 7fcccfc69740 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 0x5642b0029ef0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009fd0 tx=0x7fccb4009500 comp rx=0 tx=0).stop 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 -- 172.21.15.31:0/79993687 shutdown_connections 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc634d0 0x5642b002a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc5b2f0 0x5642b0029ef0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 --2- 172.21.15.31:0/79993687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b00299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 -- 172.21.15.31:0/79993687 >> 172.21.15.31:0/79993687 conn(0x5642afc78300 msgr2=0x5642b00278a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 -- 172.21.15.31:0/79993687 shutdown_connections 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 -- 172.21.15.31:0/79993687 wait complete. 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fccac0b6680 0x7fccac0b8b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0095f0 tx=0x7fccbc02c040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 -- start start 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.033+0000 7fcccfc69740 1 Processor -- start 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0035090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b00355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b0041450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fccc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b0041450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fccc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b0041450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53362/0 (socket says 172.21.15.31:53362) 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fccc1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b00355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fccc2d9d640 1 -- 172.21.15.31:0/3900548537 learned_addr learned my addr 172.21.15.31:0/3900548537 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x5642afec5ab0 con 0x5642afc841b0 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5642aff33730 con 0x5642afc634d0 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5642b0038130 con 0x5642afc5b2f0 2024-09-21T06:25:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.034+0000 7fccc259c640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0035090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.035+0000 7fcccfc69740 10 client.0 _ll_register_callbacks cb 0x5642aff876b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.035+0000 7fccc259c640 1 -- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 msgr2=0x5642b00355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.035+0000 7fccc259c640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b00355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.035+0000 7fccc259c640 1 -- 172.21.15.31:0/3900548537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 msgr2=0x5642b0041450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.035+0000 7fccc259c640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b0041450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.035+0000 7fccc259c640 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642afcef790 con 0x5642afc5b2f0 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fccc259c640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0035090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fccb4000f80 tx=0x7fccb4004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55082549bf710336 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fcca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fcca9ffb640 1 -- 172.21.15.31:0/3900548537 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4025020 con 0x5642afc5b2f0 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fcca9ffb640 1 -- 172.21.15.31:0/3900548537 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fccb4004a40 con 0x5642afc5b2f0 2024-09-21T06:25:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb401d070 con 0x5642afc5b2f0 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b0041450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b00355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.036+0000 7fcca9ffb640 1 -- 172.21.15.31:0/3900548537 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fccb4030050 con 0x5642afc5b2f0 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.037+0000 7fcccfc69740 10 client.28090 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.037+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5642afd68620 con 0x5642afc5b2f0 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.037+0000 7fcca9ffb640 1 -- 172.21.15.31:0/3900548537 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb401d070 con 0x5642afc5b2f0 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.037+0000 7fcca9ffb640 1 -- 172.21.15.31:0/3900548537 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fccb4018c60 con 0x5642afc5b2f0 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.037+0000 7fcccfc69740 10 client.28090 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.037+0000 7fcccfc69740 10 client.28090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.038+0000 7fcccfc69740 20 client.28090 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.038+0000 7fcccfc69740 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc42d80 0x5642b0047120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.038+0000 7fcccfc69740 4 client.28090 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.038+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642affa0640 con 0x5642afc42d80 2024-09-21T06:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.038+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc42d80 0x5642b0047120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.039+0000 7fcca9ffb640 10 client.28090 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.039+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc42d80 0x5642b0047120 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d23c893dbfc6862 server_cookie=f11ee66d82dc6d9e in_seq=0 out_seq=0 2024-09-21T06:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcca9ffb640 1 -- 172.21.15.31:0/3900548537 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642affa0640 con 0x5642afc42d80 2024-09-21T06:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcca9ffb640 10 client.28090 handle_command_reply: tid=0 2024-09-21T06:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 client.28090 shutdown 2024-09-21T06:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 msgr2=0x5642b0035090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0035090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fccb4000f80 tx=0x7fccb4004280 comp rx=0 tx=0).stop 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 shutdown_connections 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc42d80 0x5642b0047120 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc841b0 0x5642b0041450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b00355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcca9ffb640 0 client.28090 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 --2- 172.21.15.31:0/3900548537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0035090 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 >> 172.21.15.31:0/3900548537 conn(0x5642afc78300 msgr2=0x5642b002a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 shutdown_connections 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 1 -- 172.21.15.31:0/3900548537 wait complete. 2024-09-21T06:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.040+0000 7fcccfc69740 20 client.28090 trim_cache size 0 max 16384 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.062+0000 7fcccfc69740 1 Processor -- start 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.062+0000 7fcccfc69740 1 -- start start 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc42d80 0x5642b002b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 0x5642b0035570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b0036cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fcccfc69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b8220 con 0x5642afc5b2f0 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fcccfc69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41baa80 con 0x5642afc634d0 2024-09-21T06:25:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fcccfc69740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b8870 con 0x5642afc42d80 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 0x5642b0035570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 0x5642b0035570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53376/0 (socket says 172.21.15.31:53376) 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc42d80 0x5642b002b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 -- 172.21.15.31:0/1654495536 learned_addr learned my addr 172.21.15.31:0/1654495536 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b0036cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 -- 172.21.15.31:0/1654495536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc42d80 msgr2=0x5642b002b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc42d80 0x5642b002b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 -- 172.21.15.31:0/1654495536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 msgr2=0x5642b0036cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b0036cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.063+0000 7fccc1d9b640 1 -- 172.21.15.31:0/1654495536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b0038010 con 0x5642afc5b2f0 2024-09-21T06:25:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.065+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 0x5642b0035570 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fccb80aa800 tx=0x7fccb800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4623d51b1c69b959 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.065+0000 7fccabfff640 1 -- 172.21.15.31:0/1654495536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80b2070 con 0x5642afc5b2f0 2024-09-21T06:25:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.065+0000 7fccabfff640 1 -- 172.21.15.31:0/1654495536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb80b6fb0 con 0x5642afc5b2f0 2024-09-21T06:25:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.065+0000 7fccabfff640 1 -- 172.21.15.31:0/1654495536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb80b6040 con 0x5642afc5b2f0 2024-09-21T06:25:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 -- 172.21.15.31:0/1654495536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 msgr2=0x5642b0035570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 0x5642b0035570 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fccb80aa800 tx=0x7fccb800c040 comp rx=0 tx=0).stop 2024-09-21T06:25:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 -- 172.21.15.31:0/1654495536 shutdown_connections 2024-09-21T06:25:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc634d0 0x5642b0036cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc5b2f0 0x5642b0035570 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 --2- 172.21.15.31:0/1654495536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc42d80 0x5642b002b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.066+0000 7fcccfc69740 1 -- 172.21.15.31:0/1654495536 >> 172.21.15.31:0/1654495536 conn(0x5642afc78300 msgr2=0x5642b00365b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.067+0000 7fcccfc69740 1 -- 172.21.15.31:0/1654495536 shutdown_connections 2024-09-21T06:25:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.067+0000 7fcccfc69740 1 -- 172.21.15.31:0/1654495536 wait complete. 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.067+0000 7fcccfc69740 1 -- start start 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.067+0000 7fcccfc69740 1 Processor -- start 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.067+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc42d80 0x5642b0049870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0049db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcccfc69740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 0x5642b00af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcccfc69740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b8f40 con 0x5642afc634d0 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcccfc69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41ba3b0 con 0x5642afc42d80 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcccfc69740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fccc41b9610 con 0x5642afc5b2f0 2024-09-21T06:25:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 0x5642b00af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc42d80 0x5642b0049870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 0x5642b00af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53380/0 (socket says 172.21.15.31:53380) 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 -- 172.21.15.31:0/510812598 learned_addr learned my addr 172.21.15.31:0/510812598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc1d9b640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0049db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcccfc69740 10 client.0 _ll_register_callbacks cb 0x7fccb8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 -- 172.21.15.31:0/510812598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 msgr2=0x5642b0049db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0049db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 -- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc42d80 msgr2=0x5642b0049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc42d80 0x5642b0049870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 -- 172.21.15.31:0/510812598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b00afde0 con 0x5642afc634d0 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc259c640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc42d80 0x5642b0049870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fccc2d9d640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 0x5642b00af8a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0057c0 tx=0x7fccbc02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5818ac8a176ddeda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:25:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.068+0000 7fcca9ffb640 1 -- 172.21.15.31:0/510812598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc046040 con 0x5642afc634d0 2024-09-21T06:25:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.069+0000 7fcca9ffb640 1 -- 172.21.15.31:0/510812598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fccbc0327f0 con 0x5642afc634d0 2024-09-21T06:25:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.069+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb80b6fb0 con 0x5642afc634d0 2024-09-21T06:25:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.069+0000 7fcca9ffb640 1 -- 172.21.15.31:0/510812598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fccbc037040 con 0x5642afc634d0 2024-09-21T06:25:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.069+0000 7fcccfc69740 10 client.18273 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.069+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5642b00b1030 con 0x5642afc634d0 2024-09-21T06:25:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcca9ffb640 1 -- 172.21.15.31:0/510812598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccb80b6fb0 con 0x5642afc634d0 2024-09-21T06:25:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcca9ffb640 1 -- 172.21.15.31:0/510812598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fccbc003c80 con 0x5642afc634d0 2024-09-21T06:25:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcccfc69740 10 client.18273 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcccfc69740 10 client.18273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcccfc69740 20 client.18273 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcccfc69740 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc841b0 0x5642b00b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcccfc69740 4 client.18273 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.070+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642b0027b10 con 0x5642afc841b0 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.071+0000 7fccc259c640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc841b0 0x5642b00b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.072+0000 7fccc259c640 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc841b0 0x5642b00b3870 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e1bcced4deec6c server_cookie=f86cc812f9d75f73 in_seq=0 out_seq=0 2024-09-21T06:25:44.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:44.072+0000 7fcca9ffb640 10 client.18273 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:44.672 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:25:44.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:25:45.084 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:45.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 845.0 s, 2115.3 tps, lat 47.678 ms stddev 57.956 2024-09-21T06:25:45.538 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:45 smithi031 ceph-mon[31237]: pgmap v761: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.1 MiB/s wr, 644 op/s 2024-09-21T06:25:45.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.682+0000 7f925baf0640 1 -- 172.21.15.31:0/1064496448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 msgr2=0x7f925410fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:45.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.682+0000 7f925baf0640 1 --2- 172.21.15.31:0/1064496448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 0x7f925410fee0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9248009860 tx=0x7f924802a360 comp rx=0 tx=0).stop 2024-09-21T06:25:45.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.690+0000 7f925baf0640 1 -- 172.21.15.31:0/1064496448 shutdown_connections 2024-09-21T06:25:45.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.690+0000 7f925baf0640 1 --2- 172.21.15.31:0/1064496448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254110420 0x7f9254116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:45.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.690+0000 7f925baf0640 1 --2- 172.21.15.31:0/1064496448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 0x7f925410fee0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:45.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.690+0000 7f925baf0640 1 --2- 172.21.15.31:0/1064496448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f925410a190 0x7f925410a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:45.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.690+0000 7f925baf0640 1 -- 172.21.15.31:0/1064496448 >> 172.21.15.31:0/1064496448 conn(0x7f92540780f0 msgr2=0x7f92541081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:45.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.692+0000 7f925baf0640 1 -- 172.21.15.31:0/1064496448 shutdown_connections 2024-09-21T06:25:45.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.692+0000 7f925baf0640 1 -- 172.21.15.31:0/1064496448 wait complete. 2024-09-21T06:25:45.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.693+0000 7f925baf0640 1 Processor -- start 2024-09-21T06:25:45.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.693+0000 7f925baf0640 1 -- start start 2024-09-21T06:25:45.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.694+0000 7f925baf0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f925410a190 0x7f92541a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:45.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.694+0000 7f925baf0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 0x7f92541a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:45.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.694+0000 7f925baf0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9254110420 0x7f92541a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:45.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.694+0000 7f925baf0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f925411a0e0 con 0x7f925410a190 2024-09-21T06:25:45.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.694+0000 7f925baf0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9254119f60 con 0x7f9254110420 2024-09-21T06:25:45.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.694+0000 7f925baf0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f925411a260 con 0x7f925410fad0 2024-09-21T06:25:45.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925988c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f925410a190 0x7f92541a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:45.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925988c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f925410a190 0x7f92541a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53394/0 (socket says 172.21.15.31:53394) 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925988c640 1 -- 172.21.15.31:0/2646620273 learned_addr learned my addr 172.21.15.31:0/2646620273 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925a08d640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9254110420 0x7f92541a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925908b640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 0x7f92541a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925a08d640 1 -- 172.21.15.31:0/2646620273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 msgr2=0x7f92541a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925a08d640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 0x7f92541a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925a08d640 1 -- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f925410a190 msgr2=0x7f92541a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925a08d640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f925410a190 0x7f92541a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.698+0000 7f925a08d640 1 -- 172.21.15.31:0/2646620273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92541a6a50 con 0x7f9254110420 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.699+0000 7f925a08d640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9254110420 0x7f92541a6350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f924c00d680 tx=0x7f924c00db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.699+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c028070 con 0x7f9254110420 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.699+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92541a6d40 con 0x7f9254110420 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.700+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925406cde0 con 0x7f9254110420 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.701+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924c009820 con 0x7f9254110420 2024-09-21T06:25:45.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.701+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c004c40 con 0x7f9254110420 2024-09-21T06:25:45.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.702+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925410b410 con 0x7f9254110420 2024-09-21T06:25:45.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.704+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f924c021470 con 0x7f9254110420 2024-09-21T06:25:45.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.706+0000 7f9252ffd640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f923c0b67e0 0x7f923c0b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:45.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.706+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f924c0f8ec0 con 0x7f9254110420 2024-09-21T06:25:45.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.708+0000 7f925988c640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f923c0b67e0 0x7f923c0b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:45.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.710+0000 7f925988c640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f923c0b67e0 0x7f923c0b8cf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9244009fd0 tx=0x7f9244010040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:45.714+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f924c0c37a0 con 0x7f9254110420 2024-09-21T06:25:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:45 smithi110 ceph-mon[34550]: pgmap v761: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.1 MiB/s wr, 644 op/s 2024-09-21T06:25:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:45 smithi107 ceph-mon[34380]: pgmap v761: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.1 MiB/s wr, 644 op/s 2024-09-21T06:25:46.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.080+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92541121d0 con 0x7f9254110420 2024-09-21T06:25:46.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.085+0000 7f9252ffd640 1 -- 172.21.15.31:0/2646620273 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f924c0c86e0 con 0x7f9254110420 2024-09-21T06:25:46.088 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:25:46.088 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:25:46.088 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:25:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f923c0b67e0 msgr2=0x7f923c0b8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f923c0b67e0 0x7f923c0b8cf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9244009fd0 tx=0x7f9244010040 comp rx=0 tx=0).stop 2024-09-21T06:25:46.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9254110420 msgr2=0x7f92541a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9254110420 0x7f92541a6350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f924c00d680 tx=0x7f924c00db40 comp rx=0 tx=0).stop 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 shutdown_connections 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9254110420 0x7f92541a6350 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f925410fad0 0x7f92541a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f923c0b67e0 0x7f923c0b8cf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.089+0000 7f925baf0640 1 --2- 172.21.15.31:0/2646620273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f925410a190 0x7f92541a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.090+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 >> 172.21.15.31:0/2646620273 conn(0x7f92540780f0 msgr2=0x7f92541081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.090+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 shutdown_connections 2024-09-21T06:25:46.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:46.092+0000 7f925baf0640 1 -- 172.21.15.31:0/2646620273 wait complete. 2024-09-21T06:25:46.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:46 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2646620273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:46 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2646620273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:46 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2646620273' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:25:47.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:25:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:47.679 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:47 smithi110 ceph-mon[34550]: pgmap v762: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-21T06:25:47.688 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:47 smithi031 ceph-mon[31237]: pgmap v762: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-21T06:25:47.689 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:47 smithi107 ceph-mon[34380]: pgmap v762: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.3 MiB/s wr, 562 op/s 2024-09-21T06:25:48.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2197362145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810b6d0 msgr2=0x7fd8f810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2197362145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810b6d0 0x7fd8f810bae0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009900 tx=0x7fd8ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2197362145 shutdown_connections 2024-09-21T06:25:48.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2197362145 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810c020 0x7fd8f8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2197362145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810b6d0 0x7fd8f810bae0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2197362145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 0x7fd8f81061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2197362145 >> 172.21.15.31:0/2197362145 conn(0x7fd8f81017c0 msgr2=0x7fd8f8103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:48.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.142+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2197362145 shutdown_connections 2024-09-21T06:25:48.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.143+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2197362145 wait complete. 2024-09-21T06:25:48.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.143+0000 7fd8fd5b9640 1 Processor -- start 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.143+0000 7fd8fd5b9640 1 -- start start 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8fd5b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 0x7fd8f819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8fd5b9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810b6d0 0x7fd8f819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8fd5b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810c020 0x7fd8f81a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8fd5b9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8115ce0 con 0x7fd8f8105d90 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8fd5b9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8115b60 con 0x7fd8f810c020 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8fd5b9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f8115e60 con 0x7fd8f810b6d0 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8f67fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810b6d0 0x7fd8f819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8f6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 0x7fd8f819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8f67fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810b6d0 0x7fd8f819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:60378/0 (socket says 172.21.15.31:60378) 2024-09-21T06:25:48.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8f67fc640 1 -- 172.21.15.31:0/2663386059 learned_addr learned my addr 172.21.15.31:0/2663386059 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:48.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.144+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810c020 0x7fd8f81a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.145+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/2663386059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810b6d0 msgr2=0x7fd8f819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.145+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810b6d0 0x7fd8f819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.145+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/2663386059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810c020 msgr2=0x7fd8f81a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.145+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810c020 0x7fd8f81a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.145+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/2663386059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f81a2680 con 0x7fd8f8105d90 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.146+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 0x7fd8f819c710 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d6d0 tx=0x7fd8e800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.146+0000 7fd8dffff640 1 -- 172.21.15.31:0/2663386059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8017070 con 0x7fd8f8105d90 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.146+0000 7fd8dffff640 1 -- 172.21.15.31:0/2663386059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e8009820 con 0x7fd8f8105d90 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.146+0000 7fd8dffff640 1 -- 172.21.15.31:0/2663386059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8004c20 con 0x7fd8f8105d90 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.146+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f81a2970 con 0x7fd8f8105d90 2024-09-21T06:25:48.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.146+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f81a90d0 con 0x7fd8f8105d90 2024-09-21T06:25:48.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.150+0000 7fd903b2d740 1 Processor -- start 2024-09-21T06:25:48.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.150+0000 7fd903b2d740 1 -- start start 2024-09-21T06:25:48.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.150+0000 7fd8dffff640 1 -- 172.21.15.31:0/2663386059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fd8e80099d0 con 0x7fd8f8105d90 2024-09-21T06:25:48.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca11b0 0x56333e071800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e0712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53422/0 (socket says 172.21.15.31:53422) 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8f77fe640 1 -- 172.21.15.31:0/2026716435 learned_addr learned my addr 172.21.15.31:0/2026716435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca9390 0x56333e071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8dffff640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8e00b6700 0x7fd8e00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56333dfe7ea0 con 0x56333dcca070 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56333dfd72d0 con 0x56333dca11b0 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56333dfdb0e0 con 0x56333dca9390 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca9390 0x56333e071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8dffff640 1 -- 172.21.15.31:0/2663386059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fd8e80e6f10 con 0x7fd8f8105d90 2024-09-21T06:25:48.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.151+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca11b0 0x56333e071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.152+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8e00b6700 0x7fd8e00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.152+0000 7fd8f77fe640 1 -- 172.21.15.31:0/2026716435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca9390 msgr2=0x56333e071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.152+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca9390 0x56333e071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.152+0000 7fd8f77fe640 1 -- 172.21.15.31:0/2026716435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca11b0 msgr2=0x56333e071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.152+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca11b0 0x56333e071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.152+0000 7fd8f77fe640 1 -- 172.21.15.31:0/2026716435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333dd35650 con 0x56333dcca070 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.153+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca11b0 0x56333e071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.153+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8e00b6700 0x7fd8e00b8c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000fc90 tx=0x7fd8f001f000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.153+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e0712c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0005c40 tx=0x7fd8f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3cde5d8362d241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.154+0000 7fd8dd7fa640 1 -- 172.21.15.31:0/2026716435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0013950 con 0x56333dcca070 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.154+0000 7fd8dd7fa640 1 -- 172.21.15.31:0/2026716435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f0013170 con 0x56333dcca070 2024-09-21T06:25:48.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.154+0000 7fd8dd7fa640 1 -- 172.21.15.31:0/2026716435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0005280 con 0x56333dcca070 2024-09-21T06:25:48.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.155+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 msgr2=0x56333e0712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.155+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e0712c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0005c40 tx=0x7fd8f000f790 comp rx=0 tx=0).stop 2024-09-21T06:25:48.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 shutdown_connections 2024-09-21T06:25:48.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca9390 0x56333e071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca11b0 0x56333e071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2026716435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e0712c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 >> 172.21.15.31:0/2026716435 conn(0x56333dcbe460 msgr2=0x56333e06f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:48.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 shutdown_connections 2024-09-21T06:25:48.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.156+0000 7fd903b2d740 1 -- 172.21.15.31:0/2026716435 wait complete. 2024-09-21T06:25:48.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd903b2d740 1 -- start start 2024-09-21T06:25:48.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd903b2d740 1 Processor -- start 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e07caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e07cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e088d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd903b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56333de38700 con 0x56333dcca070 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd8f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e07cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd8f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e07cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48080/0 (socket says 172.21.15.31:48080) 2024-09-21T06:25:48.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd8f67fc640 1 -- 172.21.15.31:0/714064487 learned_addr learned my addr 172.21.15.31:0/714064487 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.157+0000 7fd8f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e07caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56333df72160 con 0x56333dca9390 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56333e07fa40 con 0x56333dca11b0 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e088d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd903b2d740 10 client.0 _ll_register_callbacks cb 0x56333df8b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f67fc640 1 -- 172.21.15.31:0/714064487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 msgr2=0x56333e07caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e07caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f67fc640 1 -- 172.21.15.31:0/714064487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 msgr2=0x56333e088d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e088d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f67fc640 1 -- 172.21.15.31:0/714064487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333df9fc90 con 0x56333dca9390 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8f0013170 con 0x56333dca9390 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e07cfe0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec006010 tx=0x7fd8ec031e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6334e567beaff46c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8daff5640 10 client.28082 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:48.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.158+0000 7fd8daff5640 1 -- 172.21.15.31:0/714064487 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec035040 con 0x56333dca9390 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.159+0000 7fd8daff5640 1 -- 172.21.15.31:0/714064487 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fd8ec0316a0 con 0x56333dca9390 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.159+0000 7fd8daff5640 1 -- 172.21.15.31:0/714064487 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd8ec0318e0 con 0x56333dca9390 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.159+0000 7fd903b2d740 10 client.28082 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.159+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=21}) -- 0x56333e08b5d0 con 0x56333dca9390 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd8daff5640 1 -- 172.21.15.31:0/714064487 <== mon.2 v2:172.21.15.107:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd8ec004040 con 0x56333dca9390 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd903b2d740 10 client.28082 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd903b2d740 10 client.28082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd903b2d740 20 client.28082 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd903b2d740 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dc889f0 0x56333e08eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd903b2d740 4 client.28082 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:48.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.160+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333dfe9df0 con 0x56333dc889f0 2024-09-21T06:25:48.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.161+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dc889f0 0x56333e08eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.161+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dc889f0 0x56333e08eab0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c532e0d689c61c45 server_cookie=d7f1985a0ccf8602 in_seq=0 out_seq=0 2024-09-21T06:25:48.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.161+0000 7fd8daff5640 10 client.28082 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.162+0000 7fd8daff5640 1 -- 172.21.15.31:0/714064487 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56333dfe9df0 con 0x56333dc889f0 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.162+0000 7fd8daff5640 10 client.28082 handle_command_reply: tid=0 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.162+0000 7fd903b2d740 1 client.28082 shutdown 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 msgr2=0x56333e07cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e07cfe0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec006010 tx=0x7fd8ec031e80 comp rx=0 tx=0).stop 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 shutdown_connections 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dc889f0 0x56333e08eab0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dcca070 0x56333e088d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd8daff5640 0 client.28082 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e07cfe0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 --2- 172.21.15.31:0/714064487 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e07caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 >> 172.21.15.31:0/714064487 conn(0x56333dcbe460 msgr2=0x56333e0744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 shutdown_connections 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 1 -- 172.21.15.31:0/714064487 wait complete. 2024-09-21T06:25:48.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.163+0000 7fd903b2d740 20 client.28082 trim_cache size 0 max 16384 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.181+0000 7fd903b2d740 1 Processor -- start 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.181+0000 7fd903b2d740 1 -- start start 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 0x56333e07e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e072a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e088c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd903b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81afa80 con 0x56333dc889f0 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd903b2d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81b22e0 con 0x56333dca9390 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd903b2d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81b00d0 con 0x56333dca11b0 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 0x56333e07e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e088c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e088c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:48090/0 (socket says 172.21.15.31:48090) 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 0x56333e07e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53438/0 (socket says 172.21.15.31:53438) 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f77fe640 1 -- 172.21.15.31:0/1600853399 learned_addr learned my addr 172.21.15.31:0/1600853399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/1600853399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 msgr2=0x56333e072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e072a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/1600853399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 msgr2=0x56333e088c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e088c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/1600853399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333e0737e0 con 0x56333dc889f0 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.182+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e088c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.183+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 0x56333e07e980 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80e4670 tx=0x7fd8e801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99b5cc1e76385c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.183+0000 7fd8dcff9640 1 -- 172.21.15.31:0/1600853399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8020040 con 0x56333dc889f0 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.183+0000 7fd8dcff9640 1 -- 172.21.15.31:0/1600853399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e80b1040 con 0x56333dc889f0 2024-09-21T06:25:48.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.183+0000 7fd8dcff9640 1 -- 172.21.15.31:0/1600853399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80a9090 con 0x56333dc889f0 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 -- 172.21.15.31:0/1600853399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 msgr2=0x56333e07e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 0x56333e07e980 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80e4670 tx=0x7fd8e801d040 comp rx=0 tx=0).stop 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 -- 172.21.15.31:0/1600853399 shutdown_connections 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e088c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dca11b0 0x56333e072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 --2- 172.21.15.31:0/1600853399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dc889f0 0x56333e07e980 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.184+0000 7fd903b2d740 1 -- 172.21.15.31:0/1600853399 >> 172.21.15.31:0/1600853399 conn(0x56333dcbe460 msgr2=0x56333e0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.185+0000 7fd903b2d740 1 -- 172.21.15.31:0/1600853399 shutdown_connections 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.185+0000 7fd903b2d740 1 -- 172.21.15.31:0/1600853399 wait complete. 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.185+0000 7fd903b2d740 1 -- start start 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.185+0000 7fd903b2d740 1 Processor -- start 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.185+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dc889f0 0x56333e082f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.185+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 0x56333e0834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd903b2d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e0f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd903b2d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81b07a0 con 0x56333dca11b0 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd903b2d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81b1c10 con 0x56333dca9390 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dc889f0 0x56333e082f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 0x56333e0834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 0x56333e0834d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53446/0 (socket says 172.21.15.31:53446) 2024-09-21T06:25:48.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f67fc640 1 -- 172.21.15.31:0/2156780650 learned_addr learned my addr 172.21.15.31:0/2156780650 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e0f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f81b0e70 con 0x56333dc889f0 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd903b2d740 10 client.0 _ll_register_callbacks cb 0x7fd8ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 msgr2=0x56333e0f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e0f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/2156780650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 msgr2=0x56333e0834d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 0x56333e0834d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 -- 172.21.15.31:0/2156780650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333e083d70 con 0x56333dc889f0 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f77fe640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e0f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 0x56333e0834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8f6ffd640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dc889f0 0x56333e082f90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80e4670 tx=0x7fd8e8009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb194f75e92766d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.186+0000 7fd8daff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.187+0000 7fd8daff5640 1 -- 172.21.15.31:0/2156780650 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8020030 con 0x56333dc889f0 2024-09-21T06:25:48.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.187+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e80b1040 con 0x56333dc889f0 2024-09-21T06:25:48.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.187+0000 7fd8daff5640 1 -- 172.21.15.31:0/2156780650 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fd8e80a9090 con 0x56333dc889f0 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd8daff5640 1 -- 172.21.15.31:0/2156780650 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd8e80c7050 con 0x56333dc889f0 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd903b2d740 10 client.28120 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56333e0f9a20 con 0x56333dc889f0 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd8daff5640 1 -- 172.21.15.31:0/2156780650 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e80b1850 con 0x56333dc889f0 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd8daff5640 1 -- 172.21.15.31:0/2156780650 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd8e8094120 con 0x56333dc889f0 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd903b2d740 10 client.28120 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd903b2d740 10 client.28120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd903b2d740 20 client.28120 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.188+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dcca070 0x56333e0fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:48.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.189+0000 7fd903b2d740 4 client.28120 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.189+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd8e80b1040 con 0x56333dcca070 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.189+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dcca070 0x56333e0fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.189+0000 7fd8daff5640 10 client.28120 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.189+0000 7fd8f67fc640 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dcca070 0x56333e0fc190 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c6ea1d9f4d8083 server_cookie=cd3e0dcb95028257 in_seq=0 out_seq=0 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd8daff5640 1 -- 172.21.15.31:0/2156780650 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd8e80b1040 con 0x56333dcca070 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd8daff5640 10 client.28120 handle_command_reply: tid=0 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd903b2d740 1 client.28120 shutdown 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dc889f0 msgr2=0x56333e082f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dc889f0 0x56333e082f90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80e4670 tx=0x7fd8e8009f20 comp rx=0 tx=0).stop 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 shutdown_connections 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.190+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56333dcca070 0x56333e0fc190 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56333dca9390 0x56333e0f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd8daff5640 0 client.28120 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56333dca11b0 0x56333e0834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd903b2d740 1 --2- 172.21.15.31:0/2156780650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56333dc889f0 0x56333e082f90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 >> 172.21.15.31:0/2156780650 conn(0x56333dcbe460 msgr2=0x56333e07e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:48.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 shutdown_connections 2024-09-21T06:25:48.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.191+0000 7fd903b2d740 1 -- 172.21.15.31:0/2156780650 wait complete. 2024-09-21T06:25:48.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.192+0000 7fd903b2d740 20 client.28120 trim_cache size 0 max 16384 2024-09-21T06:25:48.201 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:25:48.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.200+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8e00b6700 msgr2=0x7fd8e00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.200+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8e00b6700 0x7fd8e00b8c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000fc90 tx=0x7fd8f001f000 comp rx=0 tx=0).stop 2024-09-21T06:25:48.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.201+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 msgr2=0x7fd8f819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:48.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.201+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 0x7fd8f819c710 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d6d0 tx=0x7fd8e800db90 comp rx=0 tx=0).stop 2024-09-21T06:25:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.205+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 shutdown_connections 2024-09-21T06:25:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.205+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f810c020 0x7fd8f81a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.205+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd8f810b6d0 0x7fd8f819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.205+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fd8e00b6700 0x7fd8e00b8c10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.205+0000 7fd8fd5b9640 1 --2- 172.21.15.31:0/2663386059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8f8105d90 0x7fd8f819c710 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.205+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 >> 172.21.15.31:0/2663386059 conn(0x7fd8f81017c0 msgr2=0x7fd8f810ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.208+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 shutdown_connections 2024-09-21T06:25:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:48.208+0000 7fd8fd5b9640 1 -- 172.21.15.31:0/2663386059 wait complete. 2024-09-21T06:25:49.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:49 smithi110 ceph-mon[34550]: pgmap v763: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.7 MiB/s wr, 607 op/s 2024-09-21T06:25:49.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:49 smithi107 ceph-mon[34380]: pgmap v763: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.7 MiB/s wr, 607 op/s 2024-09-21T06:25:49.887 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:49 smithi031 ceph-mon[31237]: pgmap v763: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.7 MiB/s wr, 607 op/s 2024-09-21T06:25:50.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:25:50.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:25:50.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 850.0 s, 2469.3 tps, lat 40.481 ms stddev 49.336 2024-09-21T06:25:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:51.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 -- 172.21.15.31:0/1953820480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58102a80 msgr2=0x7fbf58102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/1953820480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58102a80 0x7fbf58102e90 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009ca0 tx=0x7fbf4802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 -- 172.21.15.31:0/1953820480 shutdown_connections 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/1953820480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58108d10 0x7fbf5810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/1953820480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/1953820480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58102a80 0x7fbf58102e90 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 -- 172.21.15.31:0/1953820480 >> 172.21.15.31:0/1953820480 conn(0x7fbf58079740 msgr2=0x7fbf58079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 -- 172.21.15.31:0/1953820480 shutdown_connections 2024-09-21T06:25:51.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.348+0000 7fbf5d3de640 1 -- 172.21.15.31:0/1953820480 wait complete. 2024-09-21T06:25:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.349+0000 7fbf5d3de640 1 Processor -- start 2024-09-21T06:25:51.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.350+0000 7fbf5d3de640 1 -- start start 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.350+0000 7fbf5d3de640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 0x7fbf581a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.350+0000 7fbf5d3de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.350+0000 7fbf5d3de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58108d10 0x7fbf5810e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.350+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 0x7fbf581a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 0x7fbf581a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34366/0 (socket says 172.21.15.31:34366) 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf4ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf5d3de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111190 con 0x7fbf58108d10 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf5d3de640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111010 con 0x7fbf581083c0 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf4ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42910/0 (socket says 172.21.15.31:42910) 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf4ffff640 1 -- 172.21.15.31:0/74815125 learned_addr learned my addr 172.21.15.31:0/74815125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf577fe640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58108d10 0x7fbf5810e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111310 con 0x7fbf58102a80 2024-09-21T06:25:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf577fe640 1 -- 172.21.15.31:0/74815125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 msgr2=0x7fbf581a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf577fe640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 0x7fbf581a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf577fe640 1 -- 172.21.15.31:0/74815125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 msgr2=0x7fbf581a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf577fe640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.351+0000 7fbf577fe640 1 -- 172.21.15.31:0/74815125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5810e9c0 con 0x7fbf58108d10 2024-09-21T06:25:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.352+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 0x7fbf581a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.352+0000 7fbf577fe640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58108d10 0x7fbf5810e1f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e970 tx=0x7fbf5000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.352+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.353+0000 7fbf54ff9640 1 -- 172.21.15.31:0/74815125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000ccc0 con 0x7fbf58108d10 2024-09-21T06:25:51.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.353+0000 7fbf54ff9640 1 -- 172.21.15.31:0/74815125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50005020 con 0x7fbf58108d10 2024-09-21T06:25:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.353+0000 7fbf54ff9640 1 -- 172.21.15.31:0/74815125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50004a00 con 0x7fbf58108d10 2024-09-21T06:25:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.353+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5810ecb0 con 0x7fbf58108d10 2024-09-21T06:25:51.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.353+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581ad3d0 con 0x7fbf58108d10 2024-09-21T06:25:51.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.357+0000 7fbf63952740 1 Processor -- start 2024-09-21T06:25:51.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.357+0000 7fbf54ff9640 1 -- 172.21.15.31:0/74815125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbf50004ba0 con 0x7fbf58108d10 2024-09-21T06:25:51.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.357+0000 7fbf63952740 1 -- start start 2024-09-21T06:25:51.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf63952740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 0x55a4dbee3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf63952740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 0x55a4dbee3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf63952740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbee3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf63952740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a4dbe59e60 con 0x55a4dbb3c350 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf63952740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a4dbe49290 con 0x55a4dbb13490 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf63952740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4dbe4d0a0 con 0x55a4dbb1b670 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 0x55a4dbee3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 0x55a4dbee3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf4ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 0x55a4dbee3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48686/0 (socket says 172.21.15.31:48686) 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 0x55a4dbee3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42922/0 (socket says 172.21.15.31:42922) 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf4ffff640 1 -- 172.21.15.31:0/3849891440 learned_addr learned my addr 172.21.15.31:0/3849891440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.358+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbee3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4ffff640 1 -- 172.21.15.31:0/3849891440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 msgr2=0x55a4dbee3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbee3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4ffff640 1 -- 172.21.15.31:0/3849891440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 msgr2=0x55a4dbee3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 0x55a4dbee3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4ffff640 1 -- 172.21.15.31:0/3849891440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4dbba7930 con 0x55a4dbb3c350 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 0x55a4dbee3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf54ff9640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b6680 0x7fbf440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 0x55a4dbee3430 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbf40017710 tx=0x7fbf40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a817560807c7afec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b6680 0x7fbf440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf54ff9640 1 -- 172.21.15.31:0/74815125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fbf500e7730 con 0x7fbf58108d10 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4d7fa640 1 -- 172.21.15.31:0/3849891440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40017c90 con 0x55a4dbb3c350 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4d7fa640 1 -- 172.21.15.31:0/3849891440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf40017e30 con 0x55a4dbb3c350 2024-09-21T06:25:51.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.359+0000 7fbf4d7fa640 1 -- 172.21.15.31:0/3849891440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40009c10 con 0x55a4dbb3c350 2024-09-21T06:25:51.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.360+0000 7fbf63952740 1 -- 172.21.15.31:0/3849891440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 msgr2=0x55a4dbee3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.360+0000 7fbf63952740 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 0x55a4dbee3430 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbf40017710 tx=0x7fbf40005f90 comp rx=0 tx=0).stop 2024-09-21T06:25:51.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b6680 0x7fbf440b8b90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005e30 tx=0x7fbf48009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 -- 172.21.15.31:0/3849891440 shutdown_connections 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbee3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb13490 0x55a4dbee3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 --2- 172.21.15.31:0/3849891440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb3c350 0x55a4dbee3430 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 -- 172.21.15.31:0/3849891440 >> 172.21.15.31:0/3849891440 conn(0x55a4dbb30740 msgr2=0x55a4dbee1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 -- 172.21.15.31:0/3849891440 shutdown_connections 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.361+0000 7fbf63952740 1 -- 172.21.15.31:0/3849891440 wait complete. 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.362+0000 7fbf63952740 1 -- start start 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.362+0000 7fbf63952740 1 Processor -- start 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbeeeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb1b670 0x55a4dbeef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb3c350 0x55a4dbefaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55a4dbcaa890 con 0x55a4dbb13490 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbeeeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a4dbde3d40 con 0x55a4dbb1b670 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a4dbef1bb0 con 0x55a4dbb3c350 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf56ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbeeeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48690/0 (socket says 172.21.15.31:48690) 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf56ffd640 1 -- 172.21.15.31:0/914591363 learned_addr learned my addr 172.21.15.31:0/914591363 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.363+0000 7fbf63952740 10 client.0 _ll_register_callbacks cb 0x55a4dbe47c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb1b670 0x55a4dbeef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf577fe640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb3c350 0x55a4dbefaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 -- 172.21.15.31:0/914591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb3c350 msgr2=0x55a4dbefaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb3c350 0x55a4dbefaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 -- 172.21.15.31:0/914591363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 msgr2=0x55a4dbeeeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbeeeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 -- 172.21.15.31:0/914591363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4dbefb440 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.364+0000 7fbf4ffff640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb1b670 0x55a4dbeef530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbf400176e0 tx=0x7fbf400098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a978c97dcc11aed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/914591363 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40004020 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/914591363 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fbf400041c0 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf40017e30 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/914591363 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf400043c0 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf63952740 10 client.28100 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.365+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a4dbefdd70 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/914591363 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf40017e30 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/914591363 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf4001a040 con 0x55a4dbb1b670 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf63952740 10 client.28100 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:51.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf63952740 10 client.28100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf63952740 20 client.28100 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf63952740 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbafacd0 0x55a4dbf01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf63952740 4 client.28100 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4dbe5bdb0 con 0x55a4dbafacd0 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.366+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbafacd0 0x55a4dbf01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.367+0000 7fbf3e7fc640 10 client.28100 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.367+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbafacd0 0x55a4dbf01180 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb25f88ea72e258f server_cookie=f1c05f0b5241a2b in_seq=0 out_seq=0 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/914591363 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a4dbe5bdb0 con 0x55a4dbafacd0 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf3e7fc640 10 client.28100 handle_command_reply: tid=0 2024-09-21T06:25:51.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 client.28100 shutdown 2024-09-21T06:25:51.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb1b670 msgr2=0x55a4dbeef530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb1b670 0x55a4dbeef530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbf400176e0 tx=0x7fbf400098f0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 shutdown_connections 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbafacd0 0x55a4dbf01180 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf3e7fc640 0 client.28100 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb3c350 0x55a4dbefaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbb1b670 0x55a4dbeef530 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 --2- 172.21.15.31:0/914591363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbeeeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 >> 172.21.15.31:0/914591363 conn(0x55a4dbb30740 msgr2=0x55a4dbee6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.368+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 shutdown_connections 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.369+0000 7fbf63952740 1 -- 172.21.15.31:0/914591363 wait complete. 2024-09-21T06:25:51.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.369+0000 7fbf63952740 20 client.28100 trim_cache size 0 max 16384 2024-09-21T06:25:51.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.397+0000 7fbf63952740 1 Processor -- start 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 -- start start 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbee8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb13490 0x55a4dbee8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb1b670 0x55a4dbeebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b3db0 con 0x55a4dbb1b670 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b6610 con 0x55a4dbafacd0 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.398+0000 7fbf63952740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b4400 con 0x55a4dbb13490 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.399+0000 7fbf4ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb13490 0x55a4dbee8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.399+0000 7fbf4ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb13490 0x55a4dbee8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34388/0 (socket says 172.21.15.31:34388) 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.399+0000 7fbf4ffff640 1 -- 172.21.15.31:0/1537516037 learned_addr learned my addr 172.21.15.31:0/1537516037 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:51.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.399+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbee8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.405+0000 7fbf56ffd640 1 -- 172.21.15.31:0/1537516037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb13490 msgr2=0x55a4dbee8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.405+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb13490 0x55a4dbee8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.405+0000 7fbf56ffd640 1 -- 172.21.15.31:0/1537516037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb1b670 msgr2=0x55a4dbeebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:51.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.405+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb1b670 0x55a4dbeebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.405+0000 7fbf56ffd640 1 -- 172.21.15.31:0/1537516037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4dbee5720 con 0x55a4dbafacd0 2024-09-21T06:25:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.409+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbee8540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55a4dbf01720 tx=0x7fbf48045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584456a4fc18dbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.412+0000 7fbf4cff9640 1 -- 172.21.15.31:0/1537516037 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4802f660 con 0x55a4dbafacd0 2024-09-21T06:25:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.412+0000 7fbf4cff9640 1 -- 172.21.15.31:0/1537516037 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48036040 con 0x55a4dbafacd0 2024-09-21T06:25:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.413+0000 7fbf63952740 1 -- 172.21.15.31:0/1537516037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 msgr2=0x55a4dbee8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.413+0000 7fbf63952740 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbee8540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55a4dbf01720 tx=0x7fbf48045000 comp rx=0 tx=0).stop 2024-09-21T06:25:51.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.416+0000 7fbf63952740 1 -- 172.21.15.31:0/1537516037 shutdown_connections 2024-09-21T06:25:51.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.416+0000 7fbf63952740 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb1b670 0x55a4dbeebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.416+0000 7fbf63952740 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb13490 0x55a4dbee8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.416+0000 7fbf63952740 1 --2- 172.21.15.31:0/1537516037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbee8540 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.416+0000 7fbf63952740 1 -- 172.21.15.31:0/1537516037 >> 172.21.15.31:0/1537516037 conn(0x55a4dbb30740 msgr2=0x55a4dbeed670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.420+0000 7fbf63952740 1 -- 172.21.15.31:0/1537516037 shutdown_connections 2024-09-21T06:25:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.424+0000 7fbf63952740 1 -- 172.21.15.31:0/1537516037 wait complete. 2024-09-21T06:25:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.424+0000 7fbf63952740 1 -- start start 2024-09-21T06:25:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.424+0000 7fbf63952740 1 Processor -- start 2024-09-21T06:25:51.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbf65180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbf656c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbef6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b4ad0 con 0x55a4dbb13490 2024-09-21T06:25:51.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b5f40 con 0x55a4dbafacd0 2024-09-21T06:25:51.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581b51a0 con 0x55a4dbb1b670 2024-09-21T06:25:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.425+0000 7fbf63952740 10 client.0 _ll_register_callbacks cb 0x7fbf4000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.426+0000 7fbf577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbef6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.426+0000 7fbf577fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbef6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34392/0 (socket says 172.21.15.31:34392) 2024-09-21T06:25:51.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.426+0000 7fbf577fe640 1 -- 172.21.15.31:0/3400373862 learned_addr learned my addr 172.21.15.31:0/3400373862 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:51.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.428+0000 7fbf577fe640 1 -- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 msgr2=0x55a4dbf65180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:25:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.428+0000 7fbf577fe640 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbf65180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.428+0000 7fbf577fe640 1 -- 172.21.15.31:0/3400373862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 msgr2=0x55a4dbf656c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:25:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.428+0000 7fbf577fe640 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbf656c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.428+0000 7fbf577fe640 1 -- 172.21.15.31:0/3400373862 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4dbef7370 con 0x55a4dbb1b670 2024-09-21T06:25:51.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.434+0000 7fbf577fe640 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbef6e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf500aa800 tx=0x7fbf500acfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfea46bafc03135 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:51.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.434+0000 7fbf3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:25:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.436+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/3400373862 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5001c030 con 0x55a4dbb1b670 2024-09-21T06:25:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.436+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf48036040 con 0x55a4dbb1b670 2024-09-21T06:25:51.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.436+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/3400373862 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7fbf500b6240 con 0x55a4dbb1b670 2024-09-21T06:25:51.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.438+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/3400373862 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf500b2070 con 0x55a4dbb1b670 2024-09-21T06:25:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.442+0000 7fbf63952740 10 client.28138 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:51.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.442+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a4dbf8e240 con 0x55a4dbb1b670 2024-09-21T06:25:51.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.442+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/3400373862 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf48036040 con 0x55a4dbb1b670 2024-09-21T06:25:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/3400373862 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf500ac2f0 con 0x55a4dbb1b670 2024-09-21T06:25:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf63952740 10 client.28138 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf63952740 10 client.28138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf63952740 20 client.28138 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf63952740 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbb3c350 0x55a4dbf90a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf63952740 4 client.28138 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:25:51.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.443+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4dbe57860 con 0x55a4dbb3c350 2024-09-21T06:25:51.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.450+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbb3c350 0x55a4dbf90a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.458+0000 7fbf3e7fc640 10 client.28138 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.458+0000 7fbf56ffd640 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbb3c350 0x55a4dbf90a80 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda2f884d64b48de server_cookie=8d70a83108a40dca in_seq=0 out_seq=0 2024-09-21T06:25:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.461+0000 7fbf3e7fc640 1 -- 172.21.15.31:0/3400373862 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4dbe57860 con 0x55a4dbb3c350 2024-09-21T06:25:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.461+0000 7fbf3e7fc640 10 client.28138 handle_command_reply: tid=0 2024-09-21T06:25:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.461+0000 7fbf63952740 1 client.28138 shutdown 2024-09-21T06:25:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.466+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 msgr2=0x55a4dbef6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.466+0000 7fbf63952740 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbef6e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf500aa800 tx=0x7fbf500acfa0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 shutdown_connections 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55a4dbb3c350 0x55a4dbf90a80 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4dbb1b670 0x55a4dbef6e30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a4dbb13490 0x55a4dbf656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 --2- 172.21.15.31:0/3400373862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a4dbafacd0 0x55a4dbf65180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf3e7fc640 0 client.28138 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 >> 172.21.15.31:0/3400373862 conn(0x55a4dbb30740 msgr2=0x55a4dbee8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 shutdown_connections 2024-09-21T06:25:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 1 -- 172.21.15.31:0/3400373862 wait complete. 2024-09-21T06:25:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.469+0000 7fbf63952740 20 client.28138 trim_cache size 0 max 16384 2024-09-21T06:25:51.472 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:25:51.472 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:25:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "97dcd776-5fd7-4ec8-ada7-f276bb218390", 2024-09-21T06:25:51.473 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:25:51.473 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:25:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b6680 msgr2=0x7fbf440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b6680 0x7fbf440b8b90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf48005e30 tx=0x7fbf48009360 comp rx=0 tx=0).stop 2024-09-21T06:25:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58108d10 msgr2=0x7fbf5810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58108d10 0x7fbf5810e1f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e970 tx=0x7fbf5000ee30 comp rx=0 tx=0).stop 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 shutdown_connections 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbf440b6680 0x7fbf440b8b90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf58108d10 0x7fbf5810e1f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf581083c0 0x7fbf581a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 --2- 172.21.15.31:0/74815125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf58102a80 0x7fbf581a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.471+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 >> 172.21.15.31:0/74815125 conn(0x7fbf58079740 msgr2=0x7fbf58101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.472+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 shutdown_connections 2024-09-21T06:25:51.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:51.472+0000 7fbf5d3de640 1 -- 172.21.15.31:0/74815125 wait complete. 2024-09-21T06:25:51.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:51 smithi031 ceph-mon[31237]: pgmap v764: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-09-21T06:25:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:51 smithi110 ceph-mon[34550]: pgmap v764: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-09-21T06:25:51.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:51 smithi107 ceph-mon[34380]: pgmap v764: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.3 MiB/s wr, 652 op/s 2024-09-21T06:25:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcca9ffb640 1 -- 172.21.15.31:0/510812598 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642b0027b10 con 0x5642afc841b0 2024-09-21T06:25:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcca9ffb640 10 client.18273 handle_command_reply: tid=0 2024-09-21T06:25:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 client.18273 shutdown 2024-09-21T06:25:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 msgr2=0x5642b00af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 0x5642b00af8a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0057c0 tx=0x7fccbc02a340 comp rx=0 tx=0).stop 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 shutdown_connections 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x5642afc841b0 0x5642b00b3870 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642afc634d0 0x5642b00af8a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5642afc5b2f0 0x5642b0049db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 --2- 172.21.15.31:0/510812598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642afc42d80 0x5642b0049870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.164+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 >> 172.21.15.31:0/510812598 conn(0x5642afc78300 msgr2=0x5642b002b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.165+0000 7fcca9ffb640 0 client.18273 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.165+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 shutdown_connections 2024-09-21T06:25:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.165+0000 7fcccfc69740 1 -- 172.21.15.31:0/510812598 wait complete. 2024-09-21T06:25:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.165+0000 7fcccfc69740 20 client.18273 trim_cache size 0 max 16384 2024-09-21T06:25:52.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:25:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:25:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:25:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:25:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fccac0b6680 msgr2=0x7fccac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fccac0b6680 0x7fccac0b8b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0095f0 tx=0x7fccbc02c040 comp rx=0 tx=0).stop 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 msgr2=0x7fccc41a0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 0x7fccc41a0f90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fccb80058c0 tx=0x7fccb800cd90 comp rx=0 tx=0).stop 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 shutdown_connections 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fccac0b6680 0x7fccac0b8b90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fccc41107b0 0x7fccc41a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccc410fe60 0x7fccc41a0f90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 --2- 172.21.15.31:0/2497016971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc410a660 0x7fccc41a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 >> 172.21.15.31:0/2497016971 conn(0x7fccc4071220 msgr2=0x7fccc410f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 shutdown_connections 2024-09-21T06:25:52.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:25:52.166+0000 7fccc8c27640 1 -- 172.21.15.31:0/2497016971 wait complete. 2024-09-21T06:25:52.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:25:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:52 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:25:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:52 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:52.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:52 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:25:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:52 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:25:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:52 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:52.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:52 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:25:52.883 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:25:52.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:52 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:25:52.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:52 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:52.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:52 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 -- 172.21.15.31:0/1674439241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c1087f0 msgr2=0x7f5f5c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 --2- 172.21.15.31:0/1674439241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c108c00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009900 tx=0x7f5f5002a410 comp rx=0 tx=0).stop 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 -- 172.21.15.31:0/1674439241 shutdown_connections 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 --2- 172.21.15.31:0/1674439241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102730 0x7f5f5c112980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 --2- 172.21.15.31:0/1674439241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c101de0 0x7f5f5c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 --2- 172.21.15.31:0/1674439241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c108c00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 -- 172.21.15.31:0/1674439241 >> 172.21.15.31:0/1674439241 conn(0x7f5f5c077c60 msgr2=0x7f5f5c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 -- 172.21.15.31:0/1674439241 shutdown_connections 2024-09-21T06:25:53.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.486+0000 7f5f61550640 1 -- 172.21.15.31:0/1674439241 wait complete. 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.487+0000 7f5f61550640 1 Processor -- start 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.487+0000 7f5f61550640 1 -- start start 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.487+0000 7f5f61550640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 0x7f5f5c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.487+0000 7f5f61550640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102730 0x7f5f5c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.487+0000 7f5f61550640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f61550640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c115ce0 con 0x7f5f5c1087f0 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f61550640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c115b60 con 0x7f5f5c101de0 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f61550640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c115e60 con 0x7f5f5c102730 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 0x7f5f5c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 0x7f5f5c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42978/0 (socket says 172.21.15.31:42978) 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 -- 172.21.15.31:0/1317132004 learned_addr learned my addr 172.21.15.31:0/1317132004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c1a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48720/0 (socket says 172.21.15.31:48720) 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102730 0x7f5f5c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 -- 172.21.15.31:0/1317132004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102730 msgr2=0x7f5f5c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102730 0x7f5f5c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 -- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1087f0 msgr2=0x7f5f5c1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 -- 172.21.15.31:0/1317132004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c1a25c0 con 0x7f5f5c101de0 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.488+0000 7f5f5affd640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 0x7f5f5c19c650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005ab0 tx=0x7f5f50003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.489+0000 7f5f43fff640 1 -- 172.21.15.31:0/1317132004 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5003f070 con 0x7f5f5c101de0 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.489+0000 7f5f43fff640 1 -- 172.21.15.31:0/1317132004 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f50031b60 con 0x7f5f5c101de0 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.489+0000 7f5f43fff640 1 -- 172.21.15.31:0/1317132004 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50035030 con 0x7f5f5c101de0 2024-09-21T06:25:53.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.489+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c1a2850 con 0x7f5f5c101de0 2024-09-21T06:25:53.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.489+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c1a9010 con 0x7f5f5c101de0 2024-09-21T06:25:53.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.491+0000 7f5f43fff640 1 -- 172.21.15.31:0/1317132004 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5f50042070 con 0x7f5f5c101de0 2024-09-21T06:25:53.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.494+0000 7f5f67ac4740 1 Processor -- start 2024-09-21T06:25:53.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.494+0000 7f5f67ac4740 1 -- start start 2024-09-21T06:25:53.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.494+0000 7f5f43fff640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f440b6700 0x7f5f440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.494+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f440b6700 0x7f5f440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.494+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b424ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b4211f1b0 0x555b424ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 0x555b424efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555b42465e50 con 0x555b4211f1b0 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555b42455280 con 0x555b42148070 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b42459090 con 0x555b42127390 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f43fff640 1 -- 172.21.15.31:0/1317132004 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f5f50107e00 con 0x7f5f5c101de0 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f5b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 0x555b424efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f5b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 0x555b424efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34420/0 (socket says 172.21.15.31:34420) 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/998557393 learned_addr learned my addr 172.21.15.31:0/998557393 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f5affd640 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b424ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.495+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b4211f1b0 0x555b424ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/998557393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 msgr2=0x555b424ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b424ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/998557393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b4211f1b0 msgr2=0x555b424ef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b4211f1b0 0x555b424ef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/998557393 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b421b3650 con 0x555b42127390 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b4211f1b0 0x555b424ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f440b6700 0x7f5f440b8c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009fd0 tx=0x7f5f54011c60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 0x555b424efcf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400fca0 tx=0x7f5f54020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede906c483d64879 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f417fa640 1 -- 172.21.15.31:0/998557393 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5401a040 con 0x555b42127390 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f417fa640 1 -- 172.21.15.31:0/998557393 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54024030 con 0x555b42127390 2024-09-21T06:25:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.496+0000 7f5f417fa640 1 -- 172.21.15.31:0/998557393 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54005560 con 0x555b42127390 2024-09-21T06:25:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.498+0000 7f5f67ac4740 1 -- 172.21.15.31:0/998557393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 msgr2=0x555b424efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.498+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 0x555b424efcf0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400fca0 tx=0x7f5f54020000 comp rx=0 tx=0).stop 2024-09-21T06:25:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.500+0000 7f5f67ac4740 1 -- 172.21.15.31:0/998557393 shutdown_connections 2024-09-21T06:25:53.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.500+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42127390 0x555b424efcf0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.500+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b4211f1b0 0x555b424ef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.504+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/998557393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b424ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.504+0000 7f5f67ac4740 1 -- 172.21.15.31:0/998557393 >> 172.21.15.31:0/998557393 conn(0x555b4213c460 msgr2=0x555b424ed160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.504+0000 7f5f67ac4740 1 -- 172.21.15.31:0/998557393 shutdown_connections 2024-09-21T06:25:53.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.504+0000 7f5f67ac4740 1 -- 172.21.15.31:0/998557393 wait complete. 2024-09-21T06:25:53.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.505+0000 7f5f67ac4740 1 -- start start 2024-09-21T06:25:53.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.505+0000 7f5f67ac4740 1 Processor -- start 2024-09-21T06:25:53.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b424fb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b424fb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b42500750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x555b422b6700 con 0x555b42127390 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555b423f0110 con 0x555b4211f1b0 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b424f7970 con 0x555b42148070 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f67ac4740 10 client.0 _ll_register_callbacks cb 0x555b42409990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b424fb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f5affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b424fb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42990/0 (socket says 172.21.15.31:42990) 2024-09-21T06:25:53.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.506+0000 7f5f5affd640 1 -- 172.21.15.31:0/240398337 learned_addr learned my addr 172.21.15.31:0/240398337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:53.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.507+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b424fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.508+0000 7f5f5affd640 1 -- 172.21.15.31:0/240398337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 msgr2=0x555b42500750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:25:53.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.508+0000 7f5f5affd640 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b42500750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.508+0000 7f5f5affd640 1 -- 172.21.15.31:0/240398337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 msgr2=0x555b424fb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.508+0000 7f5f5affd640 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b424fb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.508+0000 7f5f5affd640 1 -- 172.21.15.31:0/240398337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b4241dc40 con 0x555b4211f1b0 2024-09-21T06:25:53.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.515+0000 7f5f5affd640 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b424fb150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5f50000c00 tx=0x7f5f500cd4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a64f915acdf3b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:53.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.515+0000 7f5f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:25:53.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.519+0000 7f5f3eff5640 1 -- 172.21.15.31:0/240398337 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50049030 con 0x555b4211f1b0 2024-09-21T06:25:53.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.519+0000 7f5f3eff5640 1 -- 172.21.15.31:0/240398337 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f5f5003b460 con 0x555b4211f1b0 2024-09-21T06:25:53.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.520+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54024030 con 0x555b4211f1b0 2024-09-21T06:25:53.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.524+0000 7f5f3eff5640 1 -- 172.21.15.31:0/240398337 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5f500b9020 con 0x555b4211f1b0 2024-09-21T06:25:53.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.527+0000 7f5f67ac4740 10 client.28115 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:53.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.527+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555b425096a0 con 0x555b4211f1b0 2024-09-21T06:25:53.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f3eff5640 1 -- 172.21.15.31:0/240398337 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54024030 con 0x555b4211f1b0 2024-09-21T06:25:53.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f3eff5640 1 -- 172.21.15.31:0/240398337 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5f500d7190 con 0x555b4211f1b0 2024-09-21T06:25:53.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f67ac4740 10 client.28115 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:53.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f67ac4740 10 client.28115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:53.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f67ac4740 20 client.28115 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:53.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b421069f0 0x555b4250cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f67ac4740 4 client.28115 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:25:53.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.531+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b42467da0 con 0x555b421069f0 2024-09-21T06:25:53.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.539+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b421069f0 0x555b4250cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.541+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b421069f0 0x555b4250cbb0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857b90b650c74b12 server_cookie=dfdfa3cea77bbcf7 in_seq=0 out_seq=0 2024-09-21T06:25:53.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.541+0000 7f5f3eff5640 10 client.28115 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:53.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.542+0000 7f5f3eff5640 1 -- 172.21.15.31:0/240398337 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b42467da0 con 0x555b421069f0 2024-09-21T06:25:53.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.542+0000 7f5f3eff5640 10 client.28115 handle_command_reply: tid=0 2024-09-21T06:25:53.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.542+0000 7f5f67ac4740 1 client.28115 shutdown 2024-09-21T06:25:53.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 msgr2=0x555b424fb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b424fb150 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5f50000c00 tx=0x7f5f500cd4e0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 shutdown_connections 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b421069f0 0x555b4250cbb0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b42500750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b424fb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/240398337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b424fb150 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 >> 172.21.15.31:0/240398337 conn(0x555b4213c460 msgr2=0x555b424f2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f3eff5640 0 client.28115 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.543+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 shutdown_connections 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.544+0000 7f5f67ac4740 1 -- 172.21.15.31:0/240398337 wait complete. 2024-09-21T06:25:53.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.544+0000 7f5f67ac4740 20 client.28115 trim_cache size 0 max 16384 2024-09-21T06:25:53.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.578+0000 7f5f67ac4740 1 Processor -- start 2024-09-21T06:25:53.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.578+0000 7f5f67ac4740 1 -- start start 2024-09-21T06:25:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.578+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 0x555b424f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.578+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42512370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.578+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b425128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ab530 con 0x555b42127390 2024-09-21T06:25:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1add70 con 0x555b42148070 2024-09-21T06:25:53.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1abb80 con 0x555b4211f1b0 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 0x555b424f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 0x555b424f2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:34426/0 (socket says 172.21.15.31:34426) 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 -- 172.21.15.31:0/179502314 learned_addr learned my addr 172.21.15.31:0/179502314 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b425128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 -- 172.21.15.31:0/179502314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 msgr2=0x555b425128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b425128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 -- 172.21.15.31:0/179502314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 msgr2=0x555b42512370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42512370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.579+0000 7f5f5affd640 1 -- 172.21.15.31:0/179502314 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b42106c20 con 0x555b4211f1b0 2024-09-21T06:25:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.580+0000 7f5f5affd640 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 0x555b424f2a20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5f500cd9b0 tx=0x7f5f500045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5fcafa98923e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.580+0000 7f5f40ff9640 1 -- 172.21.15.31:0/179502314 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50049020 con 0x555b4211f1b0 2024-09-21T06:25:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.580+0000 7f5f40ff9640 1 -- 172.21.15.31:0/179502314 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f500d7620 con 0x555b4211f1b0 2024-09-21T06:25:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.580+0000 7f5f40ff9640 1 -- 172.21.15.31:0/179502314 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500d7190 con 0x555b4211f1b0 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 -- 172.21.15.31:0/179502314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 msgr2=0x555b424f2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 0x555b424f2a20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5f500cd9b0 tx=0x7f5f500045e0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 -- 172.21.15.31:0/179502314 shutdown_connections 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b42148070 0x555b425128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42512370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/179502314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b4211f1b0 0x555b424f2a20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 -- 172.21.15.31:0/179502314 >> 172.21.15.31:0/179502314 conn(0x555b4213c460 msgr2=0x555b424f2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 -- 172.21.15.31:0/179502314 shutdown_connections 2024-09-21T06:25:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.582+0000 7f5f67ac4740 1 -- 172.21.15.31:0/179502314 wait complete. 2024-09-21T06:25:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 -- start start 2024-09-21T06:25:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 Processor -- start 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b42504e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42505370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b425058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ac230 con 0x555b42127390 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ad6a0 con 0x555b4211f1b0 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ac900 con 0x555b42148070 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f67ac4740 10 client.0 _ll_register_callbacks cb 0x7f5f500c9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f5a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42505370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f5a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42505370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48766/0 (socket says 172.21.15.31:48766) 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f5a7fc640 1 -- 172.21.15.31:0/1875477517 learned_addr learned my addr 172.21.15.31:0/1875477517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.583+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b425058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5affd640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b42504e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5a7fc640 1 -- 172.21.15.31:0/1875477517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 msgr2=0x555b425058b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b425058b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5a7fc640 1 -- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 msgr2=0x555b42504e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b42504e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5a7fc640 1 -- 172.21.15.31:0/1875477517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b42576780 con 0x555b42127390 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5b7fe640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b425058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f5a7fc640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42505370 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c075820 tx=0x7f5f4c009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5a358205775a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f3eff5640 1 -- 172.21.15.31:0/1875477517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c04c060 con 0x555b42127390 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f3eff5640 1 -- 172.21.15.31:0/1875477517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f5f4c040830 con 0x555b42127390 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.584+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f500d7620 con 0x555b42127390 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.585+0000 7f5f3eff5640 1 -- 172.21.15.31:0/1875477517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5f4c040a30 con 0x555b42127390 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.585+0000 7f5f67ac4740 10 client.18324 fetch_fsmap learned FSMap version 21 2024-09-21T06:25:53.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.585+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555b42577aa0 con 0x555b42127390 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.585+0000 7f5f3eff5640 1 -- 172.21.15.31:0/1875477517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f500d7620 con 0x555b42127390 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.585+0000 7f5f3eff5640 1 -- 172.21.15.31:0/1875477517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5f4c04fc30 con 0x555b42127390 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f67ac4740 10 client.18324 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f67ac4740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f67ac4740 20 client.18324 populate_metadata read hostname 'smithi031' 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b420dbda0 0x555b4257a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f67ac4740 4 client.18324 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b42463850 con 0x555b420dbda0 2024-09-21T06:25:53.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f5affd640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b420dbda0 0x555b4257a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f3eff5640 10 client.18324 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.586+0000 7f5f5affd640 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b420dbda0 0x555b4257a2c0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7110cd284b7890 server_cookie=673e071d15939787 in_seq=0 out_seq=0 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f3eff5640 1 -- 172.21.15.31:0/1875477517 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9658 (crc 0 0 0) 0x555b42463850 con 0x555b420dbda0 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f3eff5640 10 client.18324 handle_command_reply: tid=0 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 client.18324 shutdown 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 msgr2=0x555b42505370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42505370 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c075820 tx=0x7f5f4c009bc0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 shutdown_connections 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x555b420dbda0 0x555b4257a2c0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b42148070 0x555b425058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555b42127390 0x555b42505370 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 --2- 172.21.15.31:0/1875477517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555b4211f1b0 0x555b42504e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 >> 172.21.15.31:0/1875477517 conn(0x555b4213c460 msgr2=0x555b424f02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f3eff5640 0 client.18324 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 shutdown_connections 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 1 -- 172.21.15.31:0/1875477517 wait complete. 2024-09-21T06:25:53.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.587+0000 7f5f67ac4740 20 client.18324 trim_cache size 0 max 16384 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "97dcd776-5fd7-4ec8-ada7-f276bb218390": { 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "97dcd776-5fd7-4ec8-ada7-f276bb218390", 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "97dcd776-5fd7-4ec8-ada7-f276bb218390": { 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:51.458278+0000", 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:10.069920+0000", 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:25:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:25:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:25:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:25:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f440b6700 msgr2=0x7f5f440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f440b6700 0x7f5f440b8c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009fd0 tx=0x7f5f54011c60 comp rx=0 tx=0).stop 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 msgr2=0x7f5f5c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:25:53.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 0x7f5f5c19c650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5f50005ab0 tx=0x7f5f50003c10 comp rx=0 tx=0).stop 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 shutdown_connections 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5f440b6700 0x7f5f440b8c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c1087f0 0x7f5f5c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f5c102730 0x7f5f5c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 --2- 172.21.15.31:0/1317132004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f5c101de0 0x7f5f5c19c650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 >> 172.21.15.31:0/1317132004 conn(0x7f5f5c077c60 msgr2=0x7f5f5c107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 shutdown_connections 2024-09-21T06:25:53.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:25:53.588+0000 7f5f61550640 1 -- 172.21.15.31:0/1317132004 wait complete. 2024-09-21T06:25:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:53 smithi031 ceph-mon[31237]: pgmap v765: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-09-21T06:25:53.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:53 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:53 smithi110 ceph-mon[34550]: pgmap v765: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-09-21T06:25:53.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:53 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:53 smithi107 ceph-mon[34380]: pgmap v765: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 698 op/s 2024-09-21T06:25:53.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:53 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:25:54.527 INFO:tasks.cephfs.filesystem:scrub status for tag:97dcd776-5fd7-4ec8-ada7-f276bb218390 - {'path': '/', 'tag': '97dcd776-5fd7-4ec8-ada7-f276bb218390', 'options': 'recursive,force'} 2024-09-21T06:25:55.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 855.0 s, 1659.2 tps, lat 60.122 ms stddev 74.288 2024-09-21T06:25:55.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:55 smithi110 ceph-mon[34550]: pgmap v766: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-09-21T06:25:55.786 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:55 smithi031 ceph-mon[31237]: pgmap v766: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-09-21T06:25:55.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:55 smithi107 ceph-mon[34380]: pgmap v766: 97 pgs: 97 active+clean; 4.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-09-21T06:25:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:57 smithi110 ceph-mon[34550]: pgmap v767: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.6 MiB/s wr, 645 op/s 2024-09-21T06:25:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:57 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:25:57.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:57 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:25:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:57 smithi107 ceph-mon[34380]: pgmap v767: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.6 MiB/s wr, 645 op/s 2024-09-21T06:25:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:57 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:25:57.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:57 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:25:57.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:57 smithi031 ceph-mon[31237]: pgmap v767: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.6 MiB/s wr, 645 op/s 2024-09-21T06:25:57.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:57 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:25:57.833 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:57 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:25:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:25:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:25:58.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:25:58.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:25:58.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:25:58.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:25:58.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:25:58.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:25:58.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:25:59.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:25:59 smithi110 ceph-mon[34550]: pgmap v768: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 7.3 MiB/s wr, 790 op/s 2024-09-21T06:25:59.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:25:59 smithi107 ceph-mon[34380]: pgmap v768: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 7.3 MiB/s wr, 790 op/s 2024-09-21T06:25:59.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:25:59 smithi031 ceph-mon[31237]: pgmap v768: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 7.3 MiB/s wr, 790 op/s 2024-09-21T06:26:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 860.0 s, 1110.6 tps, lat 89.705 ms stddev 103.176 2024-09-21T06:26:01.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:01 smithi110 ceph-mon[34550]: pgmap v769: 97 pgs: 97 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 9.4 MiB/s wr, 847 op/s 2024-09-21T06:26:01.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:01 smithi107 ceph-mon[34380]: pgmap v769: 97 pgs: 97 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 9.4 MiB/s wr, 847 op/s 2024-09-21T06:26:01.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:01 smithi031 ceph-mon[31237]: pgmap v769: 97 pgs: 97 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 9.4 MiB/s wr, 847 op/s 2024-09-21T06:26:03.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:03 smithi110 ceph-mon[34550]: pgmap v770: 97 pgs: 97 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2024-09-21T06:26:03.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:03 smithi107 ceph-mon[34380]: pgmap v770: 97 pgs: 97 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2024-09-21T06:26:03.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:03 smithi031 ceph-mon[31237]: pgmap v770: 97 pgs: 97 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2024-09-21T06:26:05.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 865.0 s, 663.6 tps, lat 151.153 ms stddev 352.990 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.323+0000 7f147b97b640 1 -- 172.21.15.31:0/1301421277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 msgr2=0x7f1474108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.323+0000 7f147b97b640 1 --2- 172.21.15.31:0/1301421277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474108390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f146c009ca0 tx=0x7f146c02a8d0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.328+0000 7f147b97b640 1 -- 172.21.15.31:0/1301421277 shutdown_connections 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.328+0000 7f147b97b640 1 --2- 172.21.15.31:0/1301421277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474108390 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.328+0000 7f147b97b640 1 --2- 172.21.15.31:0/1301421277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1474107220 0x7f1474107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.328+0000 7f147b97b640 1 --2- 172.21.15.31:0/1301421277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474102060 0x7f1474102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.328+0000 7f147b97b640 1 -- 172.21.15.31:0/1301421277 >> 172.21.15.31:0/1301421277 conn(0x7f1474079e90 msgr2=0x7f147407a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.329+0000 7f147b97b640 1 -- 172.21.15.31:0/1301421277 shutdown_connections 2024-09-21T06:26:05.331 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.329+0000 7f147b97b640 1 -- 172.21.15.31:0/1301421277 wait complete. 2024-09-21T06:26:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.330+0000 7f147b97b640 1 Processor -- start 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 -- start start 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1474102060 0x7f14741171f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474107220 0x7f1474117770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474111660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14741099c0 con 0x7f1474107220 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1474109840 con 0x7f1474102060 2024-09-21T06:26:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.331+0000 7f147b97b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1474109b40 con 0x7f1474107b70 2024-09-21T06:26:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1478f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474107220 0x7f1474117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474111660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474111660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59866/0 (socket says 172.21.15.31:59866) 2024-09-21T06:26:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1478f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474107220 0x7f1474117770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45066/0 (socket says 172.21.15.31:45066) 2024-09-21T06:26:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 -- 172.21.15.31:0/64036538 learned_addr learned my addr 172.21.15.31:0/64036538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479717640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1474102060 0x7f14741171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 -- 172.21.15.31:0/64036538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1474102060 msgr2=0x7f14741171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1474102060 0x7f14741171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 -- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474107220 msgr2=0x7f1474117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474107220 0x7f1474117770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 -- 172.21.15.31:0/64036538 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1474111e80 con 0x7f1474107b70 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.332+0000 7f1479f18640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474111660 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f146c031a00 tx=0x7f146c031a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.333+0000 7f146a7fc640 1 -- 172.21.15.31:0/64036538 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c003b70 con 0x7f1474107b70 2024-09-21T06:26:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.333+0000 7f146a7fc640 1 -- 172.21.15.31:0/64036538 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146c003d10 con 0x7f1474107b70 2024-09-21T06:26:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.333+0000 7f146a7fc640 1 -- 172.21.15.31:0/64036538 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c02fac0 con 0x7f1474107b70 2024-09-21T06:26:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.333+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1474112110 con 0x7f1474107b70 2024-09-21T06:26:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.333+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741b5b20 con 0x7f1474107b70 2024-09-21T06:26:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.338+0000 7f14829c7740 1 Processor -- start 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f146a7fc640 1 -- 172.21.15.31:0/64036538 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f146c02fc60 con 0x7f1474107b70 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 -- start start 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a099d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a099def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a099e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c0a09126f0 con 0x55c0a05f81b0 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0a0901b20 con 0x55c0a05cf2f0 2024-09-21T06:26:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f14829c7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c0a0905930 con 0x55c0a05d74d0 2024-09-21T06:26:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a099def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1478f16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a099d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a099def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:53260/0 (socket says 172.21.15.31:53260) 2024-09-21T06:26:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 -- 172.21.15.31:0/3842511720 learned_addr learned my addr 172.21.15.31:0/3842511720 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1478f16640 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a099e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 -- 172.21.15.31:0/3842511720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 msgr2=0x55c0a099e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a099e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 -- 172.21.15.31:0/3842511720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 msgr2=0x55c0a099d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a099d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1479717640 1 -- 172.21.15.31:0/3842511720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a08c7530 con 0x55c0a05cf2f0 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.339+0000 7f1478f16640 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a099e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f146a7fc640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14600b66b0 0x7f14600b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f1479717640 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a099def0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f146400bce0 tx=0x7f14640107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198c388723760987 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f145f7fe640 1 -- 172.21.15.31:0/3842511720 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146401c480 con 0x55c0a05cf2f0 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f1478f16640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14600b66b0 0x7f14600b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f145f7fe640 1 -- 172.21.15.31:0/3842511720 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146401cad0 con 0x55c0a05cf2f0 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f145f7fe640 1 -- 172.21.15.31:0/3842511720 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1464011ca0 con 0x55c0a05cf2f0 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.340+0000 7f146a7fc640 1 -- 172.21.15.31:0/64036538 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f146c108c60 con 0x7f1474107b70 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.341+0000 7f1478f16640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14600b66b0 0x7f14600b8bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1474104d00 tx=0x7f1470009460 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.341+0000 7f14829c7740 1 -- 172.21.15.31:0/3842511720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 msgr2=0x55c0a099def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.341+0000 7f14829c7740 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a099def0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f146400bce0 tx=0x7f14640107a0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 -- 172.21.15.31:0/3842511720 shutdown_connections 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a099e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a099def0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 --2- 172.21.15.31:0/3842511720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a099d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 -- 172.21.15.31:0/3842511720 >> 172.21.15.31:0/3842511720 conn(0x55c0a05ec300 msgr2=0x55c0a099b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 -- 172.21.15.31:0/3842511720 shutdown_connections 2024-09-21T06:26:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.342+0000 7f14829c7740 1 -- 172.21.15.31:0/3842511720 wait complete. 2024-09-21T06:26:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.343+0000 7f14829c7740 1 -- start start 2024-09-21T06:26:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.343+0000 7f14829c7740 1 Processor -- start 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a09a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09a9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a09a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a09a9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45076/0 (socket says 172.21.15.31:45076) 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 -- 172.21.15.31:0/1396544687 learned_addr learned my addr 172.21.15.31:0/1396544687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05f81b0 0x55c0a09b5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55c0a0839ab0 con 0x55c0a05cf2f0 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0a08a7730 con 0x55c0a05f81b0 2024-09-21T06:26:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c0a09ac130 con 0x55c0a05d74d0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1478f16640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479f18640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05f81b0 0x55c0a09b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f14829c7740 10 client.0 _ll_register_callbacks cb 0x55c0a08fb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 -- 172.21.15.31:0/1396544687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 msgr2=0x55c0a09a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 -- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05f81b0 msgr2=0x55c0a09b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05f81b0 0x55c0a09b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.344+0000 7f1479717640 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a0663790 con 0x55c0a05cf2f0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f1479717640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a09a9110 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f146401dec0 tx=0x7f146401def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1ccbc24899d759 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f145cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f145cff9640 1 -- 172.21.15.31:0/1396544687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146402b060 con 0x55c0a05cf2f0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f146401cad0 con 0x55c0a05cf2f0 2024-09-21T06:26:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f145cff9640 1 -- 172.21.15.31:0/1396544687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f146401c690 con 0x55c0a05cf2f0 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f145cff9640 1 -- 172.21.15.31:0/1396544687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f146401c890 con 0x55c0a05cf2f0 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f14829c7740 10 client.18336 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.345+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c0a06dc620 con 0x55c0a05cf2f0 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f145cff9640 1 -- 172.21.15.31:0/1396544687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146401cad0 con 0x55c0a05cf2f0 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f145cff9640 1 -- 172.21.15.31:0/1396544687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1464025070 con 0x55c0a05cf2f0 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f14829c7740 10 client.18336 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f14829c7740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f14829c7740 20 client.18336 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a05b6d80 0x55c0a09bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f14829c7740 4 client.18336 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.346+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0a0914640 con 0x55c0a05b6d80 2024-09-21T06:26:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.347+0000 7f1478f16640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a05b6d80 0x55c0a09bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.348+0000 7f1478f16640 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a05b6d80 0x55c0a09bb0c0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3edec0742cbd19 server_cookie=e9c25f858914af01 in_seq=0 out_seq=0 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.348+0000 7f145cff9640 10 client.18336 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f145cff9640 1 -- 172.21.15.31:0/1396544687 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0a0914640 con 0x55c0a05b6d80 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f145cff9640 10 client.18336 handle_command_reply: tid=0 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 client.18336 shutdown 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 msgr2=0x55c0a09a9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a09a9110 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f146401dec0 tx=0x7f146401def0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 shutdown_connections 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a05b6d80 0x55c0a09bb0c0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05f81b0 0x55c0a09b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 --2- 172.21.15.31:0/1396544687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a09a9110 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f145cff9640 0 client.18336 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 >> 172.21.15.31:0/1396544687 conn(0x55c0a05ec300 msgr2=0x55c0a099e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 shutdown_connections 2024-09-21T06:26:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 1 -- 172.21.15.31:0/1396544687 wait complete. 2024-09-21T06:26:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.349+0000 7f14829c7740 20 client.18336 trim_cache size 0 max 16384 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 Processor -- start 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 -- start start 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a099f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05d74d0 0x55c0a09b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 0x55c0a099aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14741bc5e0 con 0x55c0a05cf2f0 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14741bee40 con 0x55c0a05d74d0 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f14829c7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14741bcc30 con 0x55c0a05f81b0 2024-09-21T06:26:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f1479717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a099f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f1479f18640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 0x55c0a099aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f1479f18640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 0x55c0a099aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59896/0 (socket says 172.21.15.31:59896) 2024-09-21T06:26:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f1479f18640 1 -- 172.21.15.31:0/3440101995 learned_addr learned my addr 172.21.15.31:0/3440101995 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f1479717640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a099f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45084/0 (socket says 172.21.15.31:45084) 2024-09-21T06:26:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.367+0000 7f1478f16640 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05d74d0 0x55c0a09b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479f18640 1 -- 172.21.15.31:0/3440101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05d74d0 msgr2=0x55c0a09b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479f18640 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05d74d0 0x55c0a09b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479f18640 1 -- 172.21.15.31:0/3440101995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 msgr2=0x55c0a099f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479f18640 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a099f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479f18640 1 -- 172.21.15.31:0/3440101995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0a099fcb0 con 0x55c0a05f81b0 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479717640 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a099f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f1479f18640 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 0x55c0a099aeb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f146c0cb030 tx=0x7f146c0f2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e3beae6c68bceb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f145effd640 1 -- 172.21.15.31:0/3440101995 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c0ce570 con 0x55c0a05f81b0 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f145effd640 1 -- 172.21.15.31:0/3440101995 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146c036030 con 0x55c0a05f81b0 2024-09-21T06:26:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.368+0000 7f145effd640 1 -- 172.21.15.31:0/3440101995 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146c0d8480 con 0x55c0a05f81b0 2024-09-21T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.369+0000 7f14829c7740 1 -- 172.21.15.31:0/3440101995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 msgr2=0x55c0a099aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.369+0000 7f14829c7740 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 0x55c0a099aeb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f146c0cb030 tx=0x7f146c0f2040 comp rx=0 tx=0).stop 2024-09-21T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.370+0000 7f14829c7740 1 -- 172.21.15.31:0/3440101995 shutdown_connections 2024-09-21T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.370+0000 7f14829c7740 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05f81b0 0x55c0a099aeb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.370+0000 7f14829c7740 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05d74d0 0x55c0a09b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.370+0000 7f14829c7740 1 --2- 172.21.15.31:0/3440101995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05cf2f0 0x55c0a099f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.370+0000 7f14829c7740 1 -- 172.21.15.31:0/3440101995 >> 172.21.15.31:0/3440101995 conn(0x55c0a05ec300 msgr2=0x55c0a099e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.370+0000 7f14829c7740 1 -- 172.21.15.31:0/3440101995 shutdown_connections 2024-09-21T06:26:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.371+0000 7f14829c7740 1 -- 172.21.15.31:0/3440101995 wait complete. 2024-09-21T06:26:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.371+0000 7f14829c7740 1 -- start start 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.371+0000 7f14829c7740 1 Processor -- start 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a09bb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a0a23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f14741bd300 con 0x55c0a05f81b0 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14741be770 con 0x55c0a05cf2f0 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f14741bd9d0 con 0x55c0a05d74d0 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1478f16640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479717640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a09bb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479f18640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a0a23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1478f16640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09bb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:59902/0 (socket says 172.21.15.31:59902) 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1478f16640 1 -- 172.21.15.31:0/2729256318 learned_addr learned my addr 172.21.15.31:0/2729256318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:05.376 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f14829c7740 10 client.0 _ll_register_callbacks cb 0x7f14700097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479717640 1 -- 172.21.15.31:0/2729256318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 msgr2=0x55c0a09bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479717640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479717640 1 -- 172.21.15.31:0/2729256318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 msgr2=0x55c0a0a23830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479717640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a0a23830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479717640 1 -- 172.21.15.31:0/2729256318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0a09bc180 con 0x55c0a05cf2f0 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1478f16640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.372+0000 7f1479f18640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a0a23830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f1479717640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a09bb3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1464011010 tx=0x7f1464004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b894d3488325f6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f145cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f145cff9640 1 -- 172.21.15.31:0/2729256318 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146403c020 con 0x55c0a05cf2f0 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f145cff9640 1 -- 172.21.15.31:0/2729256318 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f146401cc40 con 0x55c0a05cf2f0 2024-09-21T06:26:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f146c036030 con 0x55c0a05cf2f0 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f145cff9640 1 -- 172.21.15.31:0/2729256318 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1464047050 con 0x55c0a05cf2f0 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f14829c7740 10 client.28136 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c0a0a24d40 con 0x55c0a05cf2f0 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f145cff9640 1 -- 172.21.15.31:0/2729256318 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f146c036030 con 0x55c0a05cf2f0 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f145cff9640 1 -- 172.21.15.31:0/2729256318 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1464025040 con 0x55c0a05cf2f0 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.373+0000 7f14829c7740 10 client.28136 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f14829c7740 10 client.28136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f14829c7740 20 client.28136 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f14829c7740 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a058c1c0 0x55c0a0a27490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f14829c7740 4 client.28136 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0a09100f0 con 0x55c0a058c1c0 2024-09-21T06:26:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f1478f16640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a058c1c0 0x55c0a0a27490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f1478f16640 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a058c1c0 0x55c0a0a27490 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffafd51262dfe741 server_cookie=1af29809eded6e2f in_seq=0 out_seq=0 2024-09-21T06:26:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.374+0000 7f145cff9640 10 client.28136 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f145cff9640 1 -- 172.21.15.31:0/2729256318 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0a09100f0 con 0x55c0a058c1c0 2024-09-21T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f145cff9640 10 client.28136 handle_command_reply: tid=0 2024-09-21T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 client.28136 shutdown 2024-09-21T06:26:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 msgr2=0x55c0a09bb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a09bb3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1464011010 tx=0x7f1464004b10 comp rx=0 tx=0).stop 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 shutdown_connections 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55c0a058c1c0 0x55c0a0a27490 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c0a05f81b0 0x55c0a0a23830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c0a05d74d0 0x55c0a09bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 --2- 172.21.15.31:0/2729256318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0a05cf2f0 0x55c0a09bb3c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f145cff9640 0 client.28136 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 >> 172.21.15.31:0/2729256318 conn(0x55c0a05ec300 msgr2=0x55c0a099e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 shutdown_connections 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 1 -- 172.21.15.31:0/2729256318 wait complete. 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.405+0000 7f14829c7740 20 client.28136 trim_cache size 0 max 16384 2024-09-21T06:26:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:26:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14600b66b0 msgr2=0x7f14600b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14600b66b0 0x7f14600b8bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1474104d00 tx=0x7f1470009460 comp rx=0 tx=0).stop 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 msgr2=0x7f1474111660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474111660 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f146c031a00 tx=0x7f146c031a30 comp rx=0 tx=0).stop 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 shutdown_connections 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1474107b70 0x7f1474111660 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f14600b66b0 0x7f14600b8bc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1474107220 0x7f1474117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 --2- 172.21.15.31:0/64036538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1474102060 0x7f14741171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 >> 172.21.15.31:0/64036538 conn(0x7f1474079e90 msgr2=0x7f1474100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 shutdown_connections 2024-09-21T06:26:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:05.406+0000 7f147b97b640 1 -- 172.21.15.31:0/64036538 wait complete. 2024-09-21T06:26:05.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:05 smithi110 ceph-mon[34550]: pgmap v771: 97 pgs: 97 active+clean; 4.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-21T06:26:05.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:05 smithi107 ceph-mon[34380]: pgmap v771: 97 pgs: 97 active+clean; 4.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-21T06:26:05.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:05 smithi031 ceph-mon[31237]: pgmap v771: 97 pgs: 97 active+clean; 4.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-21T06:26:07.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:07 smithi110 ceph-mon[34550]: pgmap v772: 97 pgs: 97 active+clean; 4.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 1.50k op/s 2024-09-21T06:26:07.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:07 smithi107 ceph-mon[34380]: pgmap v772: 97 pgs: 97 active+clean; 4.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 1.50k op/s 2024-09-21T06:26:07.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:07 smithi031 ceph-mon[31237]: pgmap v772: 97 pgs: 97 active+clean; 4.5 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 22 MiB/s wr, 1.50k op/s 2024-09-21T06:26:08.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:08 smithi110 ceph-mon[34550]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T06:26:08.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:08 smithi107 ceph-mon[34380]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T06:26:08.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:08 smithi031 ceph-mon[31237]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T06:26:09.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:09 smithi110 ceph-mon[34550]: pgmap v774: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 MiB/s wr, 1.98k op/s 2024-09-21T06:26:09.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:09 smithi107 ceph-mon[34380]: pgmap v774: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 MiB/s wr, 1.98k op/s 2024-09-21T06:26:09.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:09 smithi031 ceph-mon[31237]: pgmap v774: 97 pgs: 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 91 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 30 MiB/s wr, 1.98k op/s 2024-09-21T06:26:10.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 870.0 s, 667.1 tps, lat 77.519 ms stddev 128.532 2024-09-21T06:26:11.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:11 smithi110 ceph-mon[34550]: pgmap v775: 97 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 63 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 29 MiB/s wr, 1.92k op/s 2024-09-21T06:26:11.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:11 smithi107 ceph-mon[34380]: pgmap v775: 97 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 63 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 29 MiB/s wr, 1.92k op/s 2024-09-21T06:26:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:11 smithi031 ceph-mon[31237]: pgmap v775: 97 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 63 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 29 MiB/s wr, 1.92k op/s 2024-09-21T06:26:13.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:13 smithi110 ceph-mon[34550]: pgmap v776: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 28 MiB/s wr, 1.85k op/s 2024-09-21T06:26:13.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:13 smithi107 ceph-mon[34380]: pgmap v776: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 28 MiB/s wr, 1.85k op/s 2024-09-21T06:26:13.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:13 smithi031 ceph-mon[31237]: pgmap v776: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 4.4 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 28 MiB/s wr, 1.85k op/s 2024-09-21T06:26:15.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 875.0 s, 662.7 tps, lat 223.399 ms stddev 589.241 2024-09-21T06:26:15.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:15 smithi110 ceph-mon[34550]: pgmap v777: 97 pgs: 33 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 45 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-09-21T06:26:15.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:15 smithi107 ceph-mon[34380]: pgmap v777: 97 pgs: 33 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 45 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-09-21T06:26:15.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:15 smithi031 ceph-mon[31237]: pgmap v777: 97 pgs: 33 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 45 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 1.15k op/s 2024-09-21T06:26:17.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:17 smithi110 ceph-mon[34550]: pgmap v778: 97 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 57 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-21T06:26:17.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:17 smithi107 ceph-mon[34380]: pgmap v778: 97 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 57 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-21T06:26:17.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:17 smithi031 ceph-mon[31237]: pgmap v778: 97 pgs: 28 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 57 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 -- 172.21.15.31:0/2206359925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15201029b0 msgr2=0x7f1520102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 --2- 172.21.15.31:0/2206359925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15201029b0 0x7f1520102dc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f151c009900 tx=0x7f151c02a410 comp rx=0 tx=0).stop 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 -- 172.21.15.31:0/2206359925 shutdown_connections 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 --2- 172.21.15.31:0/2206359925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1520108c30 0x7f152010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 --2- 172.21.15.31:0/2206359925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15201082e0 0x7f15201086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 --2- 172.21.15.31:0/2206359925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15201029b0 0x7f1520102dc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 -- 172.21.15.31:0/2206359925 >> 172.21.15.31:0/2206359925 conn(0x7f152007a740 msgr2=0x7f152007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 -- 172.21.15.31:0/2206359925 shutdown_connections 2024-09-21T06:26:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.551+0000 7f152809b640 1 -- 172.21.15.31:0/2206359925 wait complete. 2024-09-21T06:26:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.552+0000 7f152809b640 1 Processor -- start 2024-09-21T06:26:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.552+0000 7f152809b640 1 -- start start 2024-09-21T06:26:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f152809b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 0x7f15201a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f152809b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15201082e0 0x7f15201a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f152809b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520108c30 0x7f15201a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f152809b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15201128f0 con 0x7f1520108c30 2024-09-21T06:26:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f152809b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1520112770 con 0x7f15201029b0 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f152809b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1520112a70 con 0x7f15201082e0 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 0x7f15201a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1526638640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520108c30 0x7f15201a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 0x7f15201a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46546/0 (socket says 172.21.15.31:46546) 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 -- 172.21.15.31:0/3594360458 learned_addr learned my addr 172.21.15.31:0/3594360458 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525636640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15201082e0 0x7f15201a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 -- 172.21.15.31:0/3594360458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15201082e0 msgr2=0x7f15201a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15201082e0 0x7f15201a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 -- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520108c30 msgr2=0x7f15201a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520108c30 0x7f15201a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1525e37640 1 -- 172.21.15.31:0/3594360458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15201a6a20 con 0x7f15201029b0 2024-09-21T06:26:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.553+0000 7f1526638640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520108c30 0x7f15201a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.554+0000 7f1525e37640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 0x7f15201a0af0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f151c005940 tx=0x7f151c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.554+0000 7f1516ffd640 1 -- 172.21.15.31:0/3594360458 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c03f030 con 0x7f15201029b0 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.554+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15201a6cb0 con 0x7f15201029b0 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.554+0000 7f1516ffd640 1 -- 172.21.15.31:0/3594360458 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c003aa0 con 0x7f15201029b0 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.554+0000 7f1516ffd640 1 -- 172.21.15.31:0/3594360458 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035070 con 0x7f15201029b0 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.554+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15201ad4a0 con 0x7f15201029b0 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.556+0000 7f152f00a740 1 Processor -- start 2024-09-21T06:26:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 -- start start 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 0x560f6b2b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f1516ffd640 1 -- 172.21.15.31:0/3594360458 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f151c042030 con 0x7f15201029b0 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b2b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aeec4d0 0x560f6b2b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560f6b2276f0 con 0x560f6aee42f0 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f6b216b20 con 0x560f6af0d1b0 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f152f00a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560f6b21a930 con 0x560f6aeec4d0 2024-09-21T06:26:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f1526638640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 0x560f6b2b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f1526638640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 0x560f6b2b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46556/0 (socket says 172.21.15.31:46556) 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f1526638640 1 -- 172.21.15.31:0/1430552081 learned_addr learned my addr 172.21.15.31:0/1430552081 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f1525e37640 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b2b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.557+0000 7f1525636640 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aeec4d0 0x560f6b2b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1526638640 1 -- 172.21.15.31:0/1430552081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aeec4d0 msgr2=0x560f6b2b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1526638640 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aeec4d0 0x560f6b2b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1526638640 1 -- 172.21.15.31:0/1430552081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 msgr2=0x560f6b2b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1526638640 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b2b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1526638640 1 -- 172.21.15.31:0/1430552081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6b1dc530 con 0x560f6af0d1b0 2024-09-21T06:26:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1526638640 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 0x560f6b2b29b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f15180117c0 tx=0x7f1518011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04056ffbf219c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.558+0000 7f1516ffd640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f150c0b6680 0x7f150c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.560+0000 7f1516ffd640 1 -- 172.21.15.31:0/3594360458 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8543+0+0 (secure 0 0 0) 0x7f151c107dc0 con 0x7f15201029b0 2024-09-21T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.560+0000 7f150bfff640 1 -- 172.21.15.31:0/1430552081 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518003c30 con 0x560f6af0d1b0 2024-09-21T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.560+0000 7f150bfff640 1 -- 172.21.15.31:0/1430552081 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151800fe50 con 0x560f6af0d1b0 2024-09-21T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.560+0000 7f150bfff640 1 -- 172.21.15.31:0/1430552081 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15180193e0 con 0x560f6af0d1b0 2024-09-21T06:26:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.560+0000 7f1525e37640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f150c0b6680 0x7f150c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 -- 172.21.15.31:0/1430552081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 msgr2=0x560f6b2b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 0x560f6b2b29b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f15180117c0 tx=0x7f1518011c80 comp rx=0 tx=0).stop 2024-09-21T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f1525e37640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f150c0b6680 0x7f150c0b8b90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f151c1055b0 tx=0x7f151c10b910 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 -- 172.21.15.31:0/1430552081 shutdown_connections 2024-09-21T06:26:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aeec4d0 0x560f6b2b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b2b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 --2- 172.21.15.31:0/1430552081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6af0d1b0 0x560f6b2b29b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 -- 172.21.15.31:0/1430552081 >> 172.21.15.31:0/1430552081 conn(0x560f6af01300 msgr2=0x560f6b2b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.561+0000 7f152f00a740 1 -- 172.21.15.31:0/1430552081 shutdown_connections 2024-09-21T06:26:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.562+0000 7f152f00a740 1 -- 172.21.15.31:0/1430552081 wait complete. 2024-09-21T06:26:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.562+0000 7f152f00a740 1 -- start start 2024-09-21T06:26:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.562+0000 7f152f00a740 1 Processor -- start 2024-09-21T06:26:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.562+0000 7f152f00a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.562+0000 7f152f00a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f152f00a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6af0d1b0 0x560f6b2ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f152f00a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560f6b14eab0 con 0x560f6af0d1b0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f152f00a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f6b1bc730 con 0x560f6aeec4d0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f152f00a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560f6b2c1130 con 0x560f6aee42f0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2be690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:46572/0 (socket says 172.21.15.31:46572) 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 -- 172.21.15.31:0/650158543 learned_addr learned my addr 172.21.15.31:0/650158543 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f152f00a740 10 client.0 _ll_register_callbacks cb 0x560f6b2106b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1526638640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1526638640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6af0d1b0 0x560f6b2ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 -- 172.21.15.31:0/650158543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 msgr2=0x560f6b2be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2be150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 -- 172.21.15.31:0/650158543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6af0d1b0 msgr2=0x560f6b2ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6af0d1b0 0x560f6b2ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 -- 172.21.15.31:0/650158543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6af78790 con 0x560f6aeec4d0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f1525636640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2be690 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f151000be20 tx=0x7f1510012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27aa580da24d0863 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.563+0000 7f15097fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f15097fa640 1 -- 172.21.15.31:0/650158543 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510005220 con 0x560f6aeec4d0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f15097fa640 1 -- 172.21.15.31:0/650158543 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8543+0+0 (secure 0 0 0) 0x7f15100053c0 con 0x560f6aeec4d0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151800fe50 con 0x560f6aeec4d0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f15097fa640 1 -- 172.21.15.31:0/650158543 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f15100055c0 con 0x560f6aeec4d0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 10 client.28148 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560f6aff1620 con 0x560f6aeec4d0 2024-09-21T06:26:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f15097fa640 1 -- 172.21.15.31:0/650158543 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151800fe50 con 0x560f6aeec4d0 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f15097fa640 1 -- 172.21.15.31:0/650158543 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1510010b80 con 0x560f6aeec4d0 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 10 client.28148 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 10 client.28148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 20 client.28148 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6aecbd80 0x560f6b2d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 4 client.28148 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f6b229640 con 0x560f6aecbd80 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.564+0000 7f1526638640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6aecbd80 0x560f6b2d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.565+0000 7f15097fa640 10 client.28148 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.565+0000 7f1526638640 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6aecbd80 0x560f6b2d02f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4396533ef60e9923 server_cookie=49a80c2548cbc3d8 in_seq=0 out_seq=0 2024-09-21T06:26:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.566+0000 7f15097fa640 1 -- 172.21.15.31:0/650158543 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f6b229640 con 0x560f6aecbd80 2024-09-21T06:26:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.566+0000 7f15097fa640 10 client.28148 handle_command_reply: tid=0 2024-09-21T06:26:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.566+0000 7f152f00a740 1 client.28148 shutdown 2024-09-21T06:26:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.566+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 msgr2=0x560f6b2be690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.566+0000 7f152f00a740 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2be690 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f151000be20 tx=0x7f1510012820 comp rx=0 tx=0).stop 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 shutdown_connections 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6aecbd80 0x560f6b2d02f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6af0d1b0 0x560f6b2ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2be690 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f15097fa640 0 client.28148 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 --2- 172.21.15.31:0/650158543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 >> 172.21.15.31:0/650158543 conn(0x560f6af01300 msgr2=0x560f6b2b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 shutdown_connections 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 1 -- 172.21.15.31:0/650158543 wait complete. 2024-09-21T06:26:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.567+0000 7f152f00a740 20 client.28148 trim_cache size 0 max 16384 2024-09-21T06:26:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 Processor -- start 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 -- start start 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aecbd80 0x560f6b2be630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b3d60 con 0x560f6aecbd80 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b65c0 con 0x560f6aeec4d0 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.584+0000 7f152f00a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b43b0 con 0x560f6aee42f0 2024-09-21T06:26:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:36934/0 (socket says 172.21.15.31:36934) 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 -- 172.21.15.31:0/1204567532 learned_addr learned my addr 172.21.15.31:0/1204567532 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 -- 172.21.15.31:0/1204567532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 msgr2=0x560f6b2ca320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2ca320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 -- 172.21.15.31:0/1204567532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aecbd80 msgr2=0x560f6b2be630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aecbd80 0x560f6b2be630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 -- 172.21.15.31:0/1204567532 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6b2b4cb0 con 0x560f6aee42f0 2024-09-21T06:26:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.585+0000 7f1525636640 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2b40f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f15100057c0 tx=0x7f1510012e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1701cde692e2d161 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.586+0000 7f150b7fe640 1 -- 172.21.15.31:0/1204567532 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510031070 con 0x560f6aee42f0 2024-09-21T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.586+0000 7f150b7fe640 1 -- 172.21.15.31:0/1204567532 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151001c5f0 con 0x560f6aee42f0 2024-09-21T06:26:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.586+0000 7f150b7fe640 1 -- 172.21.15.31:0/1204567532 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510028070 con 0x560f6aee42f0 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 -- 172.21.15.31:0/1204567532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 msgr2=0x560f6b2b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2b40f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f15100057c0 tx=0x7f1510012e20 comp rx=0 tx=0).stop 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 -- 172.21.15.31:0/1204567532 shutdown_connections 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aee42f0 0x560f6b2b40f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 --2- 172.21.15.31:0/1204567532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aecbd80 0x560f6b2be630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 -- 172.21.15.31:0/1204567532 >> 172.21.15.31:0/1204567532 conn(0x560f6af01300 msgr2=0x560f6b2b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 -- 172.21.15.31:0/1204567532 shutdown_connections 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.587+0000 7f152f00a740 1 -- 172.21.15.31:0/1204567532 wait complete. 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 -- start start 2024-09-21T06:26:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 Processor -- start 2024-09-21T06:26:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aecbd80 0x560f6b334ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b3350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b4a80 con 0x560f6aee42f0 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b5ef0 con 0x560f6aeec4d0 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b5150 con 0x560f6aecbd80 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f152f00a740 10 client.0 _ll_register_callbacks cb 0x7f151000ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.588+0000 7f1525636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b3350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1526638640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aecbd80 0x560f6b334ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1526638640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aecbd80 0x560f6b334ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:36942/0 (socket says 172.21.15.31:36942) 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1526638640 1 -- 172.21.15.31:0/3412145502 learned_addr learned my addr 172.21.15.31:0/3412145502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1526638640 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1525636640 1 -- 172.21.15.31:0/3412145502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aecbd80 msgr2=0x560f6b334ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1525636640 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aecbd80 0x560f6b334ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1525636640 1 -- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 msgr2=0x560f6b2c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1525636640 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1525636640 1 -- 172.21.15.31:0/3412145502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f6b2c59e0 con 0x560f6aee42f0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f1525636640 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b3350a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1510005d00 tx=0x7f151001ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3332ea37db18f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f15097fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f15097fa640 1 -- 172.21.15.31:0/3412145502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510028070 con 0x560f6aee42f0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151001c5f0 con 0x560f6aee42f0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.589+0000 7f15097fa640 1 -- 172.21.15.31:0/3412145502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8543+0+0 (secure 0 0 0) 0x7f1510004020 con 0x560f6aee42f0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f15097fa640 1 -- 172.21.15.31:0/3412145502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1510043050 con 0x560f6aee42f0 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 10 client.18357 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560f6b2c6c70 con 0x560f6aee42f0 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f15097fa640 1 -- 172.21.15.31:0/3412145502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151001c5f0 con 0x560f6aee42f0 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f15097fa640 1 -- 172.21.15.31:0/3412145502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f15100037f0 con 0x560f6aee42f0 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 10 client.18357 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 10 client.18357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 20 client.18357 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6af0d1b0 0x560f6b34dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 4 client.18357 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f6b2250f0 con 0x560f6af0d1b0 2024-09-21T06:26:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.590+0000 7f1526638640 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6af0d1b0 0x560f6b34dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.591+0000 7f1526638640 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6af0d1b0 0x560f6b34dd90 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543d214721b45087 server_cookie=1f253be38e695d82 in_seq=0 out_seq=0 2024-09-21T06:26:18.596 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.591+0000 7f15097fa640 10 client.18357 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f15097fa640 1 -- 172.21.15.31:0/3412145502 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f6b2250f0 con 0x560f6af0d1b0 2024-09-21T06:26:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f15097fa640 10 client.18357 handle_command_reply: tid=0 2024-09-21T06:26:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 client.18357 shutdown 2024-09-21T06:26:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 msgr2=0x560f6b3350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b3350a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1510005d00 tx=0x7f151001ce60 comp rx=0 tx=0).stop 2024-09-21T06:26:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 shutdown_connections 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x560f6af0d1b0 0x560f6b34dd90 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f6aeec4d0 0x560f6b2c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560f6aee42f0 0x560f6b3350a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 --2- 172.21.15.31:0/3412145502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f6aecbd80 0x560f6b334ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 >> 172.21.15.31:0/3412145502 conn(0x560f6af01300 msgr2=0x560f6b2b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f15097fa640 0 client.18357 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 shutdown_connections 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 1 -- 172.21.15.31:0/3412145502 wait complete. 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.618+0000 7f152f00a740 20 client.18357 trim_cache size 0 max 16384 2024-09-21T06:26:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:26:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:26:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:26:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f150c0b6680 msgr2=0x7f150c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f150c0b6680 0x7f150c0b8b90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f151c1055b0 tx=0x7f151c10b910 comp rx=0 tx=0).stop 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 msgr2=0x7f15201a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 0x7f15201a0af0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f151c005940 tx=0x7f151c004270 comp rx=0 tx=0).stop 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 shutdown_connections 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f150c0b6680 0x7f150c0b8b90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1520108c30 0x7f15201a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15201082e0 0x7f15201a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 --2- 172.21.15.31:0/3594360458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15201029b0 0x7f15201a0af0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 >> 172.21.15.31:0/3594360458 conn(0x7f152007a740 msgr2=0x7f152006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 shutdown_connections 2024-09-21T06:26:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:18.619+0000 7f152809b640 1 -- 172.21.15.31:0/3594360458 wait complete. 2024-09-21T06:26:19.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:19 smithi110 ceph-mon[34550]: pgmap v779: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.1 MiB/s wr, 449 op/s 2024-09-21T06:26:19.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:19 smithi107 ceph-mon[34380]: pgmap v779: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.1 MiB/s wr, 449 op/s 2024-09-21T06:26:19.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:19 smithi031 ceph-mon[31237]: pgmap v779: 97 pgs: 12 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 81 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.1 MiB/s wr, 449 op/s 2024-09-21T06:26:20.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 880.0 s, 1257.0 tps, lat 78.862 ms stddev 98.954 2024-09-21T06:26:21.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:21 smithi110 ceph-mon[34550]: pgmap v780: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.5 MiB/s wr, 470 op/s 2024-09-21T06:26:21.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:21 smithi107 ceph-mon[34380]: pgmap v780: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.5 MiB/s wr, 470 op/s 2024-09-21T06:26:21.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:21 smithi031 ceph-mon[31237]: pgmap v780: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.5 MiB/s wr, 470 op/s 2024-09-21T06:26:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:23 smithi107 ceph-mon[34380]: pgmap v781: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 479 op/s 2024-09-21T06:26:23.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:23 smithi107 ceph-mon[34380]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T06:26:23.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:23 smithi031 ceph-mon[31237]: pgmap v781: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 479 op/s 2024-09-21T06:26:23.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:23 smithi031 ceph-mon[31237]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T06:26:24.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:23 smithi110 ceph-mon[34550]: pgmap v781: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 MiB/s wr, 479 op/s 2024-09-21T06:26:24.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:23 smithi110 ceph-mon[34550]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T06:26:24.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:26:24.972 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 885.0 s, 1301.0 tps, lat 76.677 ms stddev 93.661 2024-09-21T06:26:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.561+0000 7ff2e3c31640 1 -- 172.21.15.31:0/3147270405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 msgr2=0x7ff2dc10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.561+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/3147270405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc10f520 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009860 tx=0x7ff2cc02a340 comp rx=0 tx=0).stop 2024-09-21T06:26:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 -- 172.21.15.31:0/3147270405 shutdown_connections 2024-09-21T06:26:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/3147270405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc10f520 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/3147270405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/3147270405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2dc102a60 0x7ff2dc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 -- 172.21.15.31:0/3147270405 >> 172.21.15.31:0/3147270405 conn(0x7ff2dc079730 msgr2=0x7ff2dc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 -- 172.21.15.31:0/3147270405 shutdown_connections 2024-09-21T06:26:25.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.562+0000 7ff2e3c31640 1 -- 172.21.15.31:0/3147270405 wait complete. 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.563+0000 7ff2e3c31640 1 Processor -- start 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.563+0000 7ff2e3c31640 1 -- start start 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e3c31640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 0x7ff2dc10fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e3c31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e3c31640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e3c31640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc113a70 con 0x7ff2dc1083a0 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e3c31640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1138f0 con 0x7ff2dc108cf0 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e3c31640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc113bf0 con 0x7ff2dc102a60 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 0x7ff2dc10fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e11cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 0x7ff2dc10fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44026/0 (socket says 172.21.15.31:44026) 2024-09-21T06:26:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 -- 172.21.15.31:0/2578031279 learned_addr learned my addr 172.21.15.31:0/2578031279 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:25.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 -- 172.21.15.31:0/2578031279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 msgr2=0x7ff2dc110830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc110830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 -- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2dc1083a0 msgr2=0x7ff2dc1102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.564+0000 7ff2e19cd640 1 -- 172.21.15.31:0/2578031279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc1a86a0 con 0x7ff2dc102a60 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.565+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 0x7ff2dc10fdb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800e9c0 tx=0x7ff2d800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.565+0000 7ff2d2ffd640 1 -- 172.21.15.31:0/2578031279 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800cde0 con 0x7ff2dc102a60 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.565+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1a8990 con 0x7ff2dc102a60 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.565+0000 7ff2d2ffd640 1 -- 172.21.15.31:0/2578031279 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d80035f0 con 0x7ff2dc102a60 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.565+0000 7ff2d2ffd640 1 -- 172.21.15.31:0/2578031279 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8010690 con 0x7ff2dc102a60 2024-09-21T06:26:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.565+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1a8f20 con 0x7ff2dc102a60 2024-09-21T06:26:25.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.569+0000 7ff2ea1a5740 1 Processor -- start 2024-09-21T06:26:25.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.569+0000 7ff2ea1a5740 1 -- start start 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.569+0000 7ff2d2ffd640 1 -- 172.21.15.31:0/2578031279 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff2d8004000 con 0x7ff2dc102a60 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.569+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e41650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.569+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e41b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.569+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e420d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557ae6db7e50 con 0x557ae6a711b0 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ae6da7280 con 0x557ae6a9a070 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557ae6dab090 con 0x557ae6a79390 2024-09-21T06:26:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2e19cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e41b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2e19cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e41b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52184/0 (socket says 172.21.15.31:52184) 2024-09-21T06:26:25.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2e19cd640 1 -- 172.21.15.31:0/3676358237 learned_addr learned my addr 172.21.15.31:0/3676358237 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:25.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2e11cc640 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.570+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e41650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.571+0000 7ff2d2ffd640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff2c80b6700 0x7ff2c80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2d2ffd640 1 -- 172.21.15.31:0/2578031279 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7ff2d80e79b0 con 0x7ff2dc102a60 2024-09-21T06:26:25.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff2c80b6700 0x7ff2c80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3676358237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 msgr2=0x557ae6e420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3676358237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 msgr2=0x557ae6e41b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e41b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.572+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3676358237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae6b05650 con 0x557ae6a9a070 2024-09-21T06:26:25.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.573+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e41650 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc02a930 tx=0x7ff2cc030f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70ea246c748bb05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.573+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff2c80b6700 0x7ff2c80b8c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc02a860 tx=0x7ff2cc045000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.574+0000 7ff2c7fff640 1 -- 172.21.15.31:0/3676358237 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc04e070 con 0x557ae6a9a070 2024-09-21T06:26:25.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.574+0000 7ff2c7fff640 1 -- 172.21.15.31:0/3676358237 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc003aa0 con 0x557ae6a9a070 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.575+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3676358237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 msgr2=0x557ae6e41650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.575+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e41650 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc02a930 tx=0x7ff2cc030f60 comp rx=0 tx=0).stop 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.576+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3676358237 shutdown_connections 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.576+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.576+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e41b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.576+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3676358237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e41650 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.576+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3676358237 >> 172.21.15.31:0/3676358237 conn(0x557ae6a8e460 msgr2=0x557ae6e3f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.580+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3676358237 shutdown_connections 2024-09-21T06:26:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.582+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3676358237 wait complete. 2024-09-21T06:26:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.582+0000 7ff2ea1a5740 1 -- start start 2024-09-21T06:26:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.582+0000 7ff2ea1a5740 1 Processor -- start 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a711b0 0x557ae6e4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557ae6c08700 con 0x557ae6a79390 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ae6d42110 con 0x557ae6a9a070 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557ae6e4fe20 con 0x557ae6a711b0 2024-09-21T06:26:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.585+0000 7ff2ea1a5740 10 client.0 _ll_register_callbacks cb 0x557ae6d5b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e4d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55546/0 (socket says 172.21.15.31:55546) 2024-09-21T06:26:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 -- 172.21.15.31:0/200105872 learned_addr learned my addr 172.21.15.31:0/200105872 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:25.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 -- 172.21.15.31:0/200105872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a711b0 msgr2=0x557ae6e4cc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:26:25.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a711b0 0x557ae6e4cc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 -- 172.21.15.31:0/200105872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 msgr2=0x557ae6e4d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.586+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.587+0000 7ff2e21ce640 1 -- 172.21.15.31:0/200105872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae6d6fc40 con 0x557ae6a9a070 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.587+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e4d710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc035040 tx=0x7ff2cc0324e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03167df9674658 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.587+0000 7ff2c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.587+0000 7ff2c57fa640 1 -- 172.21.15.31:0/200105872 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc04e050 con 0x557ae6a9a070 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.588+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2cc003aa0 con 0x557ae6a9a070 2024-09-21T06:26:25.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.588+0000 7ff2c57fa640 1 -- 172.21.15.31:0/200105872 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7ff2cc003c50 con 0x557ae6a9a070 2024-09-21T06:26:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.588+0000 7ff2c57fa640 1 -- 172.21.15.31:0/200105872 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff2cc05f050 con 0x557ae6a9a070 2024-09-21T06:26:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.590+0000 7ff2ea1a5740 10 client.28163 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:25.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.590+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557ae6e5baf0 con 0x557ae6a9a070 2024-09-21T06:26:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.590+0000 7ff2c57fa640 1 -- 172.21.15.31:0/200105872 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc003aa0 con 0x557ae6a9a070 2024-09-21T06:26:25.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.590+0000 7ff2c57fa640 1 -- 172.21.15.31:0/200105872 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff2cc048040 con 0x557ae6a9a070 2024-09-21T06:26:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.591+0000 7ff2ea1a5740 10 client.28163 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.591+0000 7ff2ea1a5740 10 client.28163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.591+0000 7ff2ea1a5740 20 client.28163 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.591+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a589f0 0x557ae6e5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.591+0000 7ff2ea1a5740 4 client.28163 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:25.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.591+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ae6db9da0 con 0x557ae6a589f0 2024-09-21T06:26:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.597+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a589f0 0x557ae6e5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.619+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a589f0 0x557ae6e5f310 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f0a3fa85795190 server_cookie=82ad68e202ce48ea in_seq=0 out_seq=0 2024-09-21T06:26:25.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.619+0000 7ff2c57fa640 10 client.28163 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.623+0000 7ff2c57fa640 1 -- 172.21.15.31:0/200105872 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ae6db9da0 con 0x557ae6a589f0 2024-09-21T06:26:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.623+0000 7ff2c57fa640 10 client.28163 handle_command_reply: tid=0 2024-09-21T06:26:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.623+0000 7ff2ea1a5740 1 client.28163 shutdown 2024-09-21T06:26:25.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.626+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 msgr2=0x557ae6e4d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.626+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e4d710 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc035040 tx=0x7ff2cc0324e0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 shutdown_connections 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a589f0 0x557ae6e5f310 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e4d710 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/200105872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a711b0 0x557ae6e4cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 >> 172.21.15.31:0/200105872 conn(0x557ae6a8e460 msgr2=0x557ae6e44830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2c57fa640 0 client.28163 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 shutdown_connections 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/200105872 wait complete. 2024-09-21T06:26:25.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.630+0000 7ff2ea1a5740 20 client.28163 trim_cache size 0 max 16384 2024-09-21T06:26:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.664+0000 7ff2ea1a5740 1 Processor -- start 2024-09-21T06:26:25.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:25 smithi031 ceph-mon[31237]: pgmap v783: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 552 op/s 2024-09-21T06:26:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.665+0000 7ff2ea1a5740 1 -- start start 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.666+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a711b0 0x557ae6e43040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.666+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.666+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 0x557ae6e49e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.666+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1af8a0 con 0x557ae6a79390 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.666+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b2100 con 0x557ae6a711b0 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.666+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1afef0 con 0x557ae6a9a070 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.667+0000 7ff2e11cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.667+0000 7ff2e21ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 0x557ae6e49e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.667+0000 7ff2e21ce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 0x557ae6e49e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44050/0 (socket says 172.21.15.31:44050) 2024-09-21T06:26:25.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.667+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3380181212 learned_addr learned my addr 172.21.15.31:0/3380181212 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:25.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.667+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a711b0 0x557ae6e43040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.669+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3380181212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a711b0 msgr2=0x557ae6e43040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.669+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a711b0 0x557ae6e43040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.669+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3380181212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 msgr2=0x557ae6e4e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.669+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.669+0000 7ff2e21ce640 1 -- 172.21.15.31:0/3380181212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae6e42470 con 0x557ae6a9a070 2024-09-21T06:26:25.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.676+0000 7ff2e21ce640 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 0x557ae6e49e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009860 tx=0x7ff2cc030380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fcf416ae06464dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.684+0000 7ff2c77fe640 1 -- 172.21.15.31:0/3380181212 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc000e60 con 0x557ae6a9a070 2024-09-21T06:26:25.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.684+0000 7ff2c77fe640 1 -- 172.21.15.31:0/3380181212 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc056070 con 0x557ae6a9a070 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.684+0000 7ff2c77fe640 1 -- 172.21.15.31:0/3380181212 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc04e040 con 0x557ae6a9a070 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3380181212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 msgr2=0x557ae6e49e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 0x557ae6e49e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009860 tx=0x7ff2cc030380 comp rx=0 tx=0).stop 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3380181212 shutdown_connections 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a9a070 0x557ae6e49e60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a79390 0x557ae6e4e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/3380181212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a711b0 0x557ae6e43040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.685+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3380181212 >> 172.21.15.31:0/3380181212 conn(0x557ae6a8e460 msgr2=0x557ae6e4e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.686+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3380181212 shutdown_connections 2024-09-21T06:26:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.686+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/3380181212 wait complete. 2024-09-21T06:26:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.687+0000 7ff2ea1a5740 1 -- start start 2024-09-21T06:26:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.687+0000 7ff2ea1a5740 1 Processor -- start 2024-09-21T06:26:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e50e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b05c0 con 0x557ae6a711b0 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b1a30 con 0x557ae6a9a070 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1b0c90 con 0x557ae6a79390 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2e19cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2e11cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e50e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2e19cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52222/0 (socket says 172.21.15.31:52222) 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2e19cd640 1 -- 172.21.15.31:0/1082421851 learned_addr learned my addr 172.21.15.31:0/1082421851 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.688+0000 7ff2ea1a5740 10 client.0 _ll_register_callbacks cb 0x7ff2cc04c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2e19cd640 1 -- 172.21.15.31:0/1082421851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 msgr2=0x557ae6e50e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e50e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2e19cd640 1 -- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 msgr2=0x557ae6e514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2e19cd640 1 -- 172.21.15.31:0/1082421851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae6e59060 con 0x557ae6a711b0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2e19cd640 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e559c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80e51c0 tx=0x7ff2d80eb050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacc07ff07152d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.689+0000 7ff2c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.690+0000 7ff2c57fa640 1 -- 172.21.15.31:0/1082421851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80e7e60 con 0x557ae6a711b0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.690+0000 7ff2c57fa640 1 -- 172.21.15.31:0/1082421851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7ff2d80e5390 con 0x557ae6a711b0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.690+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2cc056070 con 0x557ae6a711b0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.691+0000 7ff2c57fa640 1 -- 172.21.15.31:0/1082421851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff2d80ad5e0 con 0x557ae6a711b0 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.694+0000 7ff2ea1a5740 10 client.18372 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.694+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557ae6edabc0 con 0x557ae6a711b0 2024-09-21T06:26:25.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.695+0000 7ff2c57fa640 1 -- 172.21.15.31:0/1082421851 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d80ad320 con 0x557ae6a711b0 2024-09-21T06:26:25.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.695+0000 7ff2c57fa640 1 -- 172.21.15.31:0/1082421851 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff2d80a9a80 con 0x557ae6a711b0 2024-09-21T06:26:25.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2ea1a5740 10 client.18372 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:25.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2ea1a5740 10 client.18372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:25.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2ea1a5740 20 client.18372 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:25.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a2dda0 0x557ae6edd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:25.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2ea1a5740 4 client.18372 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:26:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff2cc056070 con 0x557ae6a2dda0 2024-09-21T06:26:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.696+0000 7ff2e11cc640 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a2dda0 0x557ae6edd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2e11cc640 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a2dda0 0x557ae6edd380 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4acb718fba274ed server_cookie=f6442f919411034d in_seq=0 out_seq=0 2024-09-21T06:26:25.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2c57fa640 10 client.18372 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2c57fa640 1 -- 172.21.15.31:0/1082421851 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9462 (crc 0 0 0) 0x7ff2cc056070 con 0x557ae6a2dda0 2024-09-21T06:26:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2c57fa640 10 client.18372 handle_command_reply: tid=0 2024-09-21T06:26:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2ea1a5740 1 client.18372 shutdown 2024-09-21T06:26:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 msgr2=0x557ae6e559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.700+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e559c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80e51c0 tx=0x7ff2d80eb050 comp rx=0 tx=0).stop 2024-09-21T06:26:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 shutdown_connections 2024-09-21T06:26:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557ae6a2dda0 0x557ae6edd380 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ae6a9a070 0x557ae6e514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2c57fa640 0 client.18372 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557ae6a79390 0x557ae6e50e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 --2- 172.21.15.31:0/1082421851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557ae6a711b0 0x557ae6e559c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 >> 172.21.15.31:0/1082421851 conn(0x557ae6a8e460 msgr2=0x557ae6e42e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 shutdown_connections 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.701+0000 7ff2ea1a5740 1 -- 172.21.15.31:0/1082421851 wait complete. 2024-09-21T06:26:25.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.702+0000 7ff2ea1a5740 20 client.18372 trim_cache size 0 max 16384 2024-09-21T06:26:25.705 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:26:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:26:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:26:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "97dcd776-5fd7-4ec8-ada7-f276bb218390": { 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:51.458278+0000", 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:26:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:10.069920+0000", 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:26:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:26:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:26:25.726 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff2c80b6700 msgr2=0x7ff2c80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff2c80b6700 0x7ff2c80b8c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc02a860 tx=0x7ff2cc045000 comp rx=0 tx=0).stop 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 msgr2=0x7ff2dc10fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 0x7ff2dc10fdb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800e9c0 tx=0x7ff2d800ee80 comp rx=0 tx=0).stop 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 shutdown_connections 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7ff2c80b6700 0x7ff2c80b8c10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 --2- 172.21.15.31:0/2578031279 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2dc102a60 0x7ff2dc10fdb0 secure :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800e9c0 tx=0x7ff2d800ee80 comp rx=0 tx=0).stop 2024-09-21T06:26:25.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 >> 172.21.15.31:0/2578031279 conn(0x7ff2dc079730 msgr2=0x7ff2dc1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.703+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 shutdown_connections 2024-09-21T06:26:25.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:25.704+0000 7ff2e3c31640 1 -- 172.21.15.31:0/2578031279 wait complete. 2024-09-21T06:26:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:25 smithi107 ceph-mon[34380]: pgmap v783: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 552 op/s 2024-09-21T06:26:26.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:25 smithi110 ceph-mon[34550]: pgmap v783: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 552 op/s 2024-09-21T06:26:26.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:26:26.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:26:27.235 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:27.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:27 smithi031 ceph-mon[31237]: pgmap v784: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-09-21T06:26:27.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 -- 172.21.15.31:0/923628320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a8102a60 msgr2=0x7f19a8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 --2- 172.21.15.31:0/923628320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a8102a60 0x7f19a8102e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009860 tx=0x7f19a402a340 comp rx=0 tx=0).stop 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 -- 172.21.15.31:0/923628320 shutdown_connections 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 --2- 172.21.15.31:0/923628320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19a8108cf0 0x7f19a810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 --2- 172.21.15.31:0/923628320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a81083a0 0x7f19a81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 --2- 172.21.15.31:0/923628320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a8102a60 0x7f19a8102e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 -- 172.21.15.31:0/923628320 >> 172.21.15.31:0/923628320 conn(0x7f19a8079730 msgr2=0x7f19a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.806+0000 7f19b0327640 1 -- 172.21.15.31:0/923628320 shutdown_connections 2024-09-21T06:26:27.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.807+0000 7f19b0327640 1 -- 172.21.15.31:0/923628320 wait complete. 2024-09-21T06:26:27.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 Processor -- start 2024-09-21T06:26:27.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 -- start start 2024-09-21T06:26:27.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a8102a60 0x7f19a819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a81083a0 0x7f19a819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19a8108cf0 0x7f19a819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19a8112960 con 0x7f19a8108cf0 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19a81127e0 con 0x7f19a81083a0 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19b0327640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f19a8112ae0 con 0x7f19a8102a60 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19ae0c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a8102a60 0x7f19a819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19ae0c3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a8102a60 0x7f19a819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:44074/0 (socket says 172.21.15.31:44074) 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19ad8c2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a81083a0 0x7f19a819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19ae0c3640 1 -- 172.21.15.31:0/2217964055 learned_addr learned my addr 172.21.15.31:0/2217964055 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:27.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.808+0000 7f19ae8c4640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19a8108cf0 0x7f19a819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f19ad8c2640 1 -- 172.21.15.31:0/2217964055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a8102a60 msgr2=0x7f19a819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f19ad8c2640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a8102a60 0x7f19a819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f19ad8c2640 1 -- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19a8108cf0 msgr2=0x7f19a819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f19ad8c2640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19a8108cf0 0x7f19a819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f19ad8c2640 1 -- 172.21.15.31:0/2217964055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a81a6560 con 0x7f19a81083a0 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f19ad8c2640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a81083a0 0x7f19a819b810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f199800e9c0 tx=0x7f199800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199800ccc0 con 0x7f19a81083a0 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.809+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1998005020 con 0x7f19a81083a0 2024-09-21T06:26:27.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.810+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998004a00 con 0x7f19a81083a0 2024-09-21T06:26:27.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.810+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a8199120 con 0x7f19a81083a0 2024-09-21T06:26:27.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.811+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a81995b0 con 0x7f19a81083a0 2024-09-21T06:26:27.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.812+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a8102e70 con 0x7f19a81083a0 2024-09-21T06:26:27.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.814+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f19980051d0 con 0x7f19a81083a0 2024-09-21T06:26:27.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.815+0000 7f199f7fe640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19940bf010 0x7f19940c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:27.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.815+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f19980e7560 con 0x7f19a81083a0 2024-09-21T06:26:27.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.815+0000 7f19ae0c3640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19940bf010 0x7f19940c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:27.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.816+0000 7f19ae0c3640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19940bf010 0x7f19940c1520 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f19a40057c0 tx=0x7f19a4030340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:27.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:27.819+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19980adac0 con 0x7f19a81083a0 2024-09-21T06:26:28.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:27 smithi110 ceph-mon[34550]: pgmap v784: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-09-21T06:26:28.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:27 smithi107 ceph-mon[34380]: pgmap v784: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 568 op/s 2024-09-21T06:26:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.160+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19a802e640 con 0x7f19a81083a0 2024-09-21T06:26:28.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.161+0000 7f199f7fe640 1 -- 172.21.15.31:0/2217964055 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f19980b1ea0 con 0x7f19a81083a0 2024-09-21T06:26:28.163 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:28.163 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:26:28.163 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:26:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19940bf010 msgr2=0x7f19940c1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19940bf010 0x7f19940c1520 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f19a40057c0 tx=0x7f19a4030340 comp rx=0 tx=0).stop 2024-09-21T06:26:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a81083a0 msgr2=0x7f19a819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a81083a0 0x7f19a819b810 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f199800e9c0 tx=0x7f199800ee80 comp rx=0 tx=0).stop 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 shutdown_connections 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f19940bf010 0x7f19940c1520 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19a8108cf0 0x7f19a819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19a81083a0 0x7f19a819b810 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 --2- 172.21.15.31:0/2217964055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f19a8102a60 0x7f19a819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 >> 172.21.15.31:0/2217964055 conn(0x7f19a8079730 msgr2=0x7f19a8101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 shutdown_connections 2024-09-21T06:26:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:28.163+0000 7f19b0327640 1 -- 172.21.15.31:0/2217964055 wait complete. 2024-09-21T06:26:28.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:28 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2217964055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:29.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:26:29.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:28 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2217964055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:29.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:28 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2217964055' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:26:29.396 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:29.778 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:29 smithi031 ceph-mon[31237]: pgmap v785: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 637 op/s 2024-09-21T06:26:30.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:29 smithi110 ceph-mon[34550]: pgmap v785: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 637 op/s 2024-09-21T06:26:30.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:29 smithi107 ceph-mon[34380]: pgmap v785: 97 pgs: 97 active+clean; 4.2 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 12 MiB/s wr, 637 op/s 2024-09-21T06:26:30.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.150+0000 7fdb2161f640 1 -- 172.21.15.31:0/982694785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c108cf0 msgr2=0x7fdb1c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.150+0000 7fdb2161f640 1 --2- 172.21.15.31:0/982694785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c10f520 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdb1401a910 tx=0x7fdb1403b450 comp rx=0 tx=0).stop 2024-09-21T06:26:30.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.155+0000 7fdb2161f640 1 -- 172.21.15.31:0/982694785 shutdown_connections 2024-09-21T06:26:30.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.155+0000 7fdb2161f640 1 --2- 172.21.15.31:0/982694785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c10f520 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.155+0000 7fdb2161f640 1 --2- 172.21.15.31:0/982694785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c1083a0 0x7fdb1c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.155+0000 7fdb2161f640 1 --2- 172.21.15.31:0/982694785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c102a60 0x7fdb1c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.155+0000 7fdb2161f640 1 -- 172.21.15.31:0/982694785 >> 172.21.15.31:0/982694785 conn(0x7fdb1c079730 msgr2=0x7fdb1c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.159+0000 7fdb2161f640 1 -- 172.21.15.31:0/982694785 shutdown_connections 2024-09-21T06:26:30.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.166+0000 7fdb2161f640 1 -- 172.21.15.31:0/982694785 wait complete. 2024-09-21T06:26:30.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.167+0000 7fdb2161f640 1 Processor -- start 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.167+0000 7fdb2161f640 1 -- start start 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.168+0000 7fdb2161f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c102a60 0x7fdb1c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.168+0000 7fdb2161f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c1083a0 0x7fdb1c06ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.168+0000 7fdb2161f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c070510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.168+0000 7fdb2161f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06f6f0 con 0x7fdb1c1083a0 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.168+0000 7fdb2161f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06f570 con 0x7fdb1c102a60 2024-09-21T06:26:30.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.168+0000 7fdb2161f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06f870 con 0x7fdb1c108cf0 2024-09-21T06:26:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.172+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c070510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.172+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c070510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40496/0 (socket says 172.21.15.31:40496) 2024-09-21T06:26:30.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.172+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/1801429564 learned_addr learned my addr 172.21.15.31:0/1801429564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.173+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c1083a0 0x7fdb1c06ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.174+0000 7fdb1affd640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c102a60 0x7fdb1c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.176+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/1801429564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c102a60 msgr2=0x7fdb1c0719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.176+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c102a60 0x7fdb1c0719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.176+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c1083a0 msgr2=0x7fdb1c06ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.176+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c1083a0 0x7fdb1c06ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.176+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/1801429564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c070d10 con 0x7fdb1c108cf0 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.176+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c070510 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb140048f0 tx=0x7fdb14004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:30.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.178+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c1083a0 0x7fdb1c06ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:30.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.179+0000 7fdb03fff640 1 -- 172.21.15.31:0/1801429564 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1404f020 con 0x7fdb1c108cf0 2024-09-21T06:26:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.179+0000 7fdb03fff640 1 -- 172.21.15.31:0/1801429564 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb140035f0 con 0x7fdb1c108cf0 2024-09-21T06:26:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.179+0000 7fdb03fff640 1 -- 172.21.15.31:0/1801429564 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14046070 con 0x7fdb1c108cf0 2024-09-21T06:26:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.179+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c197e50 con 0x7fdb1c108cf0 2024-09-21T06:26:30.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.179+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1982e0 con 0x7fdb1c108cf0 2024-09-21T06:26:30.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.187+0000 7fdb03fff640 1 -- 172.21.15.31:0/1801429564 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdb14052020 con 0x7fdb1c108cf0 2024-09-21T06:26:30.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.191+0000 7fdb03fff640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdb040b6850 0x7fdb040b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.191+0000 7fdb03fff640 1 -- 172.21.15.31:0/1801429564 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fdb14118980 con 0x7fdb1c108cf0 2024-09-21T06:26:30.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.192+0000 7fdb1affd640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdb040b6850 0x7fdb040b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.197+0000 7fdb27b93740 1 Processor -- start 2024-09-21T06:26:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.197+0000 7fdb27b93740 1 -- start start 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.197+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.197+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dc98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb1affd640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdb040b6850 0x7fdb040b8d60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb10006fd0 tx=0x7fdb10008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb1a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb27b93740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56016dc0dea0 con 0x56016d8cf390 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb27b93740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56016dbfd2d0 con 0x56016d8c71b0 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb27b93740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56016dc010e0 con 0x56016d8f0070 2024-09-21T06:26:30.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dc98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb1a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc97be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59604/0 (socket says 172.21.15.31:59604) 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb1a7fc640 1 -- 172.21.15.31:0/86059499 learned_addr learned my addr 172.21.15.31:0/86059499 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.198+0000 7fdb1affd640 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.199+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/86059499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 msgr2=0x56016dc976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.199+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.199+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/86059499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 msgr2=0x56016dc97be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.199+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc97be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.199+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/86059499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016d95b650 con 0x56016d8cf390 2024-09-21T06:26:30.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.200+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dc98120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdb14116080 tx=0x7fdb140de0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198d7506c5c6c4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:30.205 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 890.0 s, 1362.2 tps, lat 72.681 ms stddev 82.875 2024-09-21T06:26:30.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.201+0000 7fdb017fa640 1 -- 172.21.15.31:0/86059499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14116500 con 0x56016d8cf390 2024-09-21T06:26:30.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.201+0000 7fdb017fa640 1 -- 172.21.15.31:0/86059499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb140dee80 con 0x56016d8cf390 2024-09-21T06:26:30.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.201+0000 7fdb017fa640 1 -- 172.21.15.31:0/86059499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140dfab0 con 0x56016d8cf390 2024-09-21T06:26:30.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.202+0000 7fdb27b93740 1 -- 172.21.15.31:0/86059499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 msgr2=0x56016dc98120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.202+0000 7fdb27b93740 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dc98120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdb14116080 tx=0x7fdb140de0d0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.204+0000 7fdb27b93740 1 -- 172.21.15.31:0/86059499 shutdown_connections 2024-09-21T06:26:30.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.204+0000 7fdb27b93740 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dc98120 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.204+0000 7fdb27b93740 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc97be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.204+0000 7fdb27b93740 1 --2- 172.21.15.31:0/86059499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.204+0000 7fdb27b93740 1 -- 172.21.15.31:0/86059499 >> 172.21.15.31:0/86059499 conn(0x56016d8e4460 msgr2=0x56016dc95590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.206+0000 7fdb27b93740 1 -- 172.21.15.31:0/86059499 shutdown_connections 2024-09-21T06:26:30.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.206+0000 7fdb27b93740 1 -- 172.21.15.31:0/86059499 wait complete. 2024-09-21T06:26:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.207+0000 7fdb27b93740 1 -- start start 2024-09-21T06:26:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.207+0000 7fdb27b93740 1 Processor -- start 2024-09-21T06:26:30.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 0x56016dca2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dca3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dcaed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 0x56016dca2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 0x56016dca2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40506/0 (socket says 172.21.15.31:40506) 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb1affd640 1 -- 172.21.15.31:0/3536838898 learned_addr learned my addr 172.21.15.31:0/3536838898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x56016da5e700 con 0x56016d8cf390 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.208+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56016db98160 con 0x56016d8f0070 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56016dc0dea0 con 0x56016d8c71b0 2024-09-21T06:26:30.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dcaed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb27b93740 10 client.0 _ll_register_callbacks cb 0x56016dbb19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:30.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/3536838898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 msgr2=0x56016dca2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 0x56016dca2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/3536838898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 msgr2=0x56016dca3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dca3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1b7fe640 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016dbc5c90 con 0x56016d8f0070 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.209+0000 7fdb1affd640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 0x56016dca2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.210+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dcaed20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x56016dca5a50 tx=0x7fdb140da4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f120c3f377c6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.210+0000 7fdafeff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.210+0000 7fdafeff5640 1 -- 172.21.15.31:0/3536838898 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14116510 con 0x56016d8f0070 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.210+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb140dee80 con 0x56016d8f0070 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.210+0000 7fdafeff5640 1 -- 172.21.15.31:0/3536838898 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fdb140ddbe0 con 0x56016d8f0070 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.211+0000 7fdafeff5640 1 -- 172.21.15.31:0/3536838898 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdb140f9050 con 0x56016d8f0070 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.211+0000 7fdb27b93740 10 client.28181 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:30.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.211+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56016dcb1750 con 0x56016d8f0070 2024-09-21T06:26:30.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.211+0000 7fdafeff5640 1 -- 172.21.15.31:0/3536838898 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb140dee80 con 0x56016d8f0070 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdafeff5640 1 -- 172.21.15.31:0/3536838898 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdb140dd650 con 0x56016d8f0070 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb27b93740 10 client.28181 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb27b93740 10 client.28181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb27b93740 20 client.28181 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d8ae9f0 0x56016dcb4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb27b93740 4 client.28181 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56016dc0fdf0 con 0x56016d8ae9f0 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.213+0000 7fdb1affd640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d8ae9f0 0x56016dcb4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.214+0000 7fdb1affd640 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d8ae9f0 0x56016dcb4ab0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7233ce30857e28e8 server_cookie=aecb50b59114f573 in_seq=0 out_seq=0 2024-09-21T06:26:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.214+0000 7fdafeff5640 10 client.28181 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.215+0000 7fdafeff5640 1 -- 172.21.15.31:0/3536838898 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56016dc0fdf0 con 0x56016d8ae9f0 2024-09-21T06:26:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.215+0000 7fdafeff5640 10 client.28181 handle_command_reply: tid=0 2024-09-21T06:26:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.215+0000 7fdb27b93740 1 client.28181 shutdown 2024-09-21T06:26:30.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 msgr2=0x56016dcaed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dcaed20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x56016dca5a50 tx=0x7fdb140da4c0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 shutdown_connections 2024-09-21T06:26:30.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d8ae9f0 0x56016dcb4ab0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dcaed20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dca3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3536838898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8c71b0 0x56016dca2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 >> 172.21.15.31:0/3536838898 conn(0x56016d8e4460 msgr2=0x56016dc9a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.218+0000 7fdafeff5640 0 client.28181 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.219+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 shutdown_connections 2024-09-21T06:26:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.219+0000 7fdb27b93740 1 -- 172.21.15.31:0/3536838898 wait complete. 2024-09-21T06:26:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.219+0000 7fdb27b93740 20 client.28181 trim_cache size 0 max 16384 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 Processor -- start 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 -- start start 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dcaebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc9a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800c710 con 0x56016d8cf390 2024-09-21T06:26:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800eee0 con 0x56016d8c71b0 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb27b93740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800ccd0 con 0x56016d8f0070 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb1a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dcaebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb1a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dcaebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43952/0 (socket says 172.21.15.31:43952) 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb1a7fc640 1 -- 172.21.15.31:0/65226996 learned_addr learned my addr 172.21.15.31:0/65226996 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.243+0000 7fdb1affd640 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc9a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1a7fc640 1 -- 172.21.15.31:0/65226996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 msgr2=0x56016dc9a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc9a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1a7fc640 1 -- 172.21.15.31:0/65226996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 msgr2=0x56016dc955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1a7fc640 1 -- 172.21.15.31:0/65226996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016dca4090 con 0x56016d8cf390 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc9a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dcaebc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c003d00 tx=0x7fdb0c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5429d650bcc4c358 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.244+0000 7fdb00ff9640 1 -- 172.21.15.31:0/65226996 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c003df0 con 0x56016d8cf390 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.245+0000 7fdb00ff9640 1 -- 172.21.15.31:0/65226996 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0c0118f0 con 0x56016d8cf390 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.245+0000 7fdb00ff9640 1 -- 172.21.15.31:0/65226996 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c010970 con 0x56016d8cf390 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 -- 172.21.15.31:0/65226996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 msgr2=0x56016dcaebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dcaebc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c003d00 tx=0x7fdb0c004410 comp rx=0 tx=0).stop 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 -- 172.21.15.31:0/65226996 shutdown_connections 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8f0070 0x56016dc9a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8cf390 0x56016dcaebc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 --2- 172.21.15.31:0/65226996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8c71b0 0x56016dc955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 -- 172.21.15.31:0/65226996 >> 172.21.15.31:0/65226996 conn(0x56016d8e4460 msgr2=0x56016dc94ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 -- 172.21.15.31:0/65226996 shutdown_connections 2024-09-21T06:26:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.246+0000 7fdb27b93740 1 -- 172.21.15.31:0/65226996 wait complete. 2024-09-21T06:26:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.247+0000 7fdb27b93740 1 -- start start 2024-09-21T06:26:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.247+0000 7fdb27b93740 1 Processor -- start 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.247+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 0x56016dcab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.247+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8cf390 0x56016dca6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.247+0000 7fdb27b93740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dca7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb27b93740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800d3a0 con 0x56016d8c71b0 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb27b93740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800e810 con 0x56016d8f0070 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb27b93740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0800da70 con 0x56016d8cf390 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb27b93740 10 client.0 _ll_register_callbacks cb 0x7fdb0c009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 0x56016dcab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 0x56016dcab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43964/0 (socket says 172.21.15.31:43964) 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 -- 172.21.15.31:0/3281239579 learned_addr learned my addr 172.21.15.31:0/3281239579 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8cf390 0x56016dca6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1b7fe640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dca7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 -- 172.21.15.31:0/3281239579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8cf390 msgr2=0x56016dca6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8cf390 0x56016dca6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 -- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 msgr2=0x56016dca7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dca7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 -- 172.21.15.31:0/3281239579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016dd1e470 con 0x56016d8c71b0 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdb1affd640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 0x56016dcab590 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000dbb0 tx=0x7fdb10010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10671b6780d434e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdafeff5640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.248+0000 7fdafeff5640 1 -- 172.21.15.31:0/3281239579 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10016990 con 0x56016d8c71b0 2024-09-21T06:26:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0c0118f0 con 0x56016d8c71b0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdafeff5640 1 -- 172.21.15.31:0/3281239579 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fdb1000f040 con 0x56016d8c71b0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdafeff5640 1 -- 172.21.15.31:0/3281239579 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdb10039050 con 0x56016d8c71b0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 10 client.18390 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56016dd1f5f0 con 0x56016d8c71b0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdafeff5640 1 -- 172.21.15.31:0/3281239579 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0c0118f0 con 0x56016d8c71b0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdafeff5640 1 -- 172.21.15.31:0/3281239579 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fdb10013070 con 0x56016d8c71b0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 10 client.18390 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 10 client.18390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 20 client.18390 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d883da0 0x56016dd21dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.249+0000 7fdb27b93740 4 client.18390 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.250+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56016dc0b8a0 con 0x56016d883da0 2024-09-21T06:26:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.250+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d883da0 0x56016dd21dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.250+0000 7fdafeff5640 10 client.18390 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.250+0000 7fdb1a7fc640 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d883da0 0x56016dd21dc0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d91c786853fdd23 server_cookie=d41de8009e62040 in_seq=0 out_seq=0 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.250+0000 7fdafeff5640 1 -- 172.21.15.31:0/3281239579 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56016dc0b8a0 con 0x56016d883da0 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.250+0000 7fdafeff5640 10 client.18390 handle_command_reply: tid=0 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 client.18390 shutdown 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 msgr2=0x56016dcab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 0x56016dcab590 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000dbb0 tx=0x7fdb10010000 comp rx=0 tx=0).stop 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 shutdown_connections 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x56016d883da0 0x56016dd21dc0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdafeff5640 0 client.18390 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56016d8f0070 0x56016dca7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016d8cf390 0x56016dca6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 --2- 172.21.15.31:0/3281239579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56016d8c71b0 0x56016dcab590 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 >> 172.21.15.31:0/3281239579 conn(0x56016d8e4460 msgr2=0x56016dc9a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 shutdown_connections 2024-09-21T06:26:30.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 1 -- 172.21.15.31:0/3281239579 wait complete. 2024-09-21T06:26:30.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.251+0000 7fdb27b93740 20 client.18390 trim_cache size 0 max 16384 2024-09-21T06:26:30.258 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:26:30.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.252+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdb040b6850 msgr2=0x7fdb040b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.252+0000 7fdb2161f640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdb040b6850 0x7fdb040b8d60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb10006fd0 tx=0x7fdb10008040 comp rx=0 tx=0).stop 2024-09-21T06:26:30.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.252+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 msgr2=0x7fdb1c070510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:30.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c070510 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb140048f0 tx=0x7fdb14004920 comp rx=0 tx=0).stop 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 shutdown_connections 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb1c108cf0 0x7fdb1c070510 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fdb040b6850 0x7fdb040b8d60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb1c1083a0 0x7fdb1c06ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 --2- 172.21.15.31:0/1801429564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c102a60 0x7fdb1c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 >> 172.21.15.31:0/1801429564 conn(0x7fdb1c079730 msgr2=0x7fdb1c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 shutdown_connections 2024-09-21T06:26:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:30.253+0000 7fdb2161f640 1 -- 172.21.15.31:0/1801429564 wait complete. 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.737+0000 7f68aadad640 1 -- 172.21.15.31:0/3268692005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4103ad0 msgr2=0x7f68a4103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.737+0000 7f68aadad640 1 --2- 172.21.15.31:0/3268692005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4103ad0 0x7f68a4103ee0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6898009900 tx=0x7f689802a410 comp rx=0 tx=0).stop 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 -- 172.21.15.31:0/3268692005 shutdown_connections 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 --2- 172.21.15.31:0/3268692005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 --2- 172.21.15.31:0/3268692005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68a4109400 0x7f68a4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 --2- 172.21.15.31:0/3268692005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4103ad0 0x7f68a4103ee0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 -- 172.21.15.31:0/3268692005 >> 172.21.15.31:0/3268692005 conn(0x7f68a40ff510 msgr2=0x7f68a4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 -- 172.21.15.31:0/3268692005 shutdown_connections 2024-09-21T06:26:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 -- 172.21.15.31:0/3268692005 wait complete. 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.738+0000 7f68aadad640 1 Processor -- start 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.739+0000 7f68aadad640 1 -- start start 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.740+0000 7f68aadad640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68a4103ad0 0x7f68a419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.740+0000 7f68aadad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4109400 0x7f68a419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.740+0000 7f68aadad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.740+0000 7f68aadad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4113a10 con 0x7f68a4109400 2024-09-21T06:26:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.740+0000 7f68aadad640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4113890 con 0x7f68a4109d50 2024-09-21T06:26:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.740+0000 7f68aadad640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4113b90 con 0x7f68a4103ad0 2024-09-21T06:26:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.741+0000 7f68a934a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.741+0000 7f68a934a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59648/0 (socket says 172.21.15.31:59648) 2024-09-21T06:26:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.741+0000 7f68a934a640 1 -- 172.21.15.31:0/558934867 learned_addr learned my addr 172.21.15.31:0/558934867 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.741+0000 7f68a3fff640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4109400 0x7f68a419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.741+0000 7f68a8b49640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68a4103ad0 0x7f68a419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a934a640 1 -- 172.21.15.31:0/558934867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68a4103ad0 msgr2=0x7f68a419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a934a640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68a4103ad0 0x7f68a419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a934a640 1 -- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4109400 msgr2=0x7f68a419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a934a640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4109400 0x7f68a419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a934a640 1 -- 172.21.15.31:0/558934867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a41a03c0 con 0x7f68a4109d50 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a3fff640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4109400 0x7f68a419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a934a640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a419fcc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f689c00eb10 tx=0x7f689c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a1ffb640 1 -- 172.21.15.31:0/558934867 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c004b20 con 0x7f68a4109d50 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a1ffb640 1 -- 172.21.15.31:0/558934867 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c004cc0 con 0x7f68a4109d50 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a41a06b0 con 0x7f68a4109d50 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68a1ffb640 1 -- 172.21.15.31:0/558934867 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c010670 con 0x7f68a4109d50 2024-09-21T06:26:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.742+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a41a6e40 con 0x7f68a4109d50 2024-09-21T06:26:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.745+0000 7f68a1ffb640 1 -- 172.21.15.31:0/558934867 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f689c005020 con 0x7f68a4109d50 2024-09-21T06:26:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 Processor -- start 2024-09-21T06:26:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 -- start start 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f811b0 0x55dd543269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f582f0 0x55dd54326ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd54327430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd5429b6f0 con 0x55dd53f604d0 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dd5428ab20 con 0x55dd53f582f0 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68b1df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dd5428e930 con 0x55dd53f811b0 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd54327430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd54327430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43996/0 (socket says 172.21.15.31:43996) 2024-09-21T06:26:31.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68a3fff640 1 -- 172.21.15.31:0/1699761414 learned_addr learned my addr 172.21.15.31:0/1699761414 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68a3fff640 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f811b0 0x55dd543269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.746+0000 7f68a8b49640 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f582f0 0x55dd54326ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a3fff640 1 -- 172.21.15.31:0/1699761414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f582f0 msgr2=0x55dd54326ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a3fff640 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f582f0 0x55dd54326ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a3fff640 1 -- 172.21.15.31:0/1699761414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 msgr2=0x55dd54327430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a3fff640 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd54327430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a3fff640 1 -- 172.21.15.31:0/1699761414 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd54250530 con 0x55dd53f811b0 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a1ffb640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f68940b6700 0x7f68940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a8b49640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f68940b6700 0x7f68940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a3fff640 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f811b0 0x55dd543269b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6890011b90 tx=0x7f6890005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26ae09ae3d75412 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f68a1ffb640 1 -- 172.21.15.31:0/558934867 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f689c0e7c60 con 0x7f68a4109d50 2024-09-21T06:26:31.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f688effd640 1 -- 172.21.15.31:0/1699761414 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890011d40 con 0x55dd53f811b0 2024-09-21T06:26:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f688effd640 1 -- 172.21.15.31:0/1699761414 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890011820 con 0x55dd53f811b0 2024-09-21T06:26:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.747+0000 7f688effd640 1 -- 172.21.15.31:0/1699761414 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890005480 con 0x55dd53f811b0 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.748+0000 7f68b1df7740 1 -- 172.21.15.31:0/1699761414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f811b0 msgr2=0x55dd543269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.748+0000 7f68a8b49640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f68940b6700 0x7f68940b8c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6898031440 tx=0x7f6898009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.748+0000 7f68b1df7740 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f811b0 0x55dd543269b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6890011b90 tx=0x7f6890005f90 comp rx=0 tx=0).stop 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.748+0000 7f68b1df7740 1 -- 172.21.15.31:0/1699761414 shutdown_connections 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd54327430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f582f0 0x55dd54326ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 --2- 172.21.15.31:0/1699761414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f811b0 0x55dd543269b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 -- 172.21.15.31:0/1699761414 >> 172.21.15.31:0/1699761414 conn(0x55dd53f75300 msgr2=0x55dd543248a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 -- 172.21.15.31:0/1699761414 shutdown_connections 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 -- 172.21.15.31:0/1699761414 wait complete. 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 -- start start 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.749+0000 7f68b1df7740 1 Processor -- start 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd543320a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f604d0 0x55dd543325e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f811b0 0x55dd5433e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55dd541c2ab0 con 0x55dd53f811b0 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dd54230730 con 0x55dd53f604d0 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dd54335130 con 0x55dd53f582f0 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68b1df7740 10 client.0 _ll_register_callbacks cb 0x55dd542846b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd543320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a934a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f811b0 0x55dd5433e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd543320a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40560/0 (socket says 172.21.15.31:40560) 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f604d0 0x55dd543325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a934a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f811b0 0x55dd5433e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44002/0 (socket says 172.21.15.31:44002) 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 -- 172.21.15.31:0/744153188 learned_addr learned my addr 172.21.15.31:0/744153188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 -- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f604d0 msgr2=0x55dd543325e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f604d0 0x55dd543325e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 -- 172.21.15.31:0/744153188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f811b0 msgr2=0x55dd5433e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f811b0 0x55dd5433e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.750+0000 7f68a8b49640 1 -- 172.21.15.31:0/744153188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd53fec790 con 0x55dd53f582f0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f68a8b49640 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd543320a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6898009900 tx=0x7f6898009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34705e9a28e4df9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f6883fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f6883fff640 1 -- 172.21.15.31:0/744153188 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689804d070 con 0x55dd53f582f0 2024-09-21T06:26:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f6883fff640 1 -- 172.21.15.31:0/744153188 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f6898004450 con 0x55dd53f582f0 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6890011820 con 0x55dd53f582f0 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f6883fff640 1 -- 172.21.15.31:0/744153188 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6898059050 con 0x55dd53f582f0 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f68b1df7740 10 client.28240 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd54065620 con 0x55dd53f582f0 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f6883fff640 1 -- 172.21.15.31:0/744153188 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890011820 con 0x55dd53f582f0 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.751+0000 7f6883fff640 1 -- 172.21.15.31:0/744153188 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6898004ca0 con 0x55dd53f582f0 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68b1df7740 10 client.28240 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68b1df7740 10 client.28240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68b1df7740 20 client.28240 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68b1df7740 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f3fd80 0x55dd543441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68b1df7740 4 client.28240 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd5429d640 con 0x55dd53f3fd80 2024-09-21T06:26:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68a3fff640 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f3fd80 0x55dd543441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f6883fff640 10 client.28240 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.752+0000 7f68a3fff640 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f3fd80 0x55dd543441a0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87161a0b5bb97e49 server_cookie=c633ddd9d587c92a in_seq=0 out_seq=0 2024-09-21T06:26:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.754+0000 7f6883fff640 1 -- 172.21.15.31:0/744153188 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd5429d640 con 0x55dd53f3fd80 2024-09-21T06:26:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.754+0000 7f6883fff640 10 client.28240 handle_command_reply: tid=0 2024-09-21T06:26:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.754+0000 7f68b1df7740 1 client.28240 shutdown 2024-09-21T06:26:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.754+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 msgr2=0x55dd543320a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.754+0000 7f68b1df7740 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd543320a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6898009900 tx=0x7f6898009260 comp rx=0 tx=0).stop 2024-09-21T06:26:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 shutdown_connections 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f3fd80 0x55dd543441a0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f6883fff640 0 client.28240 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f811b0 0x55dd5433e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f604d0 0x55dd543325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 --2- 172.21.15.31:0/744153188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd543320a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 >> 172.21.15.31:0/744153188 conn(0x55dd53f75300 msgr2=0x55dd543279e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.755+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 shutdown_connections 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.756+0000 7f68b1df7740 1 -- 172.21.15.31:0/744153188 wait complete. 2024-09-21T06:26:31.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.756+0000 7f68b1df7740 20 client.28240 trim_cache size 0 max 16384 2024-09-21T06:26:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.773+0000 7f68b1df7740 1 Processor -- start 2024-09-21T06:26:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.773+0000 7f68b1df7740 1 -- start start 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd543283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54333cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd5432f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68b1df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ad5f0 con 0x55dd53f604d0 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68b1df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41afe50 con 0x55dd53f3fd80 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68b1df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41adc40 con 0x55dd53f582f0 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54333cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54333cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40576/0 (socket says 172.21.15.31:40576) 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 -- 172.21.15.31:0/3967624657 learned_addr learned my addr 172.21.15.31:0/3967624657 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a934a640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd5432f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a8b49640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd543283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 -- 172.21.15.31:0/3967624657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 msgr2=0x55dd543283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd543283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 -- 172.21.15.31:0/3967624657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 msgr2=0x55dd5432f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd5432f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a3fff640 1 -- 172.21.15.31:0/3967624657 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd54334680 con 0x55dd53f582f0 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a934a640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd5432f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.774+0000 7f68a8b49640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd543283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.775+0000 7f68a3fff640 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54333cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6890011b60 tx=0x7f689001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b5fc5f40bfbc452 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.775+0000 7f688e7fc640 1 -- 172.21.15.31:0/3967624657 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890011250 con 0x55dd53f582f0 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.775+0000 7f688e7fc640 1 -- 172.21.15.31:0/3967624657 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890005360 con 0x55dd53f582f0 2024-09-21T06:26:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.775+0000 7f688e7fc640 1 -- 172.21.15.31:0/3967624657 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890003aa0 con 0x55dd53f582f0 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 -- 172.21.15.31:0/3967624657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 msgr2=0x55dd54333cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54333cd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6890011b60 tx=0x7f689001a040 comp rx=0 tx=0).stop 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 -- 172.21.15.31:0/3967624657 shutdown_connections 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd5432f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54333cd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 --2- 172.21.15.31:0/3967624657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd543283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 -- 172.21.15.31:0/3967624657 >> 172.21.15.31:0/3967624657 conn(0x55dd53f75300 msgr2=0x55dd543335c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 -- 172.21.15.31:0/3967624657 shutdown_connections 2024-09-21T06:26:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 -- 172.21.15.31:0/3967624657 wait complete. 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 -- start start 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.776+0000 7f68b1df7740 1 Processor -- start 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd54337b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54338070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd543acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ae310 con 0x55dd53f604d0 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41af780 con 0x55dd53f3fd80 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41ae9e0 con 0x55dd53f582f0 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68b1df7740 10 client.0 _ll_register_callbacks cb 0x7f6898009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd543acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a8b49640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd54337b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd543acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44028/0 (socket says 172.21.15.31:44028) 2024-09-21T06:26:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a8b49640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd54337b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59680/0 (socket says 172.21.15.31:59680) 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 -- 172.21.15.31:0/2650379933 learned_addr learned my addr 172.21.15.31:0/2650379933 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a3fff640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54338070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 -- 172.21.15.31:0/2650379933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 msgr2=0x55dd54338070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54338070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 -- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 msgr2=0x55dd54337b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd54337b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a934a640 1 -- 172.21.15.31:0/2650379933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd543ad160 con 0x55dd53f604d0 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a3fff640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54338070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.777+0000 7f68a8b49640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd54337b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f68a934a640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd543acba0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ada50 tx=0x7f689c0ad4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a020364e451c4dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f6883fff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f6883fff640 1 -- 172.21.15.31:0/2650379933 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0ae1c0 con 0x55dd53f604d0 2024-09-21T06:26:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f6883fff640 1 -- 172.21.15.31:0/2650379933 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f689c0b7050 con 0x55dd53f604d0 2024-09-21T06:26:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6890005360 con 0x55dd53f604d0 2024-09-21T06:26:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f6883fff640 1 -- 172.21.15.31:0/2650379933 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f689c0b25a0 con 0x55dd53f604d0 2024-09-21T06:26:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f68b1df7740 10 client.18408 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.778+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd543ae450 con 0x55dd53f604d0 2024-09-21T06:26:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.779+0000 7f6883fff640 1 -- 172.21.15.31:0/2650379933 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890005360 con 0x55dd53f604d0 2024-09-21T06:26:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.779+0000 7f6883fff640 1 -- 172.21.15.31:0/2650379933 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f689c0e5b10 con 0x55dd53f604d0 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68b1df7740 10 client.18408 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68b1df7740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68b1df7740 20 client.18408 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68b1df7740 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f811b0 0x55dd543b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68b1df7740 4 client.18408 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd542990f0 con 0x55dd53f811b0 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.780+0000 7f68a8b49640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f811b0 0x55dd543b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.781+0000 7f68a8b49640 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f811b0 0x55dd543b0c60 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f774aaca3b38588 server_cookie=7ad95fcbbedcdb3a in_seq=0 out_seq=0 2024-09-21T06:26:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.781+0000 7f6883fff640 10 client.18408 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.841+0000 7f6883fff640 1 -- 172.21.15.31:0/2650379933 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd542990f0 con 0x55dd53f811b0 2024-09-21T06:26:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.841+0000 7f6883fff640 10 client.18408 handle_command_reply: tid=0 2024-09-21T06:26:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.843+0000 7f68b1df7740 1 client.18408 shutdown 2024-09-21T06:26:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.849+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 msgr2=0x55dd543acba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.849+0000 7f68b1df7740 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd543acba0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f689c0ada50 tx=0x7f689c0ad4d0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.850+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 shutdown_connections 2024-09-21T06:26:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.850+0000 7f68b1df7740 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55dd53f811b0 0x55dd543b0c60 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.850+0000 7f68b1df7740 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd53f604d0 0x55dd543acba0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.850+0000 7f68b1df7740 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dd53f582f0 0x55dd54338070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.850+0000 7f68b1df7740 1 --2- 172.21.15.31:0/2650379933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dd53f3fd80 0x55dd54337b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.850+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 >> 172.21.15.31:0/2650379933 conn(0x55dd53f75300 msgr2=0x55dd54328170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.851+0000 7f6883fff640 0 client.18408 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.854+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 shutdown_connections 2024-09-21T06:26:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.858+0000 7f68b1df7740 1 -- 172.21.15.31:0/2650379933 wait complete. 2024-09-21T06:26:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.860+0000 7f68b1df7740 20 client.18408 trim_cache size 0 max 16384 2024-09-21T06:26:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:26:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:26:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:26:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:26:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f68940b6700 msgr2=0x7f68940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f68940b6700 0x7f68940b8c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6898031440 tx=0x7f6898009560 comp rx=0 tx=0).stop 2024-09-21T06:26:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 msgr2=0x7f68a419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a419fcc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f689c00eb10 tx=0x7f689c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 shutdown_connections 2024-09-21T06:26:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a4109d50 0x7f68a419fcc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f68940b6700 0x7f68940b8c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68a4109400 0x7f68a419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 --2- 172.21.15.31:0/558934867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68a4103ad0 0x7f68a419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.874+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 >> 172.21.15.31:0/558934867 conn(0x7f68a40ff510 msgr2=0x7f68a4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.875+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 shutdown_connections 2024-09-21T06:26:31.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:31.876+0000 7f68aadad640 1 -- 172.21.15.31:0/558934867 wait complete. 2024-09-21T06:26:31.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:31 smithi031 ceph-mon[31237]: pgmap v786: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 688 op/s 2024-09-21T06:26:31.927 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:26:31.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:26:32.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:31 smithi110 ceph-mon[34550]: pgmap v786: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 688 op/s 2024-09-21T06:26:32.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:31 smithi107 ceph-mon[34380]: pgmap v786: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 12 MiB/s wr, 688 op/s 2024-09-21T06:26:32.327 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 -- 172.21.15.31:0/40189139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 msgr2=0x7f7fd010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 --2- 172.21.15.31:0/40189139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 0x7f7fd010a3c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009900 tx=0x7f7fc402a410 comp rx=0 tx=0).stop 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 -- 172.21.15.31:0/40189139 shutdown_connections 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 --2- 172.21.15.31:0/40189139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd011cc30 0x7f7fd011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 --2- 172.21.15.31:0/40189139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 --2- 172.21.15.31:0/40189139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 0x7f7fd010a3c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.912+0000 7f7fd5483640 1 -- 172.21.15.31:0/40189139 >> 172.21.15.31:0/40189139 conn(0x7f7fd0070ec0 msgr2=0x7f7fd00712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:32.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.914+0000 7f7fd5483640 1 -- 172.21.15.31:0/40189139 shutdown_connections 2024-09-21T06:26:32.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.919+0000 7f7fd5483640 1 -- 172.21.15.31:0/40189139 wait complete. 2024-09-21T06:26:32.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.920+0000 7f7fd5483640 1 Processor -- start 2024-09-21T06:26:32.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 -- start start 2024-09-21T06:26:32.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd011c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 0x7f7fd0116800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd011cc30 0x7f7fd0116e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112c80 con 0x7f7fd011cc30 2024-09-21T06:26:32.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112b00 con 0x7f7fd010a9e0 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fd5483640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd0112e00 con 0x7f7fd0111f40 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd011c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd011c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59694/0 (socket says 172.21.15.31:59694) 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.923+0000 7f7fceffd640 1 -- 172.21.15.31:0/3788372762 learned_addr learned my addr 172.21.15.31:0/3788372762 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.924+0000 7f7fceffd640 1 -- 172.21.15.31:0/3788372762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 msgr2=0x7f7fd0116800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.924+0000 7f7fceffd640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 0x7f7fd0116800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.924+0000 7f7fceffd640 1 -- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd011cc30 msgr2=0x7f7fd0116e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.924+0000 7f7fceffd640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd011cc30 0x7f7fd0116e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.924+0000 7f7fceffd640 1 -- 172.21.15.31:0/3788372762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd01bddd0 con 0x7f7fd010a9e0 2024-09-21T06:26:32.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.924+0000 7f7fceffd640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd011c540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4004770 tx=0x7f7fc4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.933+0000 7f7fb7fff640 1 -- 172.21.15.31:0/3788372762 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc403f070 con 0x7f7fd010a9e0 2024-09-21T06:26:32.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.933+0000 7f7fb7fff640 1 -- 172.21.15.31:0/3788372762 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc403bb10 con 0x7f7fd010a9e0 2024-09-21T06:26:32.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.933+0000 7f7fb7fff640 1 -- 172.21.15.31:0/3788372762 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4035030 con 0x7f7fd010a9e0 2024-09-21T06:26:32.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.934+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd01bdfa0 con 0x7f7fd010a9e0 2024-09-21T06:26:32.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.934+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd01be340 con 0x7f7fd010a9e0 2024-09-21T06:26:32.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.939+0000 7f7fdb9f7740 1 Processor -- start 2024-09-21T06:26:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.942+0000 7f7fdb9f7740 1 -- start start 2024-09-21T06:26:32.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d2e350 0x55e8980d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980d5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 0x55e8980d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e89804be60 con 0x55e897d0d670 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e89803b290 con 0x55e897d2e350 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e89803f0a0 con 0x55e897d05490 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 0x55e8980d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 0x55e8980d5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44056/0 (socket says 172.21.15.31:44056) 2024-09-21T06:26:32.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.947+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3492977022 learned_addr learned my addr 172.21.15.31:0/3492977022 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:32.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.948+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3492977022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 msgr2=0x55e8980d5970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:26:32.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.948+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980d5970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.948+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3492977022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d2e350 msgr2=0x55e8980d5430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:26:32.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.948+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d2e350 0x55e8980d5430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.948+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3492977022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e897d99930 con 0x55e897d0d670 2024-09-21T06:26:32.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.949+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 0x55e8980d5eb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009dc0 tx=0x7f7fc800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcc87fc46b5e4b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.949+0000 7f7fb57fa640 1 -- 172.21.15.31:0/3492977022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8010430 con 0x55e897d0d670 2024-09-21T06:26:32.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.950+0000 7f7fb57fa640 1 -- 172.21.15.31:0/3492977022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc8010a80 con 0x55e897d0d670 2024-09-21T06:26:32.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.950+0000 7f7fb7fff640 1 -- 172.21.15.31:0/3788372762 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7fc4042070 con 0x7f7fd010a9e0 2024-09-21T06:26:32.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.951+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3492977022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 msgr2=0x55e8980d5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.951+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 0x55e8980d5eb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009dc0 tx=0x7f7fc800e720 comp rx=0 tx=0).stop 2024-09-21T06:26:32.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.951+0000 7f7fb7fff640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fb80b67d0 0x7f7fb80b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.951+0000 7f7fcf7fe640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fb80b67d0 0x7f7fb80b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.952+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3492977022 shutdown_connections 2024-09-21T06:26:32.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.952+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d0d670 0x55e8980d5eb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.952+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980d5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.952+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3492977022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d2e350 0x55e8980d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.952+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3492977022 >> 172.21.15.31:0/3492977022 conn(0x55e897d22740 msgr2=0x55e8980d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:32.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.952+0000 7f7fb7fff640 1 -- 172.21.15.31:0/3788372762 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f7fc4107c40 con 0x7f7fd010a9e0 2024-09-21T06:26:32.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.956+0000 7f7fcf7fe640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fb80b67d0 0x7f7fb80b8ce0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00095b0 tx=0x7f7fc0006c20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.956+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3492977022 shutdown_connections 2024-09-21T06:26:32.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.956+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3492977022 wait complete. 2024-09-21T06:26:32.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.957+0000 7f7fdb9f7740 1 -- start start 2024-09-21T06:26:32.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.957+0000 7f7fdb9f7740 1 Processor -- start 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55e897e9c890 con 0x55e897d05490 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e897fd5d40 con 0x55e897d0d670 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e8980e3c60 con 0x55e897d2e350 2024-09-21T06:26:32.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fdb9f7740 10 client.0 _ll_register_callbacks cb 0x55e898039c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:40584/0 (socket says 172.21.15.31:40584) 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fcf7fe640 1 -- 172.21.15.31:0/3327919238 learned_addr learned my addr 172.21.15.31:0/3327919238 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.958+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.959+0000 7f7fcf7fe640 1 -- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 msgr2=0x55e8980e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.959+0000 7f7fcf7fe640 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.959+0000 7f7fcf7fe640 1 -- 172.21.15.31:0/3327919238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 msgr2=0x55e8980e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:32.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.959+0000 7f7fcf7fe640 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.959+0000 7f7fcf7fe640 1 -- 172.21.15.31:0/3327919238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8980ed570 con 0x55e897d2e350 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.959+0000 7f7fcf7fe640 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ed030 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0013f50 tx=0x7f7fc00047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3744e8d23d0fc057 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.960+0000 7f7fb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.960+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/3327919238 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc001d5f0 con 0x55e897d2e350 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.960+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/3327919238 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f7fc001d790 con 0x55e897d2e350 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.961+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc8010a80 con 0x55e897d2e350 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.961+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/3327919238 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7fc002b050 con 0x55e897d2e350 2024-09-21T06:26:32.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.961+0000 7f7fdb9f7740 10 client.28252 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.961+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e8980efeb0 con 0x55e897d2e350 2024-09-21T06:26:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.961+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/3327919238 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc8010a80 con 0x55e897d2e350 2024-09-21T06:26:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/3327919238 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7fc0012070 con 0x55e897d2e350 2024-09-21T06:26:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fdb9f7740 10 client.28252 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fdb9f7740 10 client.28252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:32.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fdb9f7740 20 client.28252 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:32.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897ceccd0 0x55e8980f3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fdb9f7740 4 client.28252 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:32.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.962+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e89804ddb0 con 0x55e897ceccd0 2024-09-21T06:26:32.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.963+0000 7f7fceffd640 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897ceccd0 0x55e8980f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.964+0000 7f7fceffd640 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897ceccd0 0x55e8980f3930 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcf0747a2946248 server_cookie=77d8f695aae7cc8a in_seq=0 out_seq=0 2024-09-21T06:26:32.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.964+0000 7f7fb2ff5640 10 client.28252 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.965+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/3327919238 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e89804ddb0 con 0x55e897ceccd0 2024-09-21T06:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.965+0000 7f7fb2ff5640 10 client.28252 handle_command_reply: tid=0 2024-09-21T06:26:32.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.965+0000 7f7fdb9f7740 1 client.28252 shutdown 2024-09-21T06:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 msgr2=0x55e8980ed030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ed030 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0013f50 tx=0x7f7fc00047d0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 shutdown_connections 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897ceccd0 0x55e8980f3930 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ed030 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fb2ff5640 0 client.28252 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3327919238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.966+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 >> 172.21.15.31:0/3327919238 conn(0x55e897d22740 msgr2=0x55e8980d85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.967+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 shutdown_connections 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.967+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3327919238 wait complete. 2024-09-21T06:26:32.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.967+0000 7f7fdb9f7740 20 client.28252 trim_cache size 0 max 16384 2024-09-21T06:26:32.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 Processor -- start 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 -- start start 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00c8d0 con 0x55e897d05490 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00f130 con 0x55e897d0d670 2024-09-21T06:26:32.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00cf20 con 0x55e897d2e350 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fceffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980ecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980ecf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59730/0 (socket says 172.21.15.31:59730) 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3295115086 learned_addr learned my addr 172.21.15.31:0/3295115086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.993+0000 7f7fcf7fe640 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3295115086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 msgr2=0x55e8980ddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ddf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3295115086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 msgr2=0x55e8980db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fce7fc640 1 -- 172.21.15.31:0/3295115086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8980dc5a0 con 0x55e897d0d670 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fceffd640 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980ecf30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009fd0 tx=0x7f7fc800ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960c05e80681d284 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:32.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fb4ff9640 1 -- 172.21.15.31:0/3295115086 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8027070 con 0x55e897d0d670 2024-09-21T06:26:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fb4ff9640 1 -- 172.21.15.31:0/3295115086 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc80049c0 con 0x55e897d0d670 2024-09-21T06:26:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.994+0000 7f7fb4ff9640 1 -- 172.21.15.31:0/3295115086 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8010900 con 0x55e897d0d670 2024-09-21T06:26:32.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3295115086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 msgr2=0x55e8980ecf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980ecf30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009fd0 tx=0x7f7fc800ef50 comp rx=0 tx=0).stop 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3295115086 shutdown_connections 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d2e350 0x55e8980ddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980ecf30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/3295115086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d05490 0x55e8980db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3295115086 >> 172.21.15.31:0/3295115086 conn(0x55e897d22740 msgr2=0x55e8980df590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3295115086 shutdown_connections 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/3295115086 wait complete. 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.996+0000 7f7fdb9f7740 1 -- start start 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 Processor -- start 2024-09-21T06:26:32.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d2e350 0x55e8980e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00d5f0 con 0x55e897d2e350 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00ea60 con 0x55e897d0d670 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc00dcc0 con 0x55e897d05490 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fdb9f7740 10 client.0 _ll_register_callbacks cb 0x7f7fc00092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fceffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fcf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d2e350 0x55e8980e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fce7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59734/0 (socket says 172.21.15.31:59734) 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.997+0000 7f7fce7fc640 1 -- 172.21.15.31:0/438781029 learned_addr learned my addr 172.21.15.31:0/438781029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fceffd640 1 -- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 msgr2=0x55e8980e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fceffd640 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fceffd640 1 -- 172.21.15.31:0/438781029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d2e350 msgr2=0x55e8980e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fceffd640 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d2e350 0x55e8980e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fceffd640 1 -- 172.21.15.31:0/438781029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89815d330 con 0x55e897d05490 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fceffd640 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980e4f20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7fc41053a0 tx=0x7f7fc402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90db450913b7efbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/438781029 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40d74c0 con 0x55e897d05490 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/438781029 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f7fc4105620 con 0x55e897d05490 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc80049c0 con 0x55e897d05490 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/438781029 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7fc40c9d10 con 0x55e897d05490 2024-09-21T06:26:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fdb9f7740 10 client.28258 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.998+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e89815e5a0 con 0x55e897d05490 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/438781029 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc4031430 con 0x55e897d05490 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/438781029 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7fc40d2050 con 0x55e897d05490 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fdb9f7740 10 client.28258 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fdb9f7740 10 client.28258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fdb9f7740 20 client.28258 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897cc2080 0x55e898160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fdb9f7740 4 client.28258 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7fc80049c0 con 0x55e897cc2080 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:32.999+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897cc2080 0x55e898160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.000+0000 7f7fce7fc640 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897cc2080 0x55e898160db0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662fe9e00e8bbe0 server_cookie=e97e8ccbc0a496f7 in_seq=0 out_seq=0 2024-09-21T06:26:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.000+0000 7f7fb2ff5640 10 client.28258 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fb2ff5640 1 -- 172.21.15.31:0/438781029 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7fc80049c0 con 0x55e897cc2080 2024-09-21T06:26:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fb2ff5640 10 client.28258 handle_command_reply: tid=0 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 client.28258 shutdown 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 msgr2=0x55e8980e4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980e4f20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7fc41053a0 tx=0x7f7fc402a410 comp rx=0 tx=0).stop 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 shutdown_connections 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55e897cc2080 0x55e898160db0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e897d2e350 0x55e8980e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e897d0d670 0x55e8980e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.003+0000 7f7fdb9f7740 1 --2- 172.21.15.31:0/438781029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e897d05490 0x55e8980e4f20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.004+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 >> 172.21.15.31:0/438781029 conn(0x55e897d22740 msgr2=0x55e8980df150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.004+0000 7f7fb2ff5640 0 client.28258 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.004+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 shutdown_connections 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.004+0000 7f7fdb9f7740 1 -- 172.21.15.31:0/438781029 wait complete. 2024-09-21T06:26:33.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.004+0000 7f7fdb9f7740 20 client.28258 trim_cache size 0 max 16384 2024-09-21T06:26:33.007 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:26:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2024-09-21T06:26:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "fe52f945-9328-4bb2-a75c-107af33e2152", 2024-09-21T06:26:33.007 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2024-09-21T06:26:33.007 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fb80b67d0 msgr2=0x7f7fb80b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fb80b67d0 0x7f7fb80b8ce0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00095b0 tx=0x7f7fc0006c20 comp rx=0 tx=0).stop 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 msgr2=0x7f7fd011c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd011c540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4004770 tx=0x7f7fc4003b90 comp rx=0 tx=0).stop 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 shutdown_connections 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f7fb80b67d0 0x7f7fb80b8ce0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fd011cc30 0x7f7fd0116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7fd0111f40 0x7f7fd0116800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 --2- 172.21.15.31:0/3788372762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fd010a9e0 0x7f7fd011c540 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 >> 172.21.15.31:0/3788372762 conn(0x7f7fd0070ec0 msgr2=0x7f7fd0073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 shutdown_connections 2024-09-21T06:26:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:33.005+0000 7f7fd5483640 1 -- 172.21.15.31:0/3788372762 wait complete. 2024-09-21T06:26:33.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:33 smithi031 ceph-mon[31237]: pgmap v787: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 702 op/s 2024-09-21T06:26:33.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:33 smithi031 ceph-mon[31237]: scrub queued for path: / 2024-09-21T06:26:33.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:33 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:26:33.901 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:33 smithi031 ceph-mon[31237]: scrub summary: active paths [/] 2024-09-21T06:26:33.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:26:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:33 smithi110 ceph-mon[34550]: pgmap v787: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 702 op/s 2024-09-21T06:26:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:33 smithi110 ceph-mon[34550]: scrub queued for path: / 2024-09-21T06:26:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:33 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:26:34.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:33 smithi110 ceph-mon[34550]: scrub summary: active paths [/] 2024-09-21T06:26:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:33 smithi107 ceph-mon[34380]: pgmap v787: 97 pgs: 97 active+clean; 4.3 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 702 op/s 2024-09-21T06:26:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:33 smithi107 ceph-mon[34380]: scrub queued for path: / 2024-09-21T06:26:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:33 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:26:34.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:33 smithi107 ceph-mon[34380]: scrub summary: active paths [/] 2024-09-21T06:26:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:26:34.802 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:34 smithi031 ceph-mon[31237]: scrub summary: idle+waiting paths [/] 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 -- 172.21.15.31:0/1572286306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 msgr2=0x7fe214110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 --2- 172.21.15.31:0/1572286306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 0x7fe214110330 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ca0 tx=0x7fe20c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 -- 172.21.15.31:0/1572286306 shutdown_connections 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 --2- 172.21.15.31:0/1572286306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 0x7fe214110330 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 --2- 172.21.15.31:0/1572286306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe214105310 0x7fe21410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 --2- 172.21.15.31:0/1572286306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2141048e0 0x7fe214104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 -- 172.21.15.31:0/1572286306 >> 172.21.15.31:0/1572286306 conn(0x7fe2140797b0 msgr2=0x7fe214079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.928+0000 7fe21af42640 1 -- 172.21.15.31:0/1572286306 shutdown_connections 2024-09-21T06:26:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.929+0000 7fe21af42640 1 -- 172.21.15.31:0/1572286306 wait complete. 2024-09-21T06:26:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.929+0000 7fe21af42640 1 Processor -- start 2024-09-21T06:26:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.929+0000 7fe21af42640 1 -- start start 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe21af42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 0x7fe21419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe21af42640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe214105310 0x7fe21419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe21af42640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 0x7fe2141a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe21af42640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe214113ac0 con 0x7fe21410de90 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe21af42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe214113940 con 0x7fe2141048e0 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe21af42640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe214113c40 con 0x7fe214105310 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 0x7fe21419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 0x7fe21419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59758/0 (socket says 172.21.15.31:59758) 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 -- 172.21.15.31:0/680232544 learned_addr learned my addr 172.21.15.31:0/680232544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe2194df640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 0x7fe2141a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe20bfff640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe214105310 0x7fe21419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 -- 172.21.15.31:0/680232544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe214105310 msgr2=0x7fe21419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe214105310 0x7fe21419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 -- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 msgr2=0x7fe2141a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 0x7fe2141a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.930+0000 7fe218cde640 1 -- 172.21.15.31:0/680232544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141a25e0 con 0x7fe2141048e0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.931+0000 7fe218cde640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 0x7fe21419c670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e9f0 tx=0x7fe21000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.931+0000 7fe209ffb640 1 -- 172.21.15.31:0/680232544 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000cd80 con 0x7fe2141048e0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.931+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2141a28d0 con 0x7fe2141048e0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.932+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141a9030 con 0x7fe2141048e0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.932+0000 7fe209ffb640 1 -- 172.21.15.31:0/680232544 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2100054d0 con 0x7fe2141048e0 2024-09-21T06:26:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.932+0000 7fe209ffb640 1 -- 172.21.15.31:0/680232544 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210010690 con 0x7fe2141048e0 2024-09-21T06:26:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.936+0000 7fe209ffb640 1 -- 172.21.15.31:0/680232544 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe2100046c0 con 0x7fe2141048e0 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 Processor -- start 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe209ffb640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1fc0b67d0 0x7fe1fc0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 -- start start 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2194df640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1fc0b67d0 0x7fe1fc0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe209ffb640 1 -- 172.21.15.31:0/680232544 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fe2100e7970 con 0x7fe2141048e0 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e06650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e06b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559fe6d7ce50 con 0x559fe6a361b0 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559fe6d6c280 con 0x559fe6a5f070 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe2214b6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559fe6d70090 con 0x559fe6a3e390 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe20bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e06b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe20bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e06b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44098/0 (socket says 172.21.15.31:44098) 2024-09-21T06:26:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.937+0000 7fe20bfff640 1 -- 172.21.15.31:0/2980671796 learned_addr learned my addr 172.21.15.31:0/2980671796 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe218cde640 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e06650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe2194df640 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe2194df640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1fc0b67d0 0x7fe1fc0b8ce0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006010 tx=0x7fe20c0093e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe20bfff640 1 -- 172.21.15.31:0/2980671796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 msgr2=0x559fe6e070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe20bfff640 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe20bfff640 1 -- 172.21.15.31:0/2980671796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 msgr2=0x559fe6e06650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe20bfff640 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e06650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe20bfff640 1 -- 172.21.15.31:0/2980671796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe6aca650 con 0x559fe6a361b0 2024-09-21T06:26:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe20bfff640 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e06b90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe20400bc10 tx=0x7fe20400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933fb767ab475d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe1faffd640 1 -- 172.21.15.31:0/2980671796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204015070 con 0x559fe6a361b0 2024-09-21T06:26:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.938+0000 7fe1faffd640 1 -- 172.21.15.31:0/2980671796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe20401a910 con 0x559fe6a361b0 2024-09-21T06:26:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.939+0000 7fe1faffd640 1 -- 172.21.15.31:0/2980671796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204011810 con 0x559fe6a361b0 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 -- 172.21.15.31:0/2980671796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 msgr2=0x559fe6e06b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e06b90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe20400bc10 tx=0x7fe20400f730 comp rx=0 tx=0).stop 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 -- 172.21.15.31:0/2980671796 shutdown_connections 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e06b90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2980671796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e06650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.941+0000 7fe2214b6740 1 -- 172.21.15.31:0/2980671796 >> 172.21.15.31:0/2980671796 conn(0x559fe6a53460 msgr2=0x559fe6e04540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.942+0000 7fe2214b6740 1 -- 172.21.15.31:0/2980671796 shutdown_connections 2024-09-21T06:26:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.942+0000 7fe2214b6740 1 -- 172.21.15.31:0/2980671796 wait complete. 2024-09-21T06:26:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.942+0000 7fe2214b6740 1 -- start start 2024-09-21T06:26:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.942+0000 7fe2214b6740 1 Processor -- start 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e11cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e12230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x559fe6bcd700 con 0x559fe6a361b0 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559fe6d07110 con 0x559fe6a5f070 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559fe6e14ed0 con 0x559fe6a3e390 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2214b6740 10 client.0 _ll_register_callbacks cb 0x559fe6d20990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59776/0 (socket says 172.21.15.31:59776) 2024-09-21T06:26:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 -- 172.21.15.31:0/468527533 learned_addr learned my addr 172.21.15.31:0/468527533 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe218cde640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e11cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe20bfff640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e12230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 -- 172.21.15.31:0/468527533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 msgr2=0x559fe6e12230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e12230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 -- 172.21.15.31:0/468527533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 msgr2=0x559fe6e11cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e11cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.943+0000 7fe2194df640 1 -- 172.21.15.31:0/468527533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe6d34c40 con 0x559fe6a5f070 2024-09-21T06:26:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.950+0000 7fe2194df640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e127b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0316b0 tx=0x7fe20c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a619cc79371c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:34.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.950+0000 7fe1f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:26:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.954+0000 7fe1f3fff640 1 -- 172.21.15.31:0/468527533 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c004240 con 0x559fe6a5f070 2024-09-21T06:26:34.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.954+0000 7fe1f3fff640 1 -- 172.21.15.31:0/468527533 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fe20c0043e0 con 0x559fe6a5f070 2024-09-21T06:26:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.954+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe20401a910 con 0x559fe6a5f070 2024-09-21T06:26:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.958+0000 7fe1f3fff640 1 -- 172.21.15.31:0/468527533 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe20c0045e0 con 0x559fe6a5f070 2024-09-21T06:26:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.958+0000 7fe2214b6740 10 client.28232 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:34.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.958+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559fe6e20c00 con 0x559fe6a5f070 2024-09-21T06:26:34.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe1f3fff640 1 -- 172.21.15.31:0/468527533 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe20c04d070 con 0x559fe6a5f070 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe1f3fff640 1 -- 172.21.15.31:0/468527533 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe20c0305c0 con 0x559fe6a5f070 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe2214b6740 10 client.28232 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe2214b6740 10 client.28232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe2214b6740 20 client.28232 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe2214b6740 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6a1d9f0 0x559fe6e24120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe2214b6740 4 client.28232 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.967+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe20401a910 con 0x559fe6a1d9f0 2024-09-21T06:26:34.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.968+0000 7fe218cde640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6a1d9f0 0x559fe6e24120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.975+0000 7fe218cde640 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6a1d9f0 0x559fe6e24120 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f9e27fef23502b server_cookie=acb6c0fcf197fe7c in_seq=0 out_seq=0 2024-09-21T06:26:34.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.976+0000 7fe1f3fff640 10 client.28232 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.980+0000 7fe1f3fff640 1 -- 172.21.15.31:0/468527533 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe20401a910 con 0x559fe6a1d9f0 2024-09-21T06:26:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.980+0000 7fe1f3fff640 10 client.28232 handle_command_reply: tid=0 2024-09-21T06:26:34.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.980+0000 7fe2214b6740 1 client.28232 shutdown 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.982+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 msgr2=0x559fe6e127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.982+0000 7fe2214b6740 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e127b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0316b0 tx=0x7fe20c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 shutdown_connections 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe2214b6740 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6a1d9f0 0x559fe6e24120 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe2214b6740 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a5f070 0x559fe6e127b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe2214b6740 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a3e390 0x559fe6e12230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe2214b6740 1 --2- 172.21.15.31:0/468527533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a361b0 0x559fe6e11cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 >> 172.21.15.31:0/468527533 conn(0x559fe6a53460 msgr2=0x559fe6e097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:34.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.984+0000 7fe1f3fff640 0 client.28232 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.988+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 shutdown_connections 2024-09-21T06:26:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.988+0000 7fe2214b6740 1 -- 172.21.15.31:0/468527533 wait complete. 2024-09-21T06:26:34.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:34.988+0000 7fe2214b6740 20 client.28232 trim_cache size 0 max 16384 2024-09-21T06:26:35.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 Processor -- start 2024-09-21T06:26:35.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 -- start start 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 0x559fe6e0eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e0f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000c890 con 0x559fe6a3e390 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000f0f0 con 0x559fe6a361b0 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.019+0000 7fe2214b6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000cee0 con 0x559fe6a5f070 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 0x559fe6e0eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe218cde640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 0x559fe6e0eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44110/0 (socket says 172.21.15.31:44110) 2024-09-21T06:26:35.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 -- 172.21.15.31:0/2090910010 learned_addr learned my addr 172.21.15.31:0/2090910010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe218cde640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:59784/0 (socket says 172.21.15.31:59784) 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe2194df640 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e0f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 -- 172.21.15.31:0/2090910010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 msgr2=0x559fe6e0f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e0f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 -- 172.21.15.31:0/2090910010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 msgr2=0x559fe6e045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.020+0000 7fe20bfff640 1 -- 172.21.15.31:0/2090910010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe6e14430 con 0x559fe6a3e390 2024-09-21T06:26:35.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.021+0000 7fe20bfff640 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 0x559fe6e0eee0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b5a0 tx=0x7fe20400b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb83c5bb42809e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:35.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.021+0000 7fe1fa7fc640 1 -- 172.21.15.31:0/2090910010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204029070 con 0x559fe6a3e390 2024-09-21T06:26:35.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.021+0000 7fe1fa7fc640 1 -- 172.21.15.31:0/2090910010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe204004680 con 0x559fe6a3e390 2024-09-21T06:26:35.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.021+0000 7fe1fa7fc640 1 -- 172.21.15.31:0/2090910010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204015050 con 0x559fe6a3e390 2024-09-21T06:26:35.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.022+0000 7fe2214b6740 1 -- 172.21.15.31:0/2090910010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 msgr2=0x559fe6e0eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:35.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.022+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 0x559fe6e0eee0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b5a0 tx=0x7fe20400b5d0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.022+0000 7fe2214b6740 1 -- 172.21.15.31:0/2090910010 shutdown_connections 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.022+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e0f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.022+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe6a3e390 0x559fe6e0eee0 secure :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b5a0 tx=0x7fe20400b5d0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.023+0000 7fe2214b6740 1 --2- 172.21.15.31:0/2090910010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.023+0000 7fe2214b6740 1 -- 172.21.15.31:0/2090910010 >> 172.21.15.31:0/2090910010 conn(0x559fe6a53460 msgr2=0x559fe6e03ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.023+0000 7fe2214b6740 1 -- 172.21.15.31:0/2090910010 shutdown_connections 2024-09-21T06:26:35.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.023+0000 7fe2214b6740 1 -- 172.21.15.31:0/2090910010 wait complete. 2024-09-21T06:26:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.023+0000 7fe2214b6740 1 -- start start 2024-09-21T06:26:35.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.023+0000 7fe2214b6740 1 Processor -- start 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e16f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe69f2da0 0x559fe6e8dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000d5b0 con 0x559fe69f2da0 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000ea20 con 0x559fe6a361b0 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe20000dc80 con 0x559fe6a5f070 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2214b6740 10 client.0 _ll_register_callbacks cb 0x7fe20c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2194df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe69f2da0 0x559fe6e8dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2194df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe69f2da0 0x559fe6e8dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44120/0 (socket says 172.21.15.31:44120) 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe2194df640 1 -- 172.21.15.31:0/447526227 learned_addr learned my addr 172.21.15.31:0/447526227 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.024+0000 7fe20bfff640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e16f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe218cde640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe20bfff640 1 -- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 msgr2=0x559fe6e169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe20bfff640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe20bfff640 1 -- 172.21.15.31:0/447526227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe69f2da0 msgr2=0x559fe6e8dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe20bfff640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe69f2da0 0x559fe6e8dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe20bfff640 1 -- 172.21.15.31:0/447526227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe6e8e120 con 0x559fe6a5f070 2024-09-21T06:26:35.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:34 smithi110 ceph-mon[34550]: scrub summary: idle+waiting paths [/] 2024-09-21T06:26:35.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe20bfff640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e16f60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x559fe6d071f0 tx=0x7fe204004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d263ade052cd8cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:35.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.025+0000 7fe1f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-09-21T06:26:35.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.026+0000 7fe1f3fff640 1 -- 172.21.15.31:0/447526227 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204029070 con 0x559fe6a5f070 2024-09-21T06:26:35.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.026+0000 7fe1f3fff640 1 -- 172.21.15.31:0/447526227 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fe204005200 con 0x559fe6a5f070 2024-09-21T06:26:35.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.026+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe204004680 con 0x559fe6a5f070 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.026+0000 7fe1f3fff640 1 -- 172.21.15.31:0/447526227 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe204015050 con 0x559fe6a5f070 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.026+0000 7fe2214b6740 10 client.28276 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.026+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559fe6e8f3b0 con 0x559fe6a5f070 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe1f3fff640 1 -- 172.21.15.31:0/447526227 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe204004680 con 0x559fe6a5f070 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe1f3fff640 1 -- 172.21.15.31:0/447526227 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe204023070 con 0x559fe6a5f070 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe2214b6740 10 client.28276 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:35.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe2214b6740 10 client.28276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe2214b6740 20 client.28276 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe2214b6740 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6e8f910 0x559fe6e91d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe2214b6740 4 client.28276 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.027+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fe6d7eda0 con 0x559fe6e8f910 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.029+0000 7fe218cde640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6e8f910 0x559fe6e91d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.029+0000 7fe218cde640 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6e8f910 0x559fe6e91d30 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b1976a4dd4b413 server_cookie=a9fae59886062321 in_seq=0 out_seq=0 2024-09-21T06:26:35.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.029+0000 7fe1f3fff640 10 client.28276 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe1f3fff640 1 -- 172.21.15.31:0/447526227 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9942 (crc 0 0 0) 0x559fe6d7eda0 con 0x559fe6e8f910 2024-09-21T06:26:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe1f3fff640 10 client.28276 handle_command_reply: tid=0 2024-09-21T06:26:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 client.28276 shutdown 2024-09-21T06:26:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 msgr2=0x559fe6e16f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e16f60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x559fe6d071f0 tx=0x7fe204004a60 comp rx=0 tx=0).stop 2024-09-21T06:26:35.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 shutdown_connections 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x559fe6e8f910 0x559fe6e91d30 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559fe69f2da0 0x559fe6e8dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fe6a5f070 0x559fe6e16f60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 --2- 172.21.15.31:0/447526227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559fe6a361b0 0x559fe6e169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe1f3fff640 0 client.28276 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 >> 172.21.15.31:0/447526227 conn(0x559fe6a53460 msgr2=0x559fe6e131a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 shutdown_connections 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 1 -- 172.21.15.31:0/447526227 wait complete. 2024-09-21T06:26:35.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.030+0000 7fe2214b6740 20 client.28276 trim_cache size 0 max 16384 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "fe52f945-9328-4bb2-a75c-107af33e2152": { 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "fe52f945-9328-4bb2-a75c-107af33e2152", 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.034 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "97dcd776-5fd7-4ec8-ada7-f276bb218390": { 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:51.458278+0000", 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:10.069920+0000", 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.035 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:26:35.036 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "fe52f945-9328-4bb2-a75c-107af33e2152": { 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:26:33.002635+0000", 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.038 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:26:35.039 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.040 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:26:35.042 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.044 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:26:35.045 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.047 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:26:35.048 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.050 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:26:35.051 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.053 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:26:35.054 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:26:35.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1fc0b67d0 msgr2=0x7fe1fc0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1fc0b67d0 0x7fe1fc0b8ce0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006010 tx=0x7fe20c0093e0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 msgr2=0x7fe21419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 0x7fe21419c670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe21000e9f0 tx=0x7fe21000edf0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 shutdown_connections 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1fc0b67d0 0x7fe1fc0b8ce0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe21410de90 0x7fe2141a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe214105310 0x7fe21419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 --2- 172.21.15.31:0/680232544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2141048e0 0x7fe21419c670 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 >> 172.21.15.31:0/680232544 conn(0x7fe2140797b0 msgr2=0x7fe2141005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 shutdown_connections 2024-09-21T06:26:35.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:35.032+0000 7fe21af42640 1 -- 172.21.15.31:0/680232544 wait complete. 2024-09-21T06:26:35.057 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:34 smithi107 ceph-mon[34380]: scrub summary: idle+waiting paths [/] 2024-09-21T06:26:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 895.0 s, 1183.0 tps, lat 85.800 ms stddev 100.060 2024-09-21T06:26:35.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:35 smithi031 ceph-mon[31237]: pgmap v788: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 786 op/s 2024-09-21T06:26:36.014 INFO:tasks.cephfs.filesystem:scrub status for tag:fe52f945-9328-4bb2-a75c-107af33e2152 - {'path': '/', 'tag': 'fe52f945-9328-4bb2-a75c-107af33e2152', 'options': 'recursive,force'} 2024-09-21T06:26:36.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:35 smithi110 ceph-mon[34550]: pgmap v788: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 786 op/s 2024-09-21T06:26:36.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:35 smithi107 ceph-mon[34380]: pgmap v788: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 786 op/s 2024-09-21T06:26:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:37 smithi031 ceph-mon[31237]: pgmap v789: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 676 op/s 2024-09-21T06:26:37.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:37 smithi031 ceph-mon[31237]: scrub summary: idle 2024-09-21T06:26:38.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:37 smithi110 ceph-mon[34550]: pgmap v789: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 676 op/s 2024-09-21T06:26:38.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:37 smithi110 ceph-mon[34550]: scrub summary: idle 2024-09-21T06:26:38.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:37 smithi107 ceph-mon[34380]: pgmap v789: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 676 op/s 2024-09-21T06:26:38.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:37 smithi107 ceph-mon[34380]: scrub summary: idle 2024-09-21T06:26:39.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:39 smithi031 ceph-mon[31237]: pgmap v790: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 12 MiB/s wr, 829 op/s 2024-09-21T06:26:40.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:39 smithi110 ceph-mon[34550]: pgmap v790: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 12 MiB/s wr, 829 op/s 2024-09-21T06:26:40.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:39 smithi107 ceph-mon[34380]: pgmap v790: 97 pgs: 97 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 12 MiB/s wr, 829 op/s 2024-09-21T06:26:40.204 INFO:teuthology.orchestra.run.smithi031.stderr:progress: 900.0 s, 1240.8 tps, lat 81.174 ms stddev 131.311 2024-09-21T06:26:40.400 INFO:teuthology.orchestra.run.smithi031.stdout:transaction type: 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:scaling factor: 32 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:query mode: simple 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:number of clients: 100 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:number of threads: 4 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:duration: 900 s 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:number of transactions actually processed: 2055404 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:latency average = 43.785 ms 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:latency stddev = 134.337 ms 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:tps = 2283.283812 (including connections establishing) 2024-09-21T06:26:40.401 INFO:teuthology.orchestra.run.smithi031.stdout:tps = 2283.290186 (excluding connections establishing) 2024-09-21T06:26:40.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-21T06:26:41.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:41 smithi031 ceph-mon[31237]: pgmap v791: 97 pgs: 97 active+clean; 4.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-09-21T06:26:42.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:41 smithi110 ceph-mon[34550]: pgmap v791: 97 pgs: 97 active+clean; 4.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-09-21T06:26:42.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:41 smithi107 ceph-mon[34380]: pgmap v791: 97 pgs: 97 active+clean; 4.4 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-09-21T06:26:43.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:43 smithi031 ceph-mon[31237]: pgmap v792: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.59k op/s 2024-09-21T06:26:44.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:43 smithi110 ceph-mon[34550]: pgmap v792: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.59k op/s 2024-09-21T06:26:44.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:43 smithi107 ceph-mon[34380]: pgmap v792: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.59k op/s 2024-09-21T06:26:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.061+0000 7fe200006640 1 -- 172.21.15.31:0/6555191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 msgr2=0x7fe1f810aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.061+0000 7fe200006640 1 --2- 172.21.15.31:0/6555191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 0x7fe1f810aa70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009900 tx=0x7fe1f002a420 comp rx=0 tx=0).stop 2024-09-21T06:26:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.066+0000 7fe200006640 1 -- 172.21.15.31:0/6555191 shutdown_connections 2024-09-21T06:26:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.066+0000 7fe200006640 1 --2- 172.21.15.31:0/6555191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1f8110350 0x7fe1f8110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.066+0000 7fe200006640 1 --2- 172.21.15.31:0/6555191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f810fa00 0x7fe1f810fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.066+0000 7fe200006640 1 --2- 172.21.15.31:0/6555191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 0x7fe1f810aa70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.066+0000 7fe200006640 1 -- 172.21.15.31:0/6555191 >> 172.21.15.31:0/6555191 conn(0x7fe1f8070dd0 msgr2=0x7fe1f80711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:45.071 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.069+0000 7fe200006640 1 -- 172.21.15.31:0/6555191 shutdown_connections 2024-09-21T06:26:45.074 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.072+0000 7fe200006640 1 -- 172.21.15.31:0/6555191 wait complete. 2024-09-21T06:26:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.073+0000 7fe200006640 1 Processor -- start 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.073+0000 7fe200006640 1 -- start start 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.073+0000 7fe200006640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 0x7fe1f8120540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.073+0000 7fe200006640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1f810fa00 0x7fe1f8119800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.073+0000 7fe200006640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 0x7fe1f8119e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.074+0000 7fe200006640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81228a0 con 0x7fe1f810a660 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.074+0000 7fe200006640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8122720 con 0x7fe1f8110350 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.074+0000 7fe200006640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8122a20 con 0x7fe1f810fa00 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.074+0000 7fe1fe5a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 0x7fe1f8119e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.074+0000 7fe1fe5a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 0x7fe1f8119e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55878/0 (socket says 172.21.15.31:55878) 2024-09-21T06:26:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.074+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/3069996188 learned_addr learned my addr 172.21.15.31:0/3069996188 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.075+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/3069996188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1f810fa00 msgr2=0x7fe1f8119800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:26:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.075+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1f810fa00 0x7fe1f8119800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.075+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 msgr2=0x7fe1f8120540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.075+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 0x7fe1f8120540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.077 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.075+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/3069996188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f811f1a0 con 0x7fe1f8110350 2024-09-21T06:26:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.079+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 0x7fe1f8119e80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800cad0 tx=0x7fe1e800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.084+0000 7fe1f6ffd640 1 -- 172.21.15.31:0/3069996188 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800ae20 con 0x7fe1f8110350 2024-09-21T06:26:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.084+0000 7fe1f6ffd640 1 -- 172.21.15.31:0/3069996188 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8005020 con 0x7fe1f8110350 2024-09-21T06:26:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.084+0000 7fe1f6ffd640 1 -- 172.21.15.31:0/3069996188 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800f6b0 con 0x7fe1f8110350 2024-09-21T06:26:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.085+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f811f490 con 0x7fe1f8110350 2024-09-21T06:26:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.085+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f811f930 con 0x7fe1f8110350 2024-09-21T06:26:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.095+0000 7fe1f6ffd640 1 -- 172.21.15.31:0/3069996188 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe1e80051d0 con 0x7fe1f8110350 2024-09-21T06:26:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.095+0000 7fe20704a740 1 Processor -- start 2024-09-21T06:26:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 -- start start 2024-09-21T06:26:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d7e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b02f0 0x558ba9d7eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558ba9cf36f0 con 0x558ba99b84d0 2024-09-21T06:26:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ba9ce2b20 con 0x558ba99d91b0 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe20704a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558ba9ce6930 con 0x558ba99b02f0 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.105+0000 7fe1fdda2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d7e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d7e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55884/0 (socket says 172.21.15.31:55884) 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 -- 172.21.15.31:0/1680721546 learned_addr learned my addr 172.21.15.31:0/1680721546 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 -- 172.21.15.31:0/1680721546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b02f0 msgr2=0x558ba9d7eef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:26:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b02f0 0x558ba9d7eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 -- 172.21.15.31:0/1680721546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 msgr2=0x558ba9d7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 -- 172.21.15.31:0/1680721546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba9ca8530 con 0x558ba99d91b0 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.106+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d7e9b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005f70 tx=0x7fe1f0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a98a7310dc8cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.107+0000 7fe1f6ffd640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1e00b6880 0x7fe1e00b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.108+0000 7fe1f6ffd640 1 -- 172.21.15.31:0/3069996188 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fe1e80e7460 con 0x7fe1f8110350 2024-09-21T06:26:45.113 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.108+0000 7fe1dffff640 1 -- 172.21.15.31:0/1680721546 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f003e070 con 0x558ba99d91b0 2024-09-21T06:26:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.111+0000 7fe1dffff640 1 -- 172.21.15.31:0/1680721546 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f0005500 con 0x558ba99d91b0 2024-09-21T06:26:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1e00b6880 0x7fe1e00b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 -- 172.21.15.31:0/1680721546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 msgr2=0x558ba9d7e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d7e9b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005f70 tx=0x7fe1f0000bc0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 -- 172.21.15.31:0/1680721546 shutdown_connections 2024-09-21T06:26:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b02f0 0x558ba9d7eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 --2- 172.21.15.31:0/1680721546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d7e9b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.112+0000 7fe20704a740 1 -- 172.21.15.31:0/1680721546 >> 172.21.15.31:0/1680721546 conn(0x558ba99cd300 msgr2=0x558ba9d7c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.116+0000 7fe20704a740 1 -- 172.21.15.31:0/1680721546 shutdown_connections 2024-09-21T06:26:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.116+0000 7fe20704a740 1 -- 172.21.15.31:0/1680721546 wait complete. 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.117+0000 7fe20704a740 1 -- start start 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.117+0000 7fe20704a740 1 Processor -- start 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.117+0000 7fe20704a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d89f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.117+0000 7fe20704a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b84d0 0x558ba9d8a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.117+0000 7fe20704a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d8a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe20704a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x558ba9c1aab0 con 0x558ba99b02f0 2024-09-21T06:26:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe20704a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ba9c88730 con 0x558ba99d91b0 2024-09-21T06:26:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe20704a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558ba9d8d170 con 0x558ba99b84d0 2024-09-21T06:26:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe20704a740 10 client.0 _ll_register_callbacks cb 0x558ba9cdc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe1fdda2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d89f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe1fdda2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d89f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39188/0 (socket says 172.21.15.31:39188) 2024-09-21T06:26:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.118+0000 7fe1fdda2640 1 -- 172.21.15.31:0/40047119 learned_addr learned my addr 172.21.15.31:0/40047119 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.119+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d8a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/40047119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b84d0 msgr2=0x558ba9d8a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b84d0 0x558ba9d8a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/40047119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 msgr2=0x558ba9d89f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d89f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/40047119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba9a44790 con 0x558ba99d91b0 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d8a9f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80e4b60 tx=0x7fe1e80ad0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc116cc767b6e9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:26:45.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.123+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/40047119 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e801d040 con 0x558ba99d91b0 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.124+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f0005500 con 0x558ba99d91b0 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.124+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/40047119 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fe1e80a8e30 con 0x558ba99d91b0 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.124+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/40047119 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe1e80aca50 con 0x558ba99d91b0 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.124+0000 7fe20704a740 10 client.28247 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.124+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558ba9abd620 con 0x558ba99d91b0 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.124+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1e00b6880 0x7fe1e00b8d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x558ba9d7d630 tx=0x7fe1ec006bb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:45.128 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.125+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/40047119 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f0005500 con 0x558ba99d91b0 2024-09-21T06:26:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/40047119 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe1e8020060 con 0x558ba99d91b0 2024-09-21T06:26:45.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe20704a740 10 client.28247 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe20704a740 10 client.28247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe20704a740 20 client.28247 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe20704a740 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba9997d80 0x558ba9dbe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe20704a740 4 client.28247 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.126+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ba9cf5640 con 0x558ba9997d80 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.127+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba9997d80 0x558ba9dbe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.127+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba9997d80 0x558ba9dbe5a0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afd9403080a82116 server_cookie=fb415e14940dc4ef in_seq=0 out_seq=0 2024-09-21T06:26:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.127+0000 7fe1dd7fa640 10 client.28247 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.128+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/40047119 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ba9cf5640 con 0x558ba9997d80 2024-09-21T06:26:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.128+0000 7fe1dd7fa640 10 client.28247 handle_command_reply: tid=0 2024-09-21T06:26:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.128+0000 7fe20704a740 1 client.28247 shutdown 2024-09-21T06:26:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 msgr2=0x558ba9d8a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d8a9f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80e4b60 tx=0x7fe1e80ad0b0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 shutdown_connections 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba9997d80 0x558ba9dbe5a0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99d91b0 0x558ba9d8a9f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99b84d0 0x558ba9d8a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 --2- 172.21.15.31:0/40047119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d89f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 >> 172.21.15.31:0/40047119 conn(0x558ba99cd300 msgr2=0x558ba9d7f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe1dd7fa640 0 client.28247 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 shutdown_connections 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.129+0000 7fe20704a740 1 -- 172.21.15.31:0/40047119 wait complete. 2024-09-21T06:26:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.131+0000 7fe20704a740 20 client.28247 trim_cache size 0 max 16384 2024-09-21T06:26:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.173+0000 7fe20704a740 1 Processor -- start 2024-09-21T06:26:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.173+0000 7fe20704a740 1 -- start start 2024-09-21T06:26:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.175+0000 7fe20704a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d7c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.175+0000 7fe20704a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b84d0 0x558ba9d87140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.175+0000 7fe20704a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.175+0000 7fe20704a740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f810b3d0 con 0x558ba99b02f0 2024-09-21T06:26:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.176+0000 7fe20704a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b05d0 con 0x558ba99b84d0 2024-09-21T06:26:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.176+0000 7fe20704a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81ae410 con 0x558ba99d91b0 2024-09-21T06:26:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.176+0000 7fe1fe5a3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.176+0000 7fe1fe5a3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d87680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.31:33182/0 (socket says 172.21.15.31:33182) 2024-09-21T06:26:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.176+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/1072394316 learned_addr learned my addr 172.21.15.31:0/1072394316 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.177+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b84d0 0x558ba9d87140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.178+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d7c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.179+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/1072394316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b84d0 msgr2=0x558ba9d87140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.179+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b84d0 0x558ba9d87140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.179+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/1072394316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 msgr2=0x558ba9d7c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.179+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d7c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.179+0000 7fe1fe5a3640 1 -- 172.21.15.31:0/1072394316 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba9997fb0 con 0x558ba99d91b0 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.180+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d87680 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80b6dd0 tx=0x7fe1e80a91f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbbce227ac574973 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.180+0000 7fe1df7fe640 1 -- 172.21.15.31:0/1072394316 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8058d70 con 0x558ba99d91b0 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.181+0000 7fe1df7fe640 1 -- 172.21.15.31:0/1072394316 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e80b1dc0 con 0x558ba99d91b0 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.181+0000 7fe1df7fe640 1 -- 172.21.15.31:0/1072394316 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8004a40 con 0x558ba99d91b0 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 -- 172.21.15.31:0/1072394316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 msgr2=0x558ba9d87680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d87680 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80b6dd0 tx=0x7fe1e80a91f0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 -- 172.21.15.31:0/1072394316 shutdown_connections 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d87680 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b84d0 0x558ba9d87140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 --2- 172.21.15.31:0/1072394316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b02f0 0x558ba9d7c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.182+0000 7fe20704a740 1 -- 172.21.15.31:0/1072394316 >> 172.21.15.31:0/1072394316 conn(0x558ba99cd300 msgr2=0x558ba9d7c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.183+0000 7fe20704a740 1 -- 172.21.15.31:0/1072394316 shutdown_connections 2024-09-21T06:26:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.183+0000 7fe20704a740 1 -- 172.21.15.31:0/1072394316 wait complete. 2024-09-21T06:26:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.184+0000 7fe20704a740 1 -- start start 2024-09-21T06:26:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.184+0000 7fe20704a740 1 Processor -- start 2024-09-21T06:26:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe20704a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 0x558ba9d92bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe20704a740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d8f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe1fdda2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 0x558ba9d92bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe1fdda2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 0x558ba9d92bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:55908/0 (socket says 172.21.15.31:55908) 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe1fdda2640 1 -- 172.21.15.31:0/2946487206 learned_addr learned my addr 172.21.15.31:0/2946487206 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d8f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81aea90 con 0x558ba99b84d0 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81aff00 con 0x558ba99b02f0 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.185+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d8f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.186+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81af160 con 0x558ba99d91b0 2024-09-21T06:26:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.186+0000 7fe20704a740 10 client.0 _ll_register_callbacks cb 0x7fe1e80a9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.186+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d8f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.187+0000 7fe1fd5a1640 1 -- 172.21.15.31:0/2946487206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 msgr2=0x558ba9d8f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.187+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d8f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.187+0000 7fe1fd5a1640 1 -- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 msgr2=0x558ba9d92bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.187+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 0x558ba9d92bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.187+0000 7fe1fd5a1640 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba9d8fca0 con 0x558ba99b84d0 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.188+0000 7fe1fd5a1640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d8f070 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8071460 tx=0x7fe1ec012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e6f645d2cccb7a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.188+0000 7fe1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.188+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/2946487206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec023020 con 0x558ba99b84d0 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.189+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/2946487206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fe1ec01f850 con 0x558ba99b84d0 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.189+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 0x558ba9d92bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.189+0000 7fe1fe5a3640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d8f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.189+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e80b1dc0 con 0x558ba99b84d0 2024-09-21T06:26:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.189+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/2946487206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe1ec02e050 con 0x558ba99b84d0 2024-09-21T06:26:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.191+0000 7fe20704a740 10 client.18450 fetch_fsmap learned FSMap version 21 2024-09-21T06:26:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.191+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558ba9e27ea0 con 0x558ba99b84d0 2024-09-21T06:26:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/2946487206 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ec01e3a0 con 0x558ba99b84d0 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/2946487206 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe1ec012070 con 0x558ba99b84d0 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe20704a740 10 client.18450 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe20704a740 10 client.18450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe20704a740 20 client.18450 populate_metadata read hostname 'smithi031' 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba996d1c0 0x558ba9e2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe20704a740 4 client.18450 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.194+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe1e80b1dc0 con 0x558ba996d1c0 2024-09-21T06:26:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.195+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba996d1c0 0x558ba9e2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.198+0000 7fe1fdda2640 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba996d1c0 0x558ba9e2a710 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c822cec929e3fee1 server_cookie=186da3c92a81cf75 in_seq=0 out_seq=0 2024-09-21T06:26:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:45.198+0000 7fe1dd7fa640 10 client.18450 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:45.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:45 smithi031 ceph-mon[31237]: pgmap v793: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-09-21T06:26:46.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:45 smithi110 ceph-mon[34550]: pgmap v793: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-09-21T06:26:46.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:45 smithi107 ceph-mon[34380]: pgmap v793: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 19 MiB/s wr, 1.57k op/s 2024-09-21T06:26:47.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:47 smithi031 ceph-mon[31237]: pgmap v794: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-09-21T06:26:48.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:47 smithi110 ceph-mon[34550]: pgmap v794: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-09-21T06:26:48.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:47 smithi107 ceph-mon[34380]: pgmap v794: 97 pgs: 97 active+clean; 4.5 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-09-21T06:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.547+0000 7fe1dd7fa640 1 -- 172.21.15.31:0/2946487206 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe1e80b1dc0 con 0x558ba996d1c0 2024-09-21T06:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.547+0000 7fe1dd7fa640 10 client.18450 handle_command_reply: tid=0 2024-09-21T06:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.547+0000 7fe20704a740 1 client.18450 shutdown 2024-09-21T06:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 msgr2=0x558ba9d8f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d8f070 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8071460 tx=0x7fe1ec012760 comp rx=0 tx=0).stop 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 shutdown_connections 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x558ba996d1c0 0x558ba9e2a710 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ba99d91b0 0x558ba9d8f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558ba99b84d0 0x558ba9d8f070 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe1dd7fa640 0 client.18450 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 --2- 172.21.15.31:0/2946487206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ba99b02f0 0x558ba9d92bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 >> 172.21.15.31:0/2946487206 conn(0x558ba99cd300 msgr2=0x558ba9d7aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 shutdown_connections 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 1 -- 172.21.15.31:0/2946487206 wait complete. 2024-09-21T06:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.548+0000 7fe20704a740 20 client.18450 trim_cache size 0 max 16384 2024-09-21T06:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2024-09-21T06:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2024-09-21T06:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2024-09-21T06:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1e00b6880 msgr2=0x7fe1e00b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1e00b6880 0x7fe1e00b8d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x558ba9d7d630 tx=0x7fe1ec006bb0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 msgr2=0x7fe1f8119e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 0x7fe1f8119e80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800cad0 tx=0x7fe1e800cf90 comp rx=0 tx=0).stop 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 shutdown_connections 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f8110350 0x7fe1f8119e80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1f810fa00 0x7fe1f8119800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fe1e00b6880 0x7fe1e00b8d90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 --2- 172.21.15.31:0/3069996188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1f810a660 0x7fe1f8120540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 >> 172.21.15.31:0/3069996188 conn(0x7fe1f8070dd0 msgr2=0x7fe1f8073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 shutdown_connections 2024-09-21T06:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2024-09-21T06:26:49.549+0000 7fe200006640 1 -- 172.21.15.31:0/3069996188 wait complete. 2024-09-21T06:26:49.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:49 smithi031 ceph-mon[31237]: pgmap v795: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2024-09-21T06:26:50.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:49 smithi110 ceph-mon[34550]: pgmap v795: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2024-09-21T06:26:50.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:49 smithi107 ceph-mon[34380]: pgmap v795: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 21 MiB/s wr, 1.79k op/s 2024-09-21T06:26:50.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T06:26:50.477 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/: 2024-09-21T06:26:50.477 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.477 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 .. 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 data 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data: 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:total 36 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 . 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 .. 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:11 base 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 21 06:11 current_logfiles 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 06:11 global 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 log 2024-09-21T06:26:50.478 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_commit_ts 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_dynshmem 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 06:11 pg_hba.conf 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 06:11 pg_ident.conf 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:26 pg_logical 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 pg_multixact 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_notify 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_replslot 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_serial 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_snapshots 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:26 pg_stat 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:26 pg_stat_tmp 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:26 pg_subtrans 2024-09-21T06:26:50.479 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_tblspc 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pg_twophase 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 06:26 pg_wal 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:16 pg_xact 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 06:11 postgresql.auto.conf 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 06:11 postgresql.conf 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 21 06:11 postmaster.opts 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:11 . 2024-09-21T06:26:50.480 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 1 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 13493 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 06:14 13494 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 pgsql_tmp 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:total 7777 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 . 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:11 .. 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 112 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 113 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 1247 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1247_fsm 2024-09-21T06:26:50.481 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1247_vm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:11 1249 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1249_fsm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1249_vm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:11 1255 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1255_fsm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1255_vm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:11 1259 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1259_fsm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1259_vm 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:11 13344 2024-09-21T06:26:50.482 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13344_fsm 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13344_vm 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13346 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13348 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13349_fsm 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349_vm 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13351 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13353 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13354_fsm 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354_vm 2024-09-21T06:26:50.483 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13356 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13358 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13359_fsm 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359_vm 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13361 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13363 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1417 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1418 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 174 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 175 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2187 2024-09-21T06:26:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2224 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2328 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2336 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2337 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2579 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2600 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2600_fsm 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2600_vm 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2601_fsm 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601_vm 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2602 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2602_fsm 2024-09-21T06:26:50.485 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2602_vm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2603 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2603_fsm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2603_vm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2604 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2605 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2605_fsm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2605_vm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2606_fsm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606_vm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2607 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2607_fsm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2607_vm 2024-09-21T06:26:50.486 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:11 2608 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2608_fsm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2608_vm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:11 2609 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2609_fsm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2609_vm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2610 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2610_fsm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2610_vm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2611 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2612_fsm 2024-09-21T06:26:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612_vm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2613 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2615_fsm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615_vm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2616 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2616_fsm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2616_vm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:11 2617 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2617_fsm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2617_vm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:11 2618 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2618_fsm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2618_vm 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:11 2619 2024-09-21T06:26:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2619_fsm 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2619_vm 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2620 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2650 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2651 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2652 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2653 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2654 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2655 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2656 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2657 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:11 2658 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 2659 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2660 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2661 2024-09-21T06:26:50.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2662 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2663 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2664 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2665 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2666 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2667 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2668 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2669 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2670 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:11 2673 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:11 2674 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:11 2675 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2678 2024-09-21T06:26:50.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2679 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2680 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2681 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2682 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2683 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2684 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2685 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2686 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2687 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2688 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2689 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:11 2690 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:11 2691 2024-09-21T06:26:50.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2692 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2693 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2696 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2699 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2701 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2702 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2703 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2704 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2753 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2753_fsm 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2753_vm 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2754 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2755 2024-09-21T06:26:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2756 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2757 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2830 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2831 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2832 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2833 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2834 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2835 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2836 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2837 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:11 2838 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2838_fsm 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2838_vm 2024-09-21T06:26:50.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2839 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840_fsm 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2840_vm 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2841 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2995 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2996 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3079_fsm 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079_vm 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3080 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3081 2024-09-21T06:26:50.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3085 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3118 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3119 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:11 3164 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3256 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3257 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3258 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3350 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3351 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3379 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3380 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3381 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3394 2024-09-21T06:26:50.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3394_fsm 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3394_vm 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3395 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3429 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3430 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3431 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3433 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3439 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3440 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3455 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:11 3456 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3456_fsm 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3456_vm 2024-09-21T06:26:50.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3466 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3467 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3468 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3501 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3502 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3503 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3534 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3541_fsm 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541_vm 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3542 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3574 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3575 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3576 2024-09-21T06:26:50.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3596 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3597 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3598 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3599 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3600_fsm 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600_vm 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3601_fsm 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601_vm 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3602_fsm 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602_vm 2024-09-21T06:26:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603_fsm 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3603_vm 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3604 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3605 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3606 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3607 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3608 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 3609 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3712 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3764_fsm 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764_vm 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3766 2024-09-21T06:26:50.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3767 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3997 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4143 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4144 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4145 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4146 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4147 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4148 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4149 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4150 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4151 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4152 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4153 2024-09-21T06:26:50.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4154 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4155 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4156 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4157 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4158 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4159 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4160 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4161 2024-09-21T06:26:50.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4162 2024-09-21T06:26:50.582 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4163 2024-09-21T06:26:50.582 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4164 2024-09-21T06:26:50.582 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4165 2024-09-21T06:26:50.582 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4166 2024-09-21T06:26:50.582 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4167 2024-09-21T06:26:50.582 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4168 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4169 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4170 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4171 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4172 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4173 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4174 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 5002 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 548 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 549 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6102 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6104 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6106 2024-09-21T06:26:50.583 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6110 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6111 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6112 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6113 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6117 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 826 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 827 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 828 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:total 7777 2024-09-21T06:26:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:11 . 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:11 .. 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 112 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 113 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 1247 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1247_fsm 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1247_vm 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:11 1249 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1249_fsm 2024-09-21T06:26:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1249_vm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:11 1255 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1255_fsm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1255_vm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:11 1259 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1259_fsm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1259_vm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:11 13344 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13344_fsm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13344_vm 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13346 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13348 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349 2024-09-21T06:26:50.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13349_fsm 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349_vm 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13351 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13353 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13354_fsm 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354_vm 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13356 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13358 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13359_fsm 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359_vm 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13361 2024-09-21T06:26:50.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13363 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1417 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1418 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 174 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 175 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2187 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2224 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2328 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2336 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2337 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2579 2024-09-21T06:26:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2600 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2600_fsm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2600_vm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2601_fsm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601_vm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2602 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2602_fsm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2602_vm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2603 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2603_fsm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2603_vm 2024-09-21T06:26:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2604 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2605 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2605_fsm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2605_vm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2606_fsm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606_vm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2607 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2607_fsm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2607_vm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:11 2608 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2608_fsm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2608_vm 2024-09-21T06:26:50.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:11 2609 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2609_fsm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2609_vm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2610 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2610_fsm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2610_vm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2611 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2612_fsm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612_vm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2613 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2615_fsm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615_vm 2024-09-21T06:26:50.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2616 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2616_fsm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2616_vm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:11 2617 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2617_fsm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2617_vm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:11 2618 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2618_fsm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2618_vm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:11 2619 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2619_fsm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2619_vm 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2620 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2650 2024-09-21T06:26:50.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2651 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2652 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2653 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2654 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2655 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2656 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2657 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:11 2658 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 2659 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2660 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2661 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2662 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2663 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2664 2024-09-21T06:26:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2665 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2666 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2667 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2668 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2669 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2670 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:11 2673 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:11 2674 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:11 2675 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2678 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2679 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2680 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2681 2024-09-21T06:26:50.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2682 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2683 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2684 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2685 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2686 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2687 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2688 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2689 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:11 2690 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:11 2691 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2692 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2693 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2696 2024-09-21T06:26:50.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2699 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2701 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2702 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2703 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2704 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2753 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2753_fsm 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2753_vm 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2754 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2755 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2756 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2757 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2830 2024-09-21T06:26:50.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2831 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2832 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2833 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2834 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2835 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2836 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2837 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:11 2838 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2838_fsm 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2838_vm 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2839 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840_fsm 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2840_vm 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2841 2024-09-21T06:26:50.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2995 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2996 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3079_fsm 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079_vm 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3080 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3081 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3085 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3118 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3119 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:11 3164 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3256 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3257 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3258 2024-09-21T06:26:50.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3350 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3351 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3379 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3380 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3381 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3394 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3394_fsm 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3394_vm 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3395 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3429 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3430 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3431 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3433 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3439 2024-09-21T06:26:50.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3440 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3455 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:11 3456 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3456_fsm 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3456_vm 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3466 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3467 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3468 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3501 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3502 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3503 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3534 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3541_fsm 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541_vm 2024-09-21T06:26:50.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3542 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3574 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3575 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3576 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3596 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3597 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3598 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3599 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3600_fsm 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600_vm 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3601_fsm 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601_vm 2024-09-21T06:26:50.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3602_fsm 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602_vm 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603_fsm 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3603_vm 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3604 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3605 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3606 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3607 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3608 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 3609 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3712 2024-09-21T06:26:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3764_fsm 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764_vm 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3766 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3767 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3997 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4143 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4144 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4145 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4146 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4147 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4148 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4149 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4150 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4151 2024-09-21T06:26:50.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4152 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4153 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4154 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4155 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4156 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4157 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4158 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4159 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4160 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4161 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4162 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4163 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4164 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4165 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4166 2024-09-21T06:26:50.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4167 2024-09-21T06:26:50.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4168 2024-09-21T06:26:50.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4169 2024-09-21T06:26:50.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4170 2024-09-21T06:26:50.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4171 2024-09-21T06:26:50.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4172 2024-09-21T06:26:50.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4173 2024-09-21T06:26:50.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4174 2024-09-21T06:26:50.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 5002 2024-09-21T06:26:50.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 548 2024-09-21T06:26:50.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 549 2024-09-21T06:26:50.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6102 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6104 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6106 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6110 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6111 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6112 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6113 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6117 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 826 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 827 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 828 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:26:50.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:total 622230 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 06:14 . 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:11 .. 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 112 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 113 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 1247 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1247_fsm 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1247_vm 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:11 1249 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1249_fsm 2024-09-21T06:26:50.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1249_vm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:11 1255 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1255_fsm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1255_vm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:26 1259 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1259_fsm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1259_vm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:11 13344 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13344_fsm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13344_vm 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13346 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13348 2024-09-21T06:26:50.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13349_fsm 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13349_vm 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13351 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13353 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13354_fsm 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13354_vm 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13356 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13358 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 13359_fsm 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13359_vm 2024-09-21T06:26:50.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 13361 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 13363 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1417 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 1418 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 445743104 Sep 21 06:26 16396 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 21 06:26 16396_fsm 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:13 16396_vm 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 876544 Sep 21 06:26 16397 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:26 16397_fsm 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:26 16397_vm 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2072576 Sep 21 06:26 16399 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:26 16399_fsm 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:26 16399_vm 2024-09-21T06:26:50.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:26 16400 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:26 16402 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 21 06:26 16404 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 108077056 Sep 21 06:26 16406 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:26 16406_fsm 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:23 16406_vm 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 174 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 175 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2187 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2224 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2328 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2336 2024-09-21T06:26:50.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2337 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2579 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2600 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2600_fsm 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2600_vm 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2601_fsm 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2601_vm 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2602 2024-09-21T06:26:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2602_fsm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2602_vm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2603 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2603_fsm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2603_vm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2604 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2605 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2605_fsm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2605_vm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2606_fsm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2606_vm 2024-09-21T06:26:50.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2607 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2607_fsm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2607_vm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:11 2608 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2608_fsm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2608_vm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:11 2609 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2609_fsm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2609_vm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2610 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2610_fsm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2610_vm 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2611 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612 2024-09-21T06:26:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2612_fsm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2612_vm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2613 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2615_fsm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2615_vm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2616 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2616_fsm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2616_vm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:11 2617 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2617_fsm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2617_vm 2024-09-21T06:26:50.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:11 2618 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2618_fsm 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2618_vm 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 21 06:26 2619 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:13 2619_fsm 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2619_vm 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2620 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2650 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2651 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2652 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2653 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2654 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2655 2024-09-21T06:26:50.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2656 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2657 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:11 2658 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:11 2659 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2660 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2661 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2662 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2663 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2664 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2665 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2666 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2667 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2668 2024-09-21T06:26:50.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2669 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2670 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:11 2673 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:11 2674 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:11 2675 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2678 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2679 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2680 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2681 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2682 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2683 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2684 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2685 2024-09-21T06:26:50.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2686 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2687 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2688 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:11 2689 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:11 2690 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:11 2691 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2692 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2693 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:15 2696 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2699 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2701 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2702 2024-09-21T06:26:50.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2703 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 2704 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2753 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2753_fsm 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2753_vm 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2754 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2755 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2756 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 2757 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2830 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2831 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2832 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2833 2024-09-21T06:26:50.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2834 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2835 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2836 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2837 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:11 2838 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2838_fsm 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2838_vm 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2839 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2840_fsm 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2840_vm 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2841 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2995 2024-09-21T06:26:50.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2996 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3079_fsm 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3079_vm 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3080 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3081 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3085 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3118 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3119 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:11 3164 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3256 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3257 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3258 2024-09-21T06:26:50.636 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3350 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3351 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3379 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3380 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3381 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3394 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3394_fsm 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3394_vm 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3395 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3429 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3430 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3431 2024-09-21T06:26:50.637 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3433 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3439 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3440 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:11 3455 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:11 3456 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3456_fsm 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3456_vm 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3466 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3467 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3468 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3501 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3502 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3503 2024-09-21T06:26:50.638 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3534 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3541_fsm 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3541_vm 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3542 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3574 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3575 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3576 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3596 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3597 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3598 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3599 2024-09-21T06:26:50.639 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3600_fsm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3600_vm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3601_fsm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3601_vm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3602_fsm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3602_vm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3603_fsm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3603_vm 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3604 2024-09-21T06:26:50.640 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3605 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3606 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3607 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3608 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:11 3609 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3712 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 3764_fsm 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3764_vm 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3766 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 3767 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3997 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4143 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4144 2024-09-21T06:26:50.641 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4145 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4146 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4147 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4148 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4149 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4150 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4151 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4152 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4153 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4154 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4155 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4156 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4157 2024-09-21T06:26:50.642 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4158 2024-09-21T06:26:50.643 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4159 2024-09-21T06:26:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4160 2024-09-21T06:26:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4161 2024-09-21T06:26:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4162 2024-09-21T06:26:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4163 2024-09-21T06:26:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4164 2024-09-21T06:26:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4165 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4166 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4167 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4168 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4169 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4170 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4171 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4172 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4173 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4174 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 5002 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 548 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 549 2024-09-21T06:26:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6102 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6104 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6106 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6110 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6111 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6112 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6113 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6117 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 826 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 827 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 828 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 21 06:11 pg_internal.init 2024-09-21T06:26:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:11 PG_VERSION 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:11 .. 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:total 560 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 06:11 . 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1213 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1213_fsm 2024-09-21T06:26:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1213_vm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1214 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1214_fsm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1214_vm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 1232 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 1233 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1260 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1260_fsm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1260_vm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1261 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1261_fsm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1261_vm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1262 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 1262_fsm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 1262_vm 2024-09-21T06:26:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2396 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:11 2396_fsm 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2396_vm 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2397 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2671 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2672 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2676 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2677 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2694 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2695 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2697 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:11 2698 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2846 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2847 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2964 2024-09-21T06:26:50.649 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2965 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 2966 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 2967 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 3592 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 3593 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4060 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4061 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4175 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4176 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4177 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4178 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4181 2024-09-21T06:26:50.650 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4182 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4183 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4184 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 4185 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 4186 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6000 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6001 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6002 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 6100 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6114 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 6115 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:26 pg_control 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:11 pg_filenode.map 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 21 06:11 pg_internal.init 2024-09-21T06:26:50.651 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:total 2 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1223 Sep 21 06:26 postgresql-Sat.log 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T06:26:50.652 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:total 1 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:26 . 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 mappings 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 06:26 replorigin_checkpoint 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 snapshots 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T06:26:50.653 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:26 .. 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:26 .. 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 . 2024-09-21T06:26:50.654 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 members 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 offsets 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:total 8 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 .. 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:11 0000 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:total 8 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:11 . 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:11 .. 2024-09-21T06:26:50.655 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:13 0000 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T06:26:50.656 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:total 15 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:26 . 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 21 06:26 db_0.stat 2024-09-21T06:26:50.657 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 21 06:26 db_13494.stat 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 21 06:26 global.stat 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:26 . 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:total 96 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:26 . 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:26 001F 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T06:26:50.659 INFO:teuthology.orchestra.run.smithi031.stdout:total 1048576 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 06:26 . 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000B8 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:19 0000000100000000000000B9 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:18 0000000100000000000000BA 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:19 0000000100000000000000BB 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:19 0000000100000000000000BC 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:18 0000000100000000000000BD 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:16 0000000100000000000000BE 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000BF 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000C0 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000C1 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:25 0000000100000000000000C2 2024-09-21T06:26:50.660 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:23 0000000100000000000000C3 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:20 0000000100000000000000C4 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000C5 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:23 0000000100000000000000C6 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000C7 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:20 0000000100000000000000C8 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000C9 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000CA 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:25 0000000100000000000000CB 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000CC 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000CD 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000CE 2024-09-21T06:26:50.661 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:23 0000000100000000000000CF 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:24 0000000100000000000000D0 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:25 0000000100000000000000D1 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000D2 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000D3 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000D4 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000D5 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000D6 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000D7 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:20 0000000100000000000000D8 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:24 0000000100000000000000D9 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000DA 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000DB 2024-09-21T06:26:50.662 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000DC 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000DD 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000DE 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000DF 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000E0 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:22 0000000100000000000000E1 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:24 0000000100000000000000E2 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000E3 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000E4 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000E5 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:21 0000000100000000000000E6 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000E7 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000E8 2024-09-21T06:26:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000E9 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000EA 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000EB 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000EC 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000ED 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000EE 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000EF 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000F0 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000F1 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000F2 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:25 0000000100000000000000F3 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000F4 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000F5 2024-09-21T06:26:50.664 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:26 0000000100000000000000F6 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:25 0000000100000000000000F7 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 archive_status 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:total 0 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:11 . 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Sep 21 06:26 .. 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:total 504 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:16 . 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 06:26 .. 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Sep 21 06:16 0000 2024-09-21T06:26:50.665 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:26 0001 2024-09-21T06:26:50.666 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T06:26:50.676 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:26:50.684 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T06:26:50.708 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T06:26:50.708 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:26:50.708 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:26:50.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T06:26:50.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T06:26:50.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.970+0000 7f91300f2640 1 -- 172.21.15.31:0/685815199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 msgr2=0x7f9128110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.970+0000 7f91300f2640 1 --2- 172.21.15.31:0/685815199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f9128110580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9120009ca0 tx=0x7f912002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 -- 172.21.15.31:0/685815199 shutdown_connections 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 --2- 172.21.15.31:0/685815199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f9128110580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 --2- 172.21.15.31:0/685815199 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9128109400 0x7f9128109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 --2- 172.21.15.31:0/685815199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9128103ad0 0x7f9128103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 -- 172.21.15.31:0/685815199 >> 172.21.15.31:0/685815199 conn(0x7f91280ff510 msgr2=0x7f9128101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 -- 172.21.15.31:0/685815199 shutdown_connections 2024-09-21T06:26:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 -- 172.21.15.31:0/685815199 wait complete. 2024-09-21T06:26:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 Processor -- start 2024-09-21T06:26:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.971+0000 7f91300f2640 1 -- start start 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f91300f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9128103ad0 0x7f912819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f91300f2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9128109400 0x7f912819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f91300f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f912819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f91300f2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9128113a60 con 0x7f9128109d50 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f91300f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91281138e0 con 0x7f9128103ad0 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f91300f2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9128113be0 con 0x7f9128109400 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f912819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f912819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34020/0 (socket says 172.21.15.31:34020) 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 -- 172.21.15.31:0/3925968023 learned_addr learned my addr 172.21.15.31:0/3925968023 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912de8e640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9128103ad0 0x7f912819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:50.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912d68d640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9128109400 0x7f912819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 -- 172.21.15.31:0/3925968023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9128109400 msgr2=0x7f912819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9128109400 0x7f912819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 -- 172.21.15.31:0/3925968023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9128103ad0 msgr2=0x7f912819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9128103ad0 0x7f912819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 -- 172.21.15.31:0/3925968023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91281a0330 con 0x7f9128109d50 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f912e68f640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f912819fc30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9120009430 tx=0x7f91200319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.972+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120003c30 con 0x7f9128109d50 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.973+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9120003dd0 con 0x7f9128109d50 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.973+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91281a05c0 con 0x7f9128109d50 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.973+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912002faa0 con 0x7f9128109d50 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.973+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91281a6db0 con 0x7f9128109d50 2024-09-21T06:26:50.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.973+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9128103ee0 con 0x7f9128109d50 2024-09-21T06:26:50.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.975+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f912002fc40 con 0x7f9128109d50 2024-09-21T06:26:50.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.976+0000 7f911effd640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91140b6680 0x7f91140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:50.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.977+0000 7f912de8e640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91140b6680 0x7f91140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:50.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.977+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f9120108bb0 con 0x7f9128109d50 2024-09-21T06:26:50.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.977+0000 7f912de8e640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91140b6680 0x7f91140b8b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9124005d00 tx=0x7f912400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:50.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:50.980+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91200cf110 con 0x7f9128109d50 2024-09-21T06:26:51.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.192+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9128063800 con 0x7f91140b6680 2024-09-21T06:26:51.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.200+0000 7f911effd640 1 -- 172.21.15.31:0/3925968023 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9128063800 con 0x7f91140b6680 2024-09-21T06:26:51.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T06:11:21", "first": "2024-09-21T06:12:00", "last": "2024-09-21T06:26:00", "last_pruned": "2024-09-21T06:26:00", "created_count": 15, "pruned_count": 9, "active": true} 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91140b6680 msgr2=0x7f91140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91140b6680 0x7f91140b8b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9124005d00 tx=0x7f912400b410 comp rx=0 tx=0).stop 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 msgr2=0x7f912819fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f912819fc30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9120009430 tx=0x7f91200319f0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 shutdown_connections 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f91140b6680 0x7f91140b8b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9128109d50 0x7f912819fc30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9128109400 0x7f912819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 --2- 172.21.15.31:0/3925968023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9128103ad0 0x7f912819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 >> 172.21.15.31:0/3925968023 conn(0x7f91280ff510 msgr2=0x7f9128108a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 shutdown_connections 2024-09-21T06:26:51.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.203+0000 7f91300f2640 1 -- 172.21.15.31:0/3925968023 wait complete. 2024-09-21T06:26:51.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T06:26:51.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 -- 172.21.15.31:0/1236287136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb638105280 msgr2=0x7fb63810d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/1236287136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb638105280 0x7fb63810d7c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb628009900 tx=0x7fb62802a410 comp rx=0 tx=0).stop 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 -- 172.21.15.31:0/1236287136 shutdown_connections 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/1236287136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63810dd90 0x7fb638110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/1236287136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb638105280 0x7fb63810d7c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/1236287136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638104930 0x7fb638104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 -- 172.21.15.31:0/1236287136 >> 172.21.15.31:0/1236287136 conn(0x7fb6380fe3e0 msgr2=0x7fb638100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 -- 172.21.15.31:0/1236287136 shutdown_connections 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.331+0000 7fb63d2cc640 1 -- 172.21.15.31:0/1236287136 wait complete. 2024-09-21T06:26:51.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 Processor -- start 2024-09-21T06:26:51.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 -- start start 2024-09-21T06:26:51.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb638104930 0x7fb63819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638105280 0x7fb63819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 0x7fb63819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381139f0 con 0x7fb63810dd90 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb638113870 con 0x7fb638104930 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb63d2cc640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb638113b70 con 0x7fb638105280 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb6377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 0x7fb63819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb6377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 0x7fb63819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34038/0 (socket says 172.21.15.31:34038) 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.332+0000 7fb6377fe640 1 -- 172.21.15.31:0/2601986150 learned_addr learned my addr 172.21.15.31:0/2601986150 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb636ffd640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb638104930 0x7fb63819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6367fc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638105280 0x7fb63819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6377fe640 1 -- 172.21.15.31:0/2601986150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638105280 msgr2=0x7fb63819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6377fe640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638105280 0x7fb63819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6377fe640 1 -- 172.21.15.31:0/2601986150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb638104930 msgr2=0x7fb63819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6377fe640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb638104930 0x7fb63819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6377fe640 1 -- 172.21.15.31:0/2601986150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6381a03a0 con 0x7fb63810dd90 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb636ffd640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb638104930 0x7fb63819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6367fc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638105280 0x7fb63819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb6377fe640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 0x7fb63819fca0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb630009fd0 tx=0x7fb63000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630017070 con 0x7fb63810dd90 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6381a0690 con 0x7fb63810dd90 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb630003e20 con 0x7fb63810dd90 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630010460 con 0x7fb63810dd90 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.333+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6381a6e20 con 0x7fb63810dd90 2024-09-21T06:26:51.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.334+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb638104d40 con 0x7fb63810dd90 2024-09-21T06:26:51.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.336+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb630003970 con 0x7fb63810dd90 2024-09-21T06:26:51.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.337+0000 7fb62ffff640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6240b6680 0x7fb6240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:26:51.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.337+0000 7fb636ffd640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6240b6680 0x7fb6240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:26:51.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.337+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7fb6300e7950 con 0x7fb63810dd90 2024-09-21T06:26:51.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.337+0000 7fb636ffd640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6240b6680 0x7fb6240b8b90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb620009c20 tx=0x7fb620005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:26:51.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.340+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6300adeb0 con 0x7fb63810dd90 2024-09-21T06:26:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.551+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 --> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb638106ec0 con 0x7fb6240b6680 2024-09-21T06:26:51.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.559+0000 7fb62ffff640 1 -- 172.21.15.31:0/2601986150 <== mgr.14226 v2:172.21.15.31:6800/2760018577 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb638106ec0 con 0x7fb6240b6680 2024-09-21T06:26:51.560 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1m 6m3h 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6240b6680 msgr2=0x7fb6240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6240b6680 0x7fb6240b8b90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb620009c20 tx=0x7fb620005f90 comp rx=0 tx=0).stop 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 msgr2=0x7fb63819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 0x7fb63819fca0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb630009fd0 tx=0x7fb63000b260 comp rx=0 tx=0).stop 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 shutdown_connections 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb6240b6680 0x7fb6240b8b90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb63810dd90 0x7fb63819fca0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb638105280 0x7fb63819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 --2- 172.21.15.31:0/2601986150 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb638104930 0x7fb63819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 >> 172.21.15.31:0/2601986150 conn(0x7fb6380fe3e0 msgr2=0x7fb6381067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 shutdown_connections 2024-09-21T06:26:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:26:51.561+0000 7fb63d2cc640 1 -- 172.21.15.31:0/2601986150 wait complete. 2024-09-21T06:26:51.574 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T06:26:51.591 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T06:26:51.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:51 smithi031 ceph-mon[31237]: pgmap v796: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-21T06:26:52.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:51 smithi110 ceph-mon[34550]: pgmap v796: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-21T06:26:52.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:51 smithi107 ceph-mon[34380]: pgmap v796: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-09-21T06:26:52.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:52 smithi031 ceph-mon[31237]: from='client.18456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:52.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:52 smithi031 ceph-mon[31237]: from='client.18462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:53.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:52 smithi110 ceph-mon[34550]: from='client.18456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:53.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:52 smithi110 ceph-mon[34550]: from='client.18462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:53.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:52 smithi107 ceph-mon[34380]: from='client.18456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:53.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:52 smithi107 ceph-mon[34380]: from='client.18462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:26:53.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:53 smithi031 ceph-mon[31237]: pgmap v797: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-09-21T06:26:54.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:53 smithi110 ceph-mon[34550]: pgmap v797: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-09-21T06:26:54.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:53 smithi107 ceph-mon[34380]: pgmap v797: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 16 MiB/s wr, 1.50k op/s 2024-09-21T06:26:55.886 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:55 smithi031 ceph-mon[31237]: pgmap v798: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 9.7 MiB/s wr, 931 op/s 2024-09-21T06:26:56.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:55 smithi110 ceph-mon[34550]: pgmap v798: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 9.7 MiB/s wr, 931 op/s 2024-09-21T06:26:56.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:55 smithi107 ceph-mon[34380]: pgmap v798: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 9.7 MiB/s wr, 931 op/s 2024-09-21T06:26:58.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:57 smithi110 ceph-mon[34550]: pgmap v799: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 8.6 MiB/s wr, 844 op/s 2024-09-21T06:26:58.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:57 smithi107 ceph-mon[34380]: pgmap v799: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 8.6 MiB/s wr, 844 op/s 2024-09-21T06:26:58.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:57 smithi031 ceph-mon[31237]: pgmap v799: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 8.6 MiB/s wr, 844 op/s 2024-09-21T06:26:59.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:59.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:59.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:26:59.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:58 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:26:59.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:59.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:59.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:26:59.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:58 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:26:59.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:26:59.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:26:59.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:26:59.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:58 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' 2024-09-21T06:27:00.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:26:59 smithi110 ceph-mon[34550]: pgmap v800: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 5.8 MiB/s wr, 624 op/s 2024-09-21T06:27:00.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:26:59 smithi107 ceph-mon[34380]: pgmap v800: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 5.8 MiB/s wr, 624 op/s 2024-09-21T06:27:00.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:26:59 smithi031 ceph-mon[31237]: pgmap v800: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 5.8 MiB/s wr, 624 op/s 2024-09-21T06:27:02.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:01 smithi110 ceph-mon[34550]: pgmap v801: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-21T06:27:02.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:01 smithi107 ceph-mon[34380]: pgmap v801: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-21T06:27:02.137 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:01 smithi031 ceph-mon[31237]: pgmap v801: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-21T06:27:04.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:03 smithi110 ceph-mon[34550]: pgmap v802: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-09-21T06:27:04.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:03 smithi107 ceph-mon[34380]: pgmap v802: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-09-21T06:27:04.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:03 smithi031 ceph-mon[31237]: pgmap v802: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-09-21T06:27:06.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:27:06.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:05 smithi110 ceph-mon[34550]: pgmap v803: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 228 KiB/s wr, 28 op/s 2024-09-21T06:27:06.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:05 smithi107 ceph-mon[34380]: pgmap v803: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 228 KiB/s wr, 28 op/s 2024-09-21T06:27:06.043 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:05 smithi031 ceph-mon[31237]: pgmap v803: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 228 KiB/s wr, 28 op/s 2024-09-21T06:27:06.289 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 -- 172.21.15.31:0/125969613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54108d00 msgr2=0x7f5c5410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/125969613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54108d00 0x7f5c5410f530 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009ca0 tx=0x7f5c4c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 -- 172.21.15.31:0/125969613 shutdown_connections 2024-09-21T06:27:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/125969613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54108d00 0x7f5c5410f530 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/125969613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c541083b0 0x7f5c541087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/125969613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54102a70 0x7f5c54102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 -- 172.21.15.31:0/125969613 >> 172.21.15.31:0/125969613 conn(0x7f5c54079730 msgr2=0x7f5c54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 -- 172.21.15.31:0/125969613 shutdown_connections 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.761+0000 7f5c58aa9640 1 -- 172.21.15.31:0/125969613 wait complete. 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 Processor -- start 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 -- start start 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 0x7f5c54199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c541083b0 0x7f5c541999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54108d00 0x7f5c5419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5c54112970 con 0x7f5c54102a70 2024-09-21T06:27:06.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541127f0 con 0x7f5c54108d00 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c58aa9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5c54112af0 con 0x7f5c541083b0 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 0x7f5c54199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 0x7f5c54199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40416/0 (socket says 172.21.15.31:40416) 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c5259c640 1 -- 172.21.15.31:0/2662068052 learned_addr learned my addr 172.21.15.31:0/2662068052 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.762+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c541083b0 0x7f5c541999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54108d00 0x7f5c5419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c5259c640 1 -- 172.21.15.31:0/2662068052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c541083b0 msgr2=0x7f5c541999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c5259c640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c541083b0 0x7f5c541999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c5259c640 1 -- 172.21.15.31:0/2662068052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54108d00 msgr2=0x7f5c5419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c5259c640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54108d00 0x7f5c5419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c5259c640 1 -- 172.21.15.31:0/2662068052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5419f3b0 con 0x7f5c54102a70 2024-09-21T06:27:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54108d00 0x7f5c5419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c5259c640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 0x7f5c54199440 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9f0 tx=0x7f5c4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c437fe640 1 -- 172.21.15.31:0/2662068052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400cd80 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c437fe640 1 -- 172.21.15.31:0/2662068052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c440108e0 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5419f6a0 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c437fe640 1 -- 172.21.15.31:0/2662068052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44018980 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.763+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5406bae0 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.766+0000 7f5c437fe640 1 -- 172.21.15.31:0/2662068052 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5c44010430 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.767+0000 7f5c5f01d740 1 Processor -- start 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.767+0000 7f5c437fe640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5c3c0b6680 0x7f5c3c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.767+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5c3c0b6680 0x7f5c3c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.767+0000 7f5c437fe640 1 -- 172.21.15.31:0/2662068052 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f5c440e7ae0 con 0x7f5c54102a70 2024-09-21T06:27:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.767+0000 7f5c5f01d740 1 -- start start 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8200d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5c3c0b6680 0x7f5c3c0b8b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c006010 tx=0x7f5c4c0093f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f8200dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557f81f83e50 con 0x557f81c45390 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557f81f73280 con 0x557f81c66070 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557f81f77090 con 0x557f81c3d1b0 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8200d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c52d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f8200dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8200d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:42506/0 (socket says 172.21.15.31:42506) 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 -- 172.21.15.31:0/249185568 learned_addr learned my addr 172.21.15.31:0/249185568 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 -- 172.21.15.31:0/249185568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 msgr2=0x557f8200d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 -- 172.21.15.31:0/249185568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 msgr2=0x557f8200dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f8200dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.768+0000 7f5c5259c640 1 -- 172.21.15.31:0/249185568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f81cd1650 con 0x557f81c66070 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.769+0000 7f5c5259c640 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8200d270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5c440a9f10 tx=0x7f5c44010720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=303adad08efca68a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.769+0000 7f5c40ff9640 1 -- 172.21.15.31:0/249185568 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440ae040 con 0x557f81c66070 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.769+0000 7f5c40ff9640 1 -- 172.21.15.31:0/249185568 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c440b2420 con 0x557f81c66070 2024-09-21T06:27:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.769+0000 7f5c40ff9640 1 -- 172.21.15.31:0/249185568 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440a94f0 con 0x557f81c66070 2024-09-21T06:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 -- 172.21.15.31:0/249185568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 msgr2=0x557f8200d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8200d270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5c440a9f10 tx=0x7f5c44010720 comp rx=0 tx=0).stop 2024-09-21T06:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 -- 172.21.15.31:0/249185568 shutdown_connections 2024-09-21T06:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f8200dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/249185568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8200d270 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 -- 172.21.15.31:0/249185568 >> 172.21.15.31:0/249185568 conn(0x557f81c5a460 msgr2=0x557f8200b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 -- 172.21.15.31:0/249185568 shutdown_connections 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 -- 172.21.15.31:0/249185568 wait complete. 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.770+0000 7f5c5f01d740 1 -- start start 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 Processor -- start 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c3d1b0 0x557f820189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f82018f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c66070 0x557f82024ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x557f81dd4700 con 0x557f81c66070 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557f81f0e110 con 0x557f81c3d1b0 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557f8201b9f0 con 0x557f81c45390 2024-09-21T06:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5f01d740 10 client.0 _ll_register_callbacks cb 0x557f81f27990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c52d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c66070 0x557f82024ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c3d1b0 0x557f820189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c52d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c66070 0x557f82024ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40426/0 (socket says 172.21.15.31:40426) 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c52d9d640 1 -- 172.21.15.31:0/1720496749 learned_addr learned my addr 172.21.15.31:0/1720496749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.771+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f82018f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5259c640 1 -- 172.21.15.31:0/1720496749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 msgr2=0x557f82018f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5259c640 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f82018f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5259c640 1 -- 172.21.15.31:0/1720496749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c66070 msgr2=0x557f82024ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5259c640 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c66070 0x557f82024ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5259c640 1 -- 172.21.15.31:0/1720496749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f81f3bc40 con 0x557f81c3d1b0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5259c640 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c3d1b0 0x557f820189c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5c440e57c0 tx=0x7f5c44018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c9cfb489a3216a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-09-21T06:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c39ffb640 1 -- 172.21.15.31:0/1720496749 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440b2040 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c39ffb640 1 -- 172.21.15.31:0/1720496749 <== mon.2 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f5c440a94a0 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c440b2420 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c39ffb640 1 -- 172.21.15.31:0/1720496749 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c440b7360 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5f01d740 10 client.28274 fetch_fsmap learned FSMap version 21 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.772+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557f820277a0 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c39ffb640 1 -- 172.21.15.31:0/1720496749 <== mon.2 v2:172.21.15.107:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c440b2420 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c39ffb640 1 -- 172.21.15.31:0/1720496749 <== mon.2 v2:172.21.15.107:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5c440ac610 con 0x557f81c3d1b0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c5f01d740 10 client.28274 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c5f01d740 10 client.28274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c5f01d740 20 client.28274 populate_metadata read hostname 'smithi031' 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81c249f0 0x557f8202acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c5f01d740 4 client.28274 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f81f85da0 con 0x557f81c249f0 2024-09-21T06:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.773+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81c249f0 0x557f8202acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c39ffb640 10 client.28274 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81c249f0 0x557f8202acf0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676b6230d84ae5d8 server_cookie=86d6d6c9ea05ca3f in_seq=0 out_seq=0 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c39ffb640 1 -- 172.21.15.31:0/1720496749 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f81f85da0 con 0x557f81c249f0 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c39ffb640 10 client.28274 handle_command_reply: tid=0 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c5f01d740 1 client.28274 shutdown 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c3d1b0 msgr2=0x557f820189c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.774+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c3d1b0 0x557f820189c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5c440e57c0 tx=0x7f5c44018420 comp rx=0 tx=0).stop 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 shutdown_connections 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81c249f0 0x557f8202acf0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c66070 0x557f82024ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f82018f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c39ffb640 0 client.28274 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/1720496749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c3d1b0 0x557f820189c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 >> 172.21.15.31:0/1720496749 conn(0x557f81c5a460 msgr2=0x557f82010450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 shutdown_connections 2024-09-21T06:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 1 -- 172.21.15.31:0/1720496749 wait complete. 2024-09-21T06:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.775+0000 7f5c5f01d740 20 client.28274 trim_cache size 0 max 16384 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 Processor -- start 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 -- start start 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f82024be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f82010370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ad830 con 0x557f81c45390 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541b0090 con 0x557f81c66070 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.792+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ade80 con 0x557f81c3d1b0 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f82024be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f82024be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40436/0 (socket says 172.21.15.31:40436) 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 -- 172.21.15.31:0/63506204 learned_addr learned my addr 172.21.15.31:0/63506204 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c5259c640 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f82010370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 -- 172.21.15.31:0/63506204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 msgr2=0x557f8200b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 -- 172.21.15.31:0/63506204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 msgr2=0x557f82010370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f82010370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 -- 172.21.15.31:0/63506204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f8201b6d0 con 0x557f81c45390 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f82010370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f82024be0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x557f8202b420 tx=0x7f5c4800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755b2d1d1472a201 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c3bfff640 1 -- 172.21.15.31:0/63506204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48003d60 con 0x557f81c45390 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c3bfff640 1 -- 172.21.15.31:0/63506204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c480118f0 con 0x557f81c45390 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.793+0000 7f5c3bfff640 1 -- 172.21.15.31:0/63506204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c480247c0 con 0x557f81c45390 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 -- 172.21.15.31:0/63506204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 msgr2=0x557f82024be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f82024be0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x557f8202b420 tx=0x7f5c4800b390 comp rx=0 tx=0).stop 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 -- 172.21.15.31:0/63506204 shutdown_connections 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f82010370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c45390 0x557f82024be0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/63506204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c3d1b0 0x557f8200b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 -- 172.21.15.31:0/63506204 >> 172.21.15.31:0/63506204 conn(0x557f81c5a460 msgr2=0x557f8200aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.794+0000 7f5c5f01d740 1 -- 172.21.15.31:0/63506204 shutdown_connections 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.795+0000 7f5c5f01d740 1 -- 172.21.15.31:0/63506204 wait complete. 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.795+0000 7f5c5f01d740 1 -- start start 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.795+0000 7f5c5f01d740 1 Processor -- start 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 0x557f82021350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f8201c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8201cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ae550 con 0x557f81c3d1b0 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541af9c0 con 0x557f81c66070 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541aec20 con 0x557f81c45390 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5f01d740 10 client.0 _ll_register_callbacks cb 0x7f5c44005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 0x557f82021350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 0x557f82021350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40438/0 (socket says 172.21.15.31:40438) 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 -- 172.21.15.31:0/4202146399 learned_addr learned my addr 172.21.15.31:0/4202146399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f8201c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c52d9d640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8201cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 -- 172.21.15.31:0/4202146399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 msgr2=0x557f8201c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f8201c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 -- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 msgr2=0x557f8201cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8201cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 -- 172.21.15.31:0/4202146399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f8201d430 con 0x557f81c3d1b0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c5259c640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 0x557f82021350 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c6a0 tx=0x7f5c440a9de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e313bdbabe597d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.796+0000 7f5c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c39ffb640 1 -- 172.21.15.31:0/4202146399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440b2040 con 0x557f81c3d1b0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c39ffb640 1 -- 172.21.15.31:0/4202146399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8499+0+0 (secure 0 0 0) 0x7f5c440c3af0 con 0x557f81c3d1b0 2024-09-21T06:27:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c480118f0 con 0x557f81c3d1b0 2024-09-21T06:27:06.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c39ffb640 1 -- 172.21.15.31:0/4202146399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c440a9f90 con 0x557f81c3d1b0 2024-09-21T06:27:06.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 10 client.18480 fetch_fsmap learned FSMap version 21 2024-09-21T06:27:06.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557f820954f0 con 0x557f81c3d1b0 2024-09-21T06:27:06.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c39ffb640 1 -- 172.21.15.31:0/4202146399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c480118f0 con 0x557f81c3d1b0 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c39ffb640 1 -- 172.21.15.31:0/4202146399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5c440be050 con 0x557f81c3d1b0 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 10 client.18480 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 10 client.18480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 20 client.18480 populate_metadata read hostname 'smithi031' 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81bf9da0 0x557f82097a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 4 client.18480 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.797+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f81f81850 con 0x557f81bf9da0 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.798+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81bf9da0 0x557f82097a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.798+0000 7f5c39ffb640 10 client.18480 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:06.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.798+0000 7f5c51d9b640 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81bf9da0 0x557f82097a90 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e174e56bfd39b14 server_cookie=673744859e0bd135 in_seq=0 out_seq=0 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.798+0000 7f5c39ffb640 1 -- 172.21.15.31:0/4202146399 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9746 (crc 0 0 0) 0x557f81f81850 con 0x557f81bf9da0 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.798+0000 7f5c39ffb640 10 client.18480 handle_command_reply: tid=0 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 client.18480 shutdown 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 msgr2=0x557f82021350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 0x557f82021350 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c6a0 tx=0x7f5c440a9de0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 shutdown_connections 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x557f81bf9da0 0x557f82097a90 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f81c66070 0x557f8201cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557f81c45390 0x557f8201c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 --2- 172.21.15.31:0/4202146399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f81c3d1b0 0x557f82021350 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 >> 172.21.15.31:0/4202146399 conn(0x557f81c5a460 msgr2=0x557f82010370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c39ffb640 0 client.18480 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 shutdown_connections 2024-09-21T06:27:06.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 1 -- 172.21.15.31:0/4202146399 wait complete. 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.799+0000 7f5c5f01d740 20 client.18480 trim_cache size 0 max 16384 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {}, 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_stats": { 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stdout: "97dcd776-5fd7-4ec8-ada7-f276bb218390": { 2024-09-21T06:27:06.806 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:51.458278+0000", 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "ccf4fa15-85d4-4e94-aab2-fbf0d30d0526": { 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:25:10.069920+0000", 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.807 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "03db4bb9-9852-48d3-b4a7-a2cb3d6a61e2": { 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:29.790547+0000", 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "57de4f22-440b-403d-a8c2-c8f5c072e14d": { 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:09.437530+0000", 2024-09-21T06:27:06.808 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "88fd931c-cb53-4956-bb22-4366d123d517": { 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:08.856553+0000", 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.809 INFO:teuthology.orchestra.run.smithi031.stdout: "8716d24c-4ac0-4c19-bb0a-5288b818dbd0": { 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:00.475683+0000", 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "fe52f945-9328-4bb2-a75c-107af33e2152": { 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:26:33.002635+0000", 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "1b0e6866-bf74-4daf-b0bc-b4a3ee098d46": { 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:38.352072+0000", 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "16b72551-9797-4fb8-a10e-11da021b88cf": { 2024-09-21T06:27:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:10.235520+0000", 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "8107147d-48c1-4246-823e-f82266cad4b4": { 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:20.250362+0000", 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.812 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "85354dac-139f-426b-98db-539d23ab1572": { 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:50.524865+0000", 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.813 INFO:teuthology.orchestra.run.smithi031.stdout: "cdc477b6-9b80-4f72-8219-838502bc271a": { 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:21.946350+0000", 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "50aa1144-1baf-4a8b-ba41-6ff8eb2fffaa": { 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:50.419858+0000", 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.814 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "06bdcfb5-152c-425c-b4aa-a2e742f70d78": { 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:24:20.413693+0000", 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "e621855d-93b4-415c-97d3-d2c5305a6bfc": { 2024-09-21T06:27:06.815 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:50.889842+0000", 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "500d07c3-510a-4870-b876-5eb317d6c7ec": { 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:00.464701+0000", 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "b82343df-281a-40b4-a471-6077c3b4ef6f": { 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:13:19.293368+0000", 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "f09db644-bf80-47e4-97a1-7c8d6f95b83b": { 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:11:30.376207+0000", 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "4ad8636e-3cfe-4a90-96d1-05b9b1c9106e": { 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:12:29.363208+0000", 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "ababb4c4-14ff-4c13-9f92-e2c95075bbb2": { 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:11.506559+0000", 2024-09-21T06:27:06.818 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "3fc27b03-07a0-480b-8d21-34b6a99891c5": { 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:08.977561+0000", 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "c43e959f-3ba1-4285-ae7f-d6acc8bd2145": { 2024-09-21T06:27:06.819 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:17:49.803888+0000", 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "54b7927b-7db3-4bc0-bddf-a85c18744a1c": { 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:18:31.411209+0000", 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.820 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "e2bbfb80-25a6-433d-9181-39df795d15f2": { 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:16:00.302694+0000", 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "7168e319-4a53-46ce-aaf6-68516fe27218": { 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:34.808232+0000", 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "8d24bb6c-bdb0-46a8-8b56-d1650bd8f571": { 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:49.080125+0000", 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.822 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "0176f173-9d5c-4b70-866d-7257c518cda6": { 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:19:52.981894+0000", 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "e31d9f72-c104-4147-9ac7-1a5c9c0d8846": { 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:20:44.272082+0000", 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "71aa8c73-a180-4b01-a247-118653b8cea4": { 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:21:26.162425+0000", 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.824 INFO:teuthology.orchestra.run.smithi031.stdout: "8d8a0e01-be41-453f-b15c-3b58c614b2da": { 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:22:07.820775+0000", 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "25079243-b4e8-4dfe-98d8-245107f7d0c4": { 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:30.535473+0000", 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.825 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "34367b1c-c534-440d-962a-b6cbdeb827e6": { 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:14:01.127669+0000", 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "266bf898-ce45-49ba-ac95-5434c34c696c": { 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:15:08.647548+0000", 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.826 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "541b6c55-3b9d-40c2-9663-35900f4b8f08": { 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "start_time": "2024-09-21T06:23:39.935326+0000", 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_started": 0, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_passed": 0, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_failed": 0, 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: "uninline_skipped": 0 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:06.827 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5c3c0b6680 msgr2=0x7f5c3c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5c3c0b6680 0x7f5c3c0b8b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c006010 tx=0x7f5c4c0093f0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 msgr2=0x7f5c54199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 0x7f5c54199440 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400e9f0 tx=0x7f5c4400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 shutdown_connections 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c54108d00 0x7f5c5419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c541083b0 0x7f5c541999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5c3c0b6680 0x7f5c3c0b8b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 --2- 172.21.15.31:0/2662068052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c54102a70 0x7f5c54199440 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 >> 172.21.15.31:0/2662068052 conn(0x7f5c54079730 msgr2=0x7f5c541044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:06.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 shutdown_connections 2024-09-21T06:27:06.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:06.800+0000 7f5c58aa9640 1 -- 172.21.15.31:0/2662068052 wait complete. 2024-09-21T06:27:07.855 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:27:07.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:27:08.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:07 smithi110 ceph-mon[34550]: pgmap v804: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 271 KiB/s wr, 43 op/s 2024-09-21T06:27:08.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:07 smithi107 ceph-mon[34380]: pgmap v804: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 271 KiB/s wr, 43 op/s 2024-09-21T06:27:08.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:07 smithi031 ceph-mon[31237]: pgmap v804: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 271 KiB/s wr, 43 op/s 2024-09-21T06:27:08.165 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 -- 172.21.15.31:0/129397257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10108cf0 msgr2=0x7f8e1010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 --2- 172.21.15.31:0/129397257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10108cf0 0x7f8e1010f520 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009ca0 tx=0x7f8e0802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 -- 172.21.15.31:0/129397257 shutdown_connections 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 --2- 172.21.15.31:0/129397257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10108cf0 0x7f8e1010f520 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 --2- 172.21.15.31:0/129397257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e101083a0 0x7f8e101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 --2- 172.21.15.31:0/129397257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e10102a60 0x7f8e10102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 -- 172.21.15.31:0/129397257 >> 172.21.15.31:0/129397257 conn(0x7f8e10079730 msgr2=0x7f8e10079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 -- 172.21.15.31:0/129397257 shutdown_connections 2024-09-21T06:27:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 -- 172.21.15.31:0/129397257 wait complete. 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.665+0000 7f8e15346640 1 Processor -- start 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 -- start start 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 0x7f8e1019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e101083a0 0x7f8e1019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e10108cf0 0x7f8e1019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112960 con 0x7f8e10102a60 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101127e0 con 0x7f8e10108cf0 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e15346640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112ae0 con 0x7f8e101083a0 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 0x7f8e1019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 0x7f8e1019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40456/0 (socket says 172.21.15.31:40456) 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 -- 172.21.15.31:0/2670974307 learned_addr learned my addr 172.21.15.31:0/2670974307 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e07fff640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e101083a0 0x7f8e1019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0f7fe640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e10108cf0 0x7f8e1019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 -- 172.21.15.31:0/2670974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e101083a0 msgr2=0x7f8e1019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e101083a0 0x7f8e1019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 -- 172.21.15.31:0/2670974307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e10108cf0 msgr2=0x7f8e1019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e10108cf0 0x7f8e1019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.666+0000 7f8e0effd640 1 -- 172.21.15.31:0/2670974307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e101a6560 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.667+0000 7f8e0effd640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 0x7f8e1019b2d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000d950 tx=0x7f8e0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.667+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00017070 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.667+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e00009c30 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.667+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e101a6850 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.667+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0000f620 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.667+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101a6de0 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.668+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e10102e70 con 0x7f8e10102a60 2024-09-21T06:27:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.670+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8e00005030 con 0x7f8e10102a60 2024-09-21T06:27:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.670+0000 7f8e0cff9640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8dfc0b6730 0x7f8dfc0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.671+0000 7f8e07fff640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8dfc0b6730 0x7f8dfc0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.671+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7f8e000e72e0 con 0x7f8e10102a60 2024-09-21T06:27:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.671+0000 7f8e07fff640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8dfc0b6730 0x7f8dfc0b8c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8e1006a3f0 tx=0x7f8df800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.674+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e000ad840 con 0x7f8e10102a60 2024-09-21T06:27:08.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:08 smithi031 ceph-mon[31237]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T06:27:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.915+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e1006b270 con 0x7f8e10102a60 2024-09-21T06:27:08.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.916+0000 7f8e0cff9640 1 -- 172.21.15.31:0/2670974307 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f8e000b1c20 con 0x7f8e10102a60 2024-09-21T06:27:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:27:08.918 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:27:08.919 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:27:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8dfc0b6730 msgr2=0x7f8dfc0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8dfc0b6730 0x7f8dfc0b8c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8e1006a3f0 tx=0x7f8df800b390 comp rx=0 tx=0).stop 2024-09-21T06:27:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 msgr2=0x7f8e1019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 0x7f8e1019b2d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000d950 tx=0x7f8e0000de10 comp rx=0 tx=0).stop 2024-09-21T06:27:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 shutdown_connections 2024-09-21T06:27:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e10108cf0 0x7f8e1019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e101083a0 0x7f8e1019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f8dfc0b6730 0x7f8dfc0b8c40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 --2- 172.21.15.31:0/2670974307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10102a60 0x7f8e1019b2d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 >> 172.21.15.31:0/2670974307 conn(0x7f8e10079730 msgr2=0x7f8e10104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 shutdown_connections 2024-09-21T06:27:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:08.918+0000 7f8e15346640 1 -- 172.21.15.31:0/2670974307 wait complete. 2024-09-21T06:27:09.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:08 smithi110 ceph-mon[34550]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T06:27:09.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:08 smithi107 ceph-mon[34380]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T06:27:09.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:27:09.937 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:10.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:09 smithi110 ceph-mon[34550]: pgmap v806: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 KiB/s wr, 24 op/s 2024-09-21T06:27:10.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:09 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2670974307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:09 smithi107 ceph-mon[34380]: pgmap v806: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 KiB/s wr, 24 op/s 2024-09-21T06:27:10.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:09 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2670974307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:10.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:09 smithi031 ceph-mon[31237]: pgmap v806: 97 pgs: 97 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 74 KiB/s wr, 24 op/s 2024-09-21T06:27:10.058 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:09 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2670974307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.457+0000 7fbe128b5640 1 -- 172.21.15.31:0/2292566946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 msgr2=0x7fbe0c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.457+0000 7fbe128b5640 1 --2- 172.21.15.31:0/2292566946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c102ef0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009900 tx=0x7fbdfc02a410 comp rx=0 tx=0).stop 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 -- 172.21.15.31:0/2292566946 shutdown_connections 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 --2- 172.21.15.31:0/2292566946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe0c108d20 0x7fbe0c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 --2- 172.21.15.31:0/2292566946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe0c1083d0 0x7fbe0c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 --2- 172.21.15.31:0/2292566946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c102ef0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 -- 172.21.15.31:0/2292566946 >> 172.21.15.31:0/2292566946 conn(0x7fbe0c0797b0 msgr2=0x7fbe0c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 -- 172.21.15.31:0/2292566946 shutdown_connections 2024-09-21T06:27:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 -- 172.21.15.31:0/2292566946 wait complete. 2024-09-21T06:27:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 Processor -- start 2024-09-21T06:27:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.458+0000 7fbe128b5640 1 -- start start 2024-09-21T06:27:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe128b5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe128b5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe0c1083d0 0x7fbe0c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe128b5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe0c108d20 0x7fbe0c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe128b5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1129e0 con 0x7fbe0c102ae0 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe128b5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c112860 con 0x7fbe0c108d20 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe128b5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c112b60 con 0x7fbe0c1083d0 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47076/0 (socket says 172.21.15.31:47076) 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 -- 172.21.15.31:0/3429050537 learned_addr learned my addr 172.21.15.31:0/3429050537 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe0c1083d0 0x7fbe0c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe10e52640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe0c108d20 0x7fbe0c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 -- 172.21.15.31:0/3429050537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe0c1083d0 msgr2=0x7fbe0c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe0c1083d0 0x7fbe0c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 -- 172.21.15.31:0/3429050537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe0c108d20 msgr2=0x7fbe0c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe0c108d20 0x7fbe0c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.459+0000 7fbe0bfff640 1 -- 172.21.15.31:0/3429050537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c1a65f0 con 0x7fbe0c102ae0 2024-09-21T06:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.460+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c19b330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc005940 tx=0x7fbdfc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.460+0000 7fbe097fa640 1 -- 172.21.15.31:0/3429050537 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc03f030 con 0x7fbe0c102ae0 2024-09-21T06:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.460+0000 7fbe097fa640 1 -- 172.21.15.31:0/3429050537 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdfc0044c0 con 0x7fbe0c102ae0 2024-09-21T06:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.460+0000 7fbe097fa640 1 -- 172.21.15.31:0/3429050537 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc035070 con 0x7fbe0c102ae0 2024-09-21T06:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.460+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c1a6880 con 0x7fbe0c102ae0 2024-09-21T06:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.460+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c1a6d10 con 0x7fbe0c102ae0 2024-09-21T06:27:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 Processor -- start 2024-09-21T06:27:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 -- start start 2024-09-21T06:27:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe097fa640 1 -- 172.21.15.31:0/3429050537 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fbdfc042030 con 0x7fbe0c102ae0 2024-09-21T06:27:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f61c666ea0 con 0x55f61c349070 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f61c6562d0 con 0x55f61c328390 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe18e29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f61c65a0e0 con 0x55f61c3201b0 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe10e52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe10e52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47090/0 (socket says 172.21.15.31:47090) 2024-09-21T06:27:10.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.463+0000 7fbe10e52640 1 -- 172.21.15.31:0/338495408 learned_addr learned my addr 172.21.15.31:0/338495408 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:10.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/338495408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 msgr2=0x55f61c6f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/338495408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 msgr2=0x55f61c6f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/338495408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61c3b4650 con 0x55f61c328390 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f1120 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c06aaa0 tx=0x7fbe00011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c126ec34c0227b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe097fa640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdf40b6700 0x7fbdf40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbdf27fc640 1 -- 172.21.15.31:0/338495408 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00004050 con 0x55f61c328390 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbdf27fc640 1 -- 172.21.15.31:0/338495408 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe000054d0 con 0x55f61c328390 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbdf27fc640 1 -- 172.21.15.31:0/338495408 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00004c80 con 0x55f61c328390 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe10e52640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdf40b6700 0x7fbdf40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.464+0000 7fbe097fa640 1 -- 172.21.15.31:0/3429050537 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7fbdfc107ae0 con 0x7fbe0c102ae0 2024-09-21T06:27:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 -- 172.21.15.31:0/338495408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 msgr2=0x55f61c6f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe10e52640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdf40b6700 0x7fbdf40b8c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009fd0 tx=0x7fbe04009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f1120 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c06aaa0 tx=0x7fbe00011810 comp rx=0 tx=0).stop 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 -- 172.21.15.31:0/338495408 shutdown_connections 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f1120 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 --2- 172.21.15.31:0/338495408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 -- 172.21.15.31:0/338495408 >> 172.21.15.31:0/338495408 conn(0x55f61c33d460 msgr2=0x55f61c6ee590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 -- 172.21.15.31:0/338495408 shutdown_connections 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.465+0000 7fbe18e29740 1 -- 172.21.15.31:0/338495408 wait complete. 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 -- start start 2024-09-21T06:27:10.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 Processor -- start 2024-09-21T06:27:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c3201b0 0x55f61c6fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 0x55f61c6fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c349070 0x55f61c6fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x55f61c4b7700 con 0x55f61c328390 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f61c5f1160 con 0x55f61c3201b0 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f61c6fee80 con 0x55f61c349070 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 0x55f61c6fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 0x55f61c6fc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47092/0 (socket says 172.21.15.31:47092) 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/4003119659 learned_addr learned my addr 172.21.15.31:0/4003119659 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.466+0000 7fbe18e29740 10 client.0 _ll_register_callbacks cb 0x55f61c60a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe10e52640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c349070 0x55f61c6fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c3201b0 0x55f61c6fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/4003119659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c349070 msgr2=0x55f61c6fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c349070 0x55f61c6fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c3201b0 msgr2=0x55f61c6fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c3201b0 0x55f61c6fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 -- 172.21.15.31:0/4003119659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61c61ec90 con 0x55f61c328390 2024-09-21T06:27:10.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 0x55f61c6fc1e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbe00005e00 tx=0x7fbe00009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9355f8f79545914 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbdeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/4003119659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00027030 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe000054d0 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.467+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/4003119659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7fbe000035f0 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/4003119659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbe0001d040 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 10 client.18498 fetch_fsmap learned FSMap version 21 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f61c70aad0 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/4003119659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe000054d0 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/4003119659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbe00004ca0 con 0x55f61c328390 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 10 client.18498 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 10 client.18498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 20 client.18498 populate_metadata read hostname 'smithi031' 2024-09-21T06:27:10.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c3079f0 0x55f61c70de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 4 client.18498 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61c668df0 con 0x55f61c3079f0 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.468+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c3079f0 0x55f61c70de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.469+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c3079f0 0x55f61c70de90 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71bb9252dc1a81e6 server_cookie=2b1a0712c725325b in_seq=0 out_seq=0 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.469+0000 7fbdeb7fe640 10 client.18498 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.469+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/4003119659 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f61c668df0 con 0x55f61c3079f0 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.469+0000 7fbdeb7fe640 10 client.18498 handle_command_reply: tid=0 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.469+0000 7fbe18e29740 1 client.18498 shutdown 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 msgr2=0x55f61c6fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 0x55f61c6fc1e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbe00005e00 tx=0x7fbe00009fa0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 shutdown_connections 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c3079f0 0x55f61c70de90 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c349070 0x55f61c6fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c328390 0x55f61c6fc1e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbdeb7fe640 0 client.18498 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 --2- 172.21.15.31:0/4003119659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c3201b0 0x55f61c6fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 >> 172.21.15.31:0/4003119659 conn(0x55f61c33d460 msgr2=0x55f61c6f3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 shutdown_connections 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 1 -- 172.21.15.31:0/4003119659 wait complete. 2024-09-21T06:27:10.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.470+0000 7fbe18e29740 20 client.18498 trim_cache size 0 max 16384 2024-09-21T06:27:10.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.483+0000 7fbe18e29740 1 Processor -- start 2024-09-21T06:27:10.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.483+0000 7fbe18e29740 1 -- start start 2024-09-21T06:27:10.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe18e29740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1ab640 con 0x55f61c349070 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe18e29740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1adea0 con 0x55f61c328390 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe18e29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1abc90 con 0x55f61c3201b0 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47102/0 (socket says 172.21.15.31:47102) 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 -- 172.21.15.31:0/3509592565 learned_addr learned my addr 172.21.15.31:0/3509592565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 -- 172.21.15.31:0/3509592565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 msgr2=0x55f61c6fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 -- 172.21.15.31:0/3509592565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 msgr2=0x55f61c6f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 -- 172.21.15.31:0/3509592565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61c6fcf60 con 0x55f61c349070 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.484+0000 7fbe10e52640 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f93d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbe0401e550 tx=0x7fbe04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d816fa8038d798ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbdf1ffb640 1 -- 172.21.15.31:0/3509592565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04004b20 con 0x55f61c349070 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbdf1ffb640 1 -- 172.21.15.31:0/3509592565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe04004cc0 con 0x55f61c349070 2024-09-21T06:27:10.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbdf1ffb640 1 -- 172.21.15.31:0/3509592565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0401c810 con 0x55f61c349070 2024-09-21T06:27:10.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 -- 172.21.15.31:0/3509592565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 msgr2=0x55f61c6f93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f93d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbe0401e550 tx=0x7fbe04019000 comp rx=0 tx=0).stop 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 -- 172.21.15.31:0/3509592565 shutdown_connections 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c349070 0x55f61c6f93d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c328390 0x55f61c6f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 --2- 172.21.15.31:0/3509592565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c3201b0 0x55f61c6fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.485+0000 7fbe18e29740 1 -- 172.21.15.31:0/3509592565 >> 172.21.15.31:0/3509592565 conn(0x55f61c33d460 msgr2=0x55f61c6f2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.486+0000 7fbe18e29740 1 -- 172.21.15.31:0/3509592565 shutdown_connections 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.486+0000 7fbe18e29740 1 -- 172.21.15.31:0/3509592565 wait complete. 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.486+0000 7fbe18e29740 1 -- start start 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.486+0000 7fbe18e29740 1 Processor -- start 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 0x55f61c700400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c328390 0x55f61c700940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 0x55f61c7776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1ac360 con 0x55f61c3201b0 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1ad7d0 con 0x55f61c349070 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1aca30 con 0x55f61c328390 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe18e29740 10 client.0 _ll_register_callbacks cb 0x7fbe040099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 0x55f61c700400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe10e52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 0x55f61c7776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe10e52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 0x55f61c7776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43712/0 (socket says 172.21.15.31:43712) 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 0x55f61c700400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47112/0 (socket says 172.21.15.31:47112) 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 -- 172.21.15.31:0/1729316013 learned_addr learned my addr 172.21.15.31:0/1729316013 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c328390 0x55f61c700940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 -- 172.21.15.31:0/1729316013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c328390 msgr2=0x55f61c700940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c328390 0x55f61c700940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 -- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 msgr2=0x55f61c7776c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 0x55f61c7776c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 -- 172.21.15.31:0/1729316013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61c701230 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe10e52640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 0x55f61c7776c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.487+0000 7fbe0bfff640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 0x55f61c700400 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc006010 tx=0x7fbdfc1128f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15f994391128abb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbdeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/1729316013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc049040 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/1729316013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7fbdfc0d6eb0 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe04004cc0 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/1729316013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbdfc0cd270 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbe18e29740 10 client.18504 fetch_fsmap learned FSMap version 21 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f61c781520 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.488+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/1729316013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe04004cc0 con 0x55f61c3201b0 2024-09-21T06:27:10.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/1729316013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbdfc0d2020 con 0x55f61c3201b0 2024-09-21T06:27:10.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe18e29740 10 client.18504 fetch_fsmap finished waiting for FSMap version 21 2024-09-21T06:27:10.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe18e29740 10 client.18504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:27:10.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe18e29740 20 client.18504 populate_metadata read hostname 'smithi031' 2024-09-21T06:27:10.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe18e29740 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c2dcda0 0x55f61c783ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:10.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe18e29740 4 client.18504 mds_command: new command op to 24719 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:27:10.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 --> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f61c6648a0 con 0x55f61c2dcda0 2024-09-21T06:27:10.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.489+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c2dcda0 0x55f61c783ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.490+0000 7fbdeb7fe640 10 client.18504 ms_handle_connect on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.490+0000 7fbe0b7fe640 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c2dcda0 0x55f61c783ce0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47773a02ab50224 server_cookie=8552f83bff04c9a5 in_seq=0 out_seq=0 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.490+0000 7fbdeb7fe640 1 -- 172.21.15.31:0/1729316013 <== mds.0 v2:172.21.15.107:6838/648775801 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f61c6648a0 con 0x55f61c2dcda0 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.490+0000 7fbdeb7fe640 10 client.18504 handle_command_reply: tid=0 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.490+0000 7fbe18e29740 1 client.18504 shutdown 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 msgr2=0x55f61c700400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 0x55f61c700400 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc006010 tx=0x7fbdfc1128f0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 shutdown_connections 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:6838/648775801,v1:172.21.15.107:6839/648775801] conn(0x55f61c2dcda0 0x55f61c783ce0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbdeb7fe640 0 client.18504 ms_handle_reset on v2:172.21.15.107:6838/648775801 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f61c349070 0x55f61c7776c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f61c328390 0x55f61c700940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 --2- 172.21.15.31:0/1729316013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f61c3201b0 0x55f61c700400 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 >> 172.21.15.31:0/1729316013 conn(0x55f61c33d460 msgr2=0x55f61c6f2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:10.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 shutdown_connections 2024-09-21T06:27:10.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 1 -- 172.21.15.31:0/1729316013 wait complete. 2024-09-21T06:27:10.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.491+0000 7fbe18e29740 20 client.18504 trim_cache size 0 max 16384 2024-09-21T06:27:10.497 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdf40b6700 msgr2=0x7fbdf40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdf40b6700 0x7fbdf40b8c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009fd0 tx=0x7fbe04009500 comp rx=0 tx=0).stop 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 msgr2=0x7fbe0c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c19b330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc005940 tx=0x7fbdfc0040c0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 shutdown_connections 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe0c108d20 0x7fbe0c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe0c1083d0 0x7fbe0c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fbdf40b6700 0x7fbdf40b8c10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 --2- 172.21.15.31:0/3429050537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe0c102ae0 0x7fbe0c19b330 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 >> 172.21.15.31:0/3429050537 conn(0x7fbe0c0797b0 msgr2=0x7fbe0c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 shutdown_connections 2024-09-21T06:27:10.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:10.492+0000 7fbe128b5640 1 -- 172.21.15.31:0/3429050537 wait complete. 2024-09-21T06:27:11.206 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T06:27:11.207 INFO:tasks.fwd_scrub:done joining 2024-09-21T06:27:11.207 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T06:27:11.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:27:11.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 -- 172.21.15.31:0/2405902162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c1024e0 msgr2=0x7fed4c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 --2- 172.21.15.31:0/2405902162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c1024e0 0x7fed4c1028f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fed48009900 tx=0x7fed4802a480 comp rx=0 tx=0).stop 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 -- 172.21.15.31:0/2405902162 shutdown_connections 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 --2- 172.21.15.31:0/2405902162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed4c110090 0x7fed4c1124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 --2- 172.21.15.31:0/2405902162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c102e30 0x7fed4c10fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 --2- 172.21.15.31:0/2405902162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c1024e0 0x7fed4c1028f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 -- 172.21.15.31:0/2405902162 >> 172.21.15.31:0/2405902162 conn(0x7fed4c0fe380 msgr2=0x7fed4c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 -- 172.21.15.31:0/2405902162 shutdown_connections 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.339+0000 7fed53320640 1 -- 172.21.15.31:0/2405902162 wait complete. 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.340+0000 7fed53320640 1 Processor -- start 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.340+0000 7fed53320640 1 -- start start 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.340+0000 7fed53320640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c1024e0 0x7fed4c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.340+0000 7fed53320640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed4c102e30 0x7fed4c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.340+0000 7fed53320640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 0x7fed4c1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.340+0000 7fed53320640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c115b80 con 0x7fed4c110090 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed53320640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c115a00 con 0x7fed4c102e30 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed53320640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c115d00 con 0x7fed4c1024e0 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 0x7fed4c1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed508bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed4c102e30 0x7fed4c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 0x7fed4c1a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47128/0 (socket says 172.21.15.31:47128) 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 -- 172.21.15.31:0/3766912016 learned_addr learned my addr 172.21.15.31:0/3766912016 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed510bc640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c1024e0 0x7fed4c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 -- 172.21.15.31:0/3766912016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c1024e0 msgr2=0x7fed4c19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c1024e0 0x7fed4c19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 -- 172.21.15.31:0/3766912016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed4c102e30 msgr2=0x7fed4c19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed4c102e30 0x7fed4c19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 -- 172.21.15.31:0/3766912016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c1a24b0 con 0x7fed4c110090 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.341+0000 7fed518bd640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 0x7fed4c1a1db0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fed4400eb10 tx=0x7fed4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.342+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44004b20 con 0x7fed4c110090 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.342+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4c1a27a0 con 0x7fed4c110090 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.342+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed44004cc0 con 0x7fed4c110090 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.342+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44010670 con 0x7fed4c110090 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.343+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4c1a8f30 con 0x7fed4c110090 2024-09-21T06:27:11.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.344+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4c06ae20 con 0x7fed4c110090 2024-09-21T06:27:11.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.345+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fed44005020 con 0x7fed4c110090 2024-09-21T06:27:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.347+0000 7fed427fc640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fed380b6700 0x7fed380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.347+0000 7fed510bc640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fed380b6700 0x7fed380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.347+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7fed440e7bf0 con 0x7fed4c110090 2024-09-21T06:27:11.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.348+0000 7fed510bc640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fed380b6700 0x7fed380b8c10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed48031440 tx=0x7fed48009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:11.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.354+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed440ae130 con 0x7fed4c110090 2024-09-21T06:27:11.555 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.554+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed4c1a91f0 con 0x7fed4c110090 2024-09-21T06:27:11.557 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.555+0000 7fed427fc640 1 -- 172.21.15.31:0/3766912016 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23248 (secure 0 0 0) 0x7fed440b2510 con 0x7fed4c110090 2024-09-21T06:27:11.557 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:27:11.557 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":97,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:27:07.665445+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:27:11.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fed380b6700 msgr2=0x7fed380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fed380b6700 0x7fed380b8c10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed48031440 tx=0x7fed48009560 comp rx=0 tx=0).stop 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 msgr2=0x7fed4c1a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 0x7fed4c1a1db0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fed4400eb10 tx=0x7fed4400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 shutdown_connections 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fed380b6700 0x7fed380b8c10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed4c110090 0x7fed4c1a1db0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed4c102e30 0x7fed4c19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 --2- 172.21.15.31:0/3766912016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c1024e0 0x7fed4c19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.557+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 >> 172.21.15.31:0/3766912016 conn(0x7fed4c0fe380 msgr2=0x7fed4c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.558+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 shutdown_connections 2024-09-21T06:27:11.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.558+0000 7fed53320640 1 -- 172.21.15.31:0/3766912016 wait complete. 2024-09-21T06:27:11.572 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:05:05.978676+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:09:33.023556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:09:34.842253+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:27:11.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:27:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 -- 172.21.15.31:0/633899293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 msgr2=0x7f5d90110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 --2- 172.21.15.31:0/633899293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90110230 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009ca0 tx=0x7f5d8802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 -- 172.21.15.31:0/633899293 shutdown_connections 2024-09-21T06:27:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 --2- 172.21.15.31:0/633899293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90110230 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 --2- 172.21.15.31:0/633899293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d90105230 0x7f5d9010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 --2- 172.21.15.31:0/633899293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d901048e0 0x7f5d90104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 -- 172.21.15.31:0/633899293 >> 172.21.15.31:0/633899293 conn(0x7f5d900fe3b0 msgr2=0x7f5d90100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 -- 172.21.15.31:0/633899293 shutdown_connections 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.688+0000 7f5d976f9640 1 -- 172.21.15.31:0/633899293 wait complete. 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 Processor -- start 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 -- start start 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d901048e0 0x7f5d90102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 0x7f5d90102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901139c0 con 0x7f5d9010dd40 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90113840 con 0x7f5d90105230 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d976f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90113b40 con 0x7f5d901048e0 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d95c96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 0x7f5d90102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 0x7f5d90102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.31:43744/0 (socket says 172.21.15.31:43744) 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d95c96640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47136/0 (socket says 172.21.15.31:47136) 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 -- 172.21.15.31:0/4183285826 learned_addr learned my addr 172.21.15.31:0/4183285826 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d95495640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d901048e0 0x7f5d90102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 -- 172.21.15.31:0/4183285826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d901048e0 msgr2=0x7f5d90102610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d901048e0 0x7f5d90102610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 -- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 msgr2=0x7f5d90103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.689+0000 7f5d94c94640 1 -- 172.21.15.31:0/4183285826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d901a8650 con 0x7f5d90105230 2024-09-21T06:27:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.690+0000 7f5d94c94640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 0x7f5d90102b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000e9c0 tx=0x7f5d8000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.690+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000ccc0 con 0x7f5d90105230 2024-09-21T06:27:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.690+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d800054d0 con 0x7f5d90105230 2024-09-21T06:27:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.690+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d901a8940 con 0x7f5d90105230 2024-09-21T06:27:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.690+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80010430 con 0x7f5d90105230 2024-09-21T06:27:11.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.690+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d901a8e90 con 0x7f5d90105230 2024-09-21T06:27:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.691+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d90104cf0 con 0x7f5d90105230 2024-09-21T06:27:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.693+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5d80005000 con 0x7f5d90105230 2024-09-21T06:27:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.693+0000 7f5d867fc640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d7c0b6730 0x7f5d7c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.693+0000 7f5d95495640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d7c0b6730 0x7f5d7c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.694+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7f5d800e7b90 con 0x7f5d90105230 2024-09-21T06:27:11.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.694+0000 7f5d95495640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d7c0b6730 0x7f5d7c0b8c40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5d90104060 tx=0x7f5d8c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:11.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.697+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d80056710 con 0x7f5d90105230 2024-09-21T06:27:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:11 smithi031 ceph-mon[31237]: pgmap v807: 97 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 63 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-09-21T06:27:11.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:11 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/3766912016' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:27:11.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.892+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5d901a9280 con 0x7f5d90105230 2024-09-21T06:27:11.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.892+0000 7f5d867fc640 1 -- 172.21.15.31:0/4183285826 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f5d800052f0 con 0x7f5d90105230 2024-09-21T06:27:11.894 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.894+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d7c0b6730 msgr2=0x7f5d7c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.894+0000 7f5d976f9640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d7c0b6730 0x7f5d7c0b8c40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5d90104060 tx=0x7f5d8c005f90 comp rx=0 tx=0).stop 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.894+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 msgr2=0x7f5d90102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.894+0000 7f5d976f9640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 0x7f5d90102b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000e9c0 tx=0x7f5d8000ee80 comp rx=0 tx=0).stop 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 shutdown_connections 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f5d7c0b6730 0x7f5d7c0b8c40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d9010dd40 0x7f5d90103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90105230 0x7f5d90102b50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 --2- 172.21.15.31:0/4183285826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d901048e0 0x7f5d90102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 >> 172.21.15.31:0/4183285826 conn(0x7f5d900fe3b0 msgr2=0x7f5d901077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:11.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 shutdown_connections 2024-09-21T06:27:11.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:11.895+0000 7f5d976f9640 1 -- 172.21.15.31:0/4183285826 wait complete. 2024-09-21T06:27:11.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:27:12.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 -- 172.21.15.31:0/823363540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210101fe0 msgr2=0x7fb2101023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 --2- 172.21.15.31:0/823363540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210101fe0 0x7fb2101023f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb200009900 tx=0x7fb20002a410 comp rx=0 tx=0).stop 2024-09-21T06:27:12.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 -- 172.21.15.31:0/823363540 shutdown_connections 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 --2- 172.21.15.31:0/823363540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb210106b10 0x7fb2101105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 --2- 172.21.15.31:0/823363540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2101061c0 0x7fb2101065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 --2- 172.21.15.31:0/823363540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210101fe0 0x7fb2101023f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 -- 172.21.15.31:0/823363540 >> 172.21.15.31:0/823363540 conn(0x7fb210079e10 msgr2=0x7fb21007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 -- 172.21.15.31:0/823363540 shutdown_connections 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.024+0000 7fb215c8b640 1 -- 172.21.15.31:0/823363540 wait complete. 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 Processor -- start 2024-09-21T06:27:12.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 -- start start 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb210101fe0 0x7fb21019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2101061c0 0x7fb21019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 0x7fb21019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb210113a00 con 0x7fb210106b10 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb210113880 con 0x7fb2101061c0 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.025+0000 7fb215c8b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb210113b80 con 0x7fb210101fe0 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 0x7fb21019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 0x7fb21019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47144/0 (socket says 172.21.15.31:47144) 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 -- 172.21.15.31:0/2988677048 learned_addr learned my addr 172.21.15.31:0/2988677048 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20effd640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2101061c0 0x7fb21019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20f7fe640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb210101fe0 0x7fb21019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 -- 172.21.15.31:0/2988677048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb210101fe0 msgr2=0x7fb21019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb210101fe0 0x7fb21019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 -- 172.21.15.31:0/2988677048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2101061c0 msgr2=0x7fb21019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2101061c0 0x7fb21019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 -- 172.21.15.31:0/2988677048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2101a0350 con 0x7fb210106b10 2024-09-21T06:27:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20effd640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2101061c0 0x7fb21019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20ffff640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 0x7fb21019fc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb20800eb10 tx=0x7fb20800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208004b20 con 0x7fb210106b10 2024-09-21T06:27:12.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2101a0640 con 0x7fb210106b10 2024-09-21T06:27:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb208004cc0 con 0x7fb210106b10 2024-09-21T06:27:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208010670 con 0x7fb210106b10 2024-09-21T06:27:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.026+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2101a6dd0 con 0x7fb210106b10 2024-09-21T06:27:12.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.027+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2101023f0 con 0x7fb210106b10 2024-09-21T06:27:12.034 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:11 smithi110 ceph-mon[34550]: pgmap v807: 97 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 63 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-09-21T06:27:12.034 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:11 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/3766912016' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:27:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.029+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb208005000 con 0x7fb210106b10 2024-09-21T06:27:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.030+0000 7fb20cff9640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb1f80b6680 0x7fb1f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.031+0000 7fb20f7fe640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb1f80b6680 0x7fb1f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.031+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7fb2080e7ee0 con 0x7fb210106b10 2024-09-21T06:27:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.031+0000 7fb20f7fe640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb1f80b6680 0x7fb1f80b8b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb200000c00 tx=0x7fb200031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:12.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.034+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb208056710 con 0x7fb210106b10 2024-09-21T06:27:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:11 smithi107 ceph-mon[34380]: pgmap v807: 97 pgs: 24 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 63 active+clean; 4.5 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-09-21T06:27:12.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:11 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/3766912016' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:27:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.227+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb21002e640 con 0x7fb210106b10 2024-09-21T06:27:12.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.228+0000 7fb20cff9640 1 -- 172.21.15.31:0/2988677048 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7fb2080ae420 con 0x7fb210106b10 2024-09-21T06:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb1f80b6680 msgr2=0x7fb1f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb1f80b6680 0x7fb1f80b8b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb200000c00 tx=0x7fb200031400 comp rx=0 tx=0).stop 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 msgr2=0x7fb21019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 0x7fb21019fc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb20800eb10 tx=0x7fb20800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 shutdown_connections 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fb1f80b6680 0x7fb1f80b8b90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb210106b10 0x7fb21019fc50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2101061c0 0x7fb21019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 --2- 172.21.15.31:0/2988677048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb210101fe0 0x7fb21019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 >> 172.21.15.31:0/2988677048 conn(0x7fb210079e10 msgr2=0x7fb210107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 shutdown_connections 2024-09-21T06:27:12.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.230+0000 7fb215c8b640 1 -- 172.21.15.31:0/2988677048 wait complete. 2024-09-21T06:27:12.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:27:12.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 -- 172.21.15.31:0/2064662866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2438101fe0 msgr2=0x7f24381023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 --2- 172.21.15.31:0/2064662866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2438101fe0 0x7f24381023f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2434009900 tx=0x7f243402a410 comp rx=0 tx=0).stop 2024-09-21T06:27:12.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 -- 172.21.15.31:0/2064662866 shutdown_connections 2024-09-21T06:27:12.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 --2- 172.21.15.31:0/2064662866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2438106b10 0x7f24381105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 --2- 172.21.15.31:0/2064662866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24381061c0 0x7f24381065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 --2- 172.21.15.31:0/2064662866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2438101fe0 0x7f24381023f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 -- 172.21.15.31:0/2064662866 >> 172.21.15.31:0/2064662866 conn(0x7f2438079e10 msgr2=0x7f243807a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:12.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 -- 172.21.15.31:0/2064662866 shutdown_connections 2024-09-21T06:27:12.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.362+0000 7f244079b640 1 -- 172.21.15.31:0/2064662866 wait complete. 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 Processor -- start 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 -- start start 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2438101fe0 0x7f243819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 0x7f243819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2438106b10 0x7f243819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2438113a50 con 0x7f24381061c0 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24381138d0 con 0x7f2438106b10 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f244079b640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2438113bd0 con 0x7f2438101fe0 2024-09-21T06:27:12.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f243dd36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 0x7f243819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f243dd36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 0x7f243819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47166/0 (socket says 172.21.15.31:47166) 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f243dd36640 1 -- 172.21.15.31:0/1190605271 learned_addr learned my addr 172.21.15.31:0/1190605271 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f243ed38640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2438106b10 0x7f243819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.363+0000 7f243e537640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2438101fe0 0x7f243819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f243dd36640 1 -- 172.21.15.31:0/1190605271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2438101fe0 msgr2=0x7f243819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f243dd36640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2438101fe0 0x7f243819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f243dd36640 1 -- 172.21.15.31:0/1190605271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2438106b10 msgr2=0x7f243819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f243dd36640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2438106b10 0x7f243819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f243dd36640 1 -- 172.21.15.31:0/1190605271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24381a0440 con 0x7f24381061c0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f243dd36640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 0x7f243819aab0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f242800d740 tx=0x7f242800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428017070 con 0x7f24381061c0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242800de50 con 0x7f24381061c0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242800f620 con 0x7f24381061c0 2024-09-21T06:27:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24381a0730 con 0x7f24381061c0 2024-09-21T06:27:12.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.364+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a6ec0 con 0x7f24381061c0 2024-09-21T06:27:12.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.365+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24381023f0 con 0x7f24381061c0 2024-09-21T06:27:12.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.367+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f242801a070 con 0x7f24381061c0 2024-09-21T06:27:12.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.368+0000 7f242f7fe640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24240b6700 0x7f24240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.368+0000 7f243e537640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24240b6700 0x7f24240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.368+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7f24280e7010 con 0x7f24381061c0 2024-09-21T06:27:12.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.368+0000 7f243e537640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24240b6700 0x7f24240b8c10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2434031440 tx=0x7f2434005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:12.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.370+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24280b19b0 con 0x7f24381061c0 2024-09-21T06:27:12.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.563+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f243810bc30 con 0x7f24381061c0 2024-09-21T06:27:12.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.564+0000 7f242f7fe640 1 -- 172.21.15.31:0/1190605271 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f24280b68f0 con 0x7f24381061c0 2024-09-21T06:27:12.567 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2024-09-21T06:27:12.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24240b6700 msgr2=0x7f24240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24240b6700 0x7f24240b8c10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2434031440 tx=0x7f2434005a20 comp rx=0 tx=0).stop 2024-09-21T06:27:12.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 msgr2=0x7f243819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 0x7f243819aab0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f242800d740 tx=0x7f242800dc00 comp rx=0 tx=0).stop 2024-09-21T06:27:12.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 shutdown_connections 2024-09-21T06:27:12.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2438106b10 0x7f243819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f24240b6700 0x7f24240b8c10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24381061c0 0x7f243819aab0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 --2- 172.21.15.31:0/1190605271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2438101fe0 0x7f243819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 >> 172.21.15.31:0/1190605271 conn(0x7f2438079e10 msgr2=0x7f24381074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:12.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 shutdown_connections 2024-09-21T06:27:12.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.566+0000 7f244079b640 1 -- 172.21.15.31:0/1190605271 wait complete. 2024-09-21T06:27:12.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1826998578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd01093e0 msgr2=0x7f9dd01097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1826998578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd01093e0 0x7f9dd01097f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009900 tx=0x7f9db802a410 comp rx=0 tx=0).stop 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1826998578 shutdown_connections 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1826998578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 0x7f9dd0110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1826998578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd01093e0 0x7f9dd01097f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1826998578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd0103ab0 0x7f9dd0103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.697+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1826998578 >> 172.21.15.31:0/1826998578 conn(0x7f9dd00ff4d0 msgr2=0x7f9dd0101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:12.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.698+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1826998578 shutdown_connections 2024-09-21T06:27:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.698+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1826998578 wait complete. 2024-09-21T06:27:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.698+0000 7f9dd5e84640 1 Processor -- start 2024-09-21T06:27:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.698+0000 7f9dd5e84640 1 -- start start 2024-09-21T06:27:12.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.698+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 0x7f9dd019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.698+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd01093e0 0x7f9dd019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dd5e84640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 0x7f9dd019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd01139f0 con 0x7f9dd0103ab0 2024-09-21T06:27:12.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0113870 con 0x7f9dd0109d30 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dd5e84640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0113b70 con 0x7f9dd01093e0 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 0x7f9dd019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 0x7f9dd019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47176/0 (socket says 172.21.15.31:47176) 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 -- 172.21.15.31:0/1815649567 learned_addr learned my addr 172.21.15.31:0/1815649567 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd01093e0 0x7f9dd019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcffff640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 0x7f9dd019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 -- 172.21.15.31:0/1815649567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd01093e0 msgr2=0x7f9dd019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd01093e0 0x7f9dd019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 -- 172.21.15.31:0/1815649567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 msgr2=0x7f9dd019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 0x7f9dd019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 -- 172.21.15.31:0/1815649567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd01a03a0 con 0x7f9dd0103ab0 2024-09-21T06:27:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcffff640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 0x7f9dd019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dc7fff640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd01093e0 0x7f9dd019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.699+0000 7f9dcf7fe640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 0x7f9dd019a470 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e9f0 tx=0x7f9dc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.705+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc000cd80 con 0x7f9dd0103ab0 2024-09-21T06:27:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.705+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dc00054d0 con 0x7f9dd0103ab0 2024-09-21T06:27:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.705+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0010690 con 0x7f9dd0103ab0 2024-09-21T06:27:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.705+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd01a0690 con 0x7f9dd0103ab0 2024-09-21T06:27:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.705+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd01a6e20 con 0x7f9dd0103ab0 2024-09-21T06:27:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.706+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0103ec0 con 0x7f9dd0103ab0 2024-09-21T06:27:12.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.707+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9dc0004e30 con 0x7f9dd0103ab0 2024-09-21T06:27:12.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.708+0000 7f9dcd7fa640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9dbc0b6700 0x7f9dbc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:12.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.708+0000 7f9dc7fff640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9dbc0b6700 0x7f9dbc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:12.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.708+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8543+0+0 (secure 0 0 0) 0x7f9dc00e7d50 con 0x7f9dd0103ab0 2024-09-21T06:27:12.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.709+0000 7f9dc7fff640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9dbc0b6700 0x7f9dbc0b8c10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009230 tx=0x7f9db8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:12.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.711+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9dc0056710 con 0x7f9dd0103ab0 2024-09-21T06:27:12.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:12 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/4183285826' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:27:12.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:12 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2988677048' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:27:12.885 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:12 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1190605271' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:27:12.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.978+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9dd002e640 con 0x7f9dd0103ab0 2024-09-21T06:27:12.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.987+0000 7f9dcd7fa640 1 -- 172.21.15.31:0/1815649567 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+834054 (secure 0 0 0) 0x7f9dc00ae290 con 0x7f9dd0103ab0 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 24, 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 14226, 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2024-09-21T06:27:12.998 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2760018577 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2760018577 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:12.999 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/2760018577", 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2024-09-21T06:03:05.499525+0000", 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24157, 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.000 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.002 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-21T06:27:13.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-21T06:27:13.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T06:27:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-21T06:27:13.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-21T06:27:13.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T06:27:13.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.028 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T06:27:13.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-21T06:27:13.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.038 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:27:13.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-21T06:27:13.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-21T06:27:13.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-21T06:27:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.049 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-21T06:27:13.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:27:13.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.058 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.059 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nginx": { 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nginx", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nginx container image", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:27:13.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:27:13.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:27:13.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba_metrics": { 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-21T06:27:13.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.079 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.081 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-21T06:27:13.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:27:13.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-21T06:27:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-21T06:27:13.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-21T06:27:13.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-21T06:27:13.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.099 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-21T06:27:13.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-21T06:27:13.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_interval": { 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:27:13.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:27:13.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.114 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:13.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:13.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-09-21T06:27:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:27:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:27:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:27:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:27:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:27:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:27:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:27:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:27:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-21T06:27:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-21T06:27:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "sso_oauth2": { 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sso_oauth2", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-21T06:27:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-21T06:27:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-21T06:27:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-21T06:27:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-21T06:27:13.215 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:27:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:27:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T06:27:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:27:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:27:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-21T06:27:13.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.347 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.352 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-21T06:27:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.358 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max_requests": { 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_requests", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.370 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-21T06:27:13.374 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.382 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.388 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-21T06:27:13.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.391 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.392 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.393 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smb", 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "internal_store_backend": { 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "internal_store_backend", 2024-09-21T06:27:13.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.406 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.408 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "update_orchestration": { 2024-09-21T06:27:13.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "update_orchestration", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:27:13.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.415 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.422 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.424 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.430 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.436 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.439 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.442 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-21T06:27:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-21T06:27:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-21T06:27:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-21T06:27:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-21T06:27:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 24170, 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-21T06:27:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-21T06:27:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-21T06:27:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-21T06:27:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:27:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T06:27:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:27:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-21T06:27:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-21T06:27:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:27:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-21T06:27:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nginx": { 2024-09-21T06:27:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nginx", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nginx container image", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba_metrics": { 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:27:13.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-21T06:27:13.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.576 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:27:13.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.581 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.585 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.586 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.587 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:27:13.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_interval": { 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:27:13.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-21T06:27:13.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-21T06:27:13.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-21T06:27:13.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.598 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-21T06:27:13.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:27:13.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.610 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.611 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.612 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.613 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.614 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:27:13.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:27:13.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:27:13.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:27:13.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:13.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.631 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.632 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:27:13.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:27:13.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:27:13.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:27:13.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.637 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.638 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.639 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.640 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:27:13.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.642 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.643 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.644 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.656 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.657 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.659 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.660 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "sso_oauth2": { 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sso_oauth2", 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T06:27:13.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-21T06:27:13.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.677 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.681 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T06:27:13.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-21T06:27:13.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.691 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.692 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.695 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.696 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.697 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.698 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.699 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-21T06:27:13.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-21T06:27:13.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.708 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-21T06:27:13.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-21T06:27:13.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-21T06:27:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.718 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.727 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-21T06:27:13.730 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-21T06:27:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-21T06:27:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-21T06:27:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-21T06:27:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.802 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.805 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.809 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.815 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.818 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.820 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-21T06:27:13.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-21T06:27:13.822 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T06:27:13.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-21T06:27:13.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.830 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.836 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-21T06:27:13.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T06:27:13.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.841 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.842 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.843 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.845 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max_requests": { 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_requests", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.847 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:13.848 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.852 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.853 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.854 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-21T06:27:13.858 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.859 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.861 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.862 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.863 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.867 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smb", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "internal_store_backend": { 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "internal_store_backend", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.890 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "update_orchestration": { 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "update_orchestration", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T06:27:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:27:13.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.899 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.904 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.913 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.919 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-21T06:27:13.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.926 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.927 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.928 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.940 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.941 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.942 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.945 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.949 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.957 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.958 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-21T06:27:13.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-21T06:27:13.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:27:13.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.964 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-21T06:27:13.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.972 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:13.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "restful", 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_schedule" 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2024-09-21T06:27:13.979 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.981 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:27:13.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2024-09-21T06:27:13.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2024-09-21T06:27:13.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2024-09-21T06:27:13.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:27:13.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:27:13.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2024-09-21T06:27:13.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:13.994 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:13.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:13.998 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:13.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2024-09-21T06:27:14.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2024-09-21T06:27:14.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2024-09-21T06:27:14.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T06:27:14.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "read", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap-read" 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2024-09-21T06:27:14.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:27:14.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:27:14.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:27:14.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:27:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:27:14.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2024-09-21T06:27:14.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:14.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2024-09-21T06:27:14.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:27:14.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:27:14.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nginx": { 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nginx", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nginx container image", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.042 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba": { 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_samba_metrics": { 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:27:14.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2024-09-21T06:27:14.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2024-09-21T06:27:14.047 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:27:14.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2024-09-21T06:27:14.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2024-09-21T06:27:14.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2024-09-21T06:27:14.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2024-09-21T06:27:14.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "grafana_dashboards_path": { 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:27:14.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2024-09-21T06:27:14.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "hw_monitoring": { 2024-09-21T06:27:14.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hw_monitoring", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2024-09-21T06:27:14.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.058 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:27:14.067 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.069 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "oob_default_addr": { 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "oob_default_addr", 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "169.254.1.1", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.073 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2024-09-21T06:27:14.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2024-09-21T06:27:14.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.076 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.077 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_keepalive_interval": { 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2024-09-21T06:27:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "[]", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "{}", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:27:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2024-09-21T06:27:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2024-09-21T06:27:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2024-09-21T06:27:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T06:27:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2024-09-21T06:27:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "sso_oauth2": { 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sso_oauth2", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T06:27:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2024-09-21T06:27:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T06:27:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2024-09-21T06:27:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2024-09-21T06:27:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:27:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2024-09-21T06:27:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2024-09-21T06:27:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.235 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.239 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.246 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.248 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.250 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.257 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.261 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.267 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.274 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.275 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.278 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.279 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.286 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:27:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T06:27:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2024-09-21T06:27:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:27:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2024-09-21T06:27:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2024-09-21T06:27:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2024-09-21T06:27:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max_requests": { 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_requests", 2024-09-21T06:27:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2024-09-21T06:27:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2024-09-21T06:27:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:27:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2024-09-21T06:27:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2024-09-21T06:27:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smb", 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "internal_store_backend": { 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "internal_store_backend", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "update_orchestration": { 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "update_orchestration", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:27:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2024-09-21T06:27:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2024-09-21T06:27:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2024-09-21T06:27:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.434 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.435 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.445 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.454 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2024-09-21T06:27:14.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2024-09-21T06:27:14.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:27:14.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:27:14.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.467 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2024-09-21T06:27:14.468 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "sqlite3_killpoint": { 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "dev", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2024-09-21T06:27:14.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2024-09-21T06:27:14.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2024-09-21T06:27:14.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:27:14.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2024-09-21T06:27:14.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2024-09-21T06:27:14.475 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T06:27:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T06:27:14.477 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "squid": [ 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2024-09-21T06:27:14.478 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 5, 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3521725758 2024-09-21T06:27:14.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2872515961 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2024-09-21T06:27:14.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1008423759 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1276701267 2024-09-21T06:27:14.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2129361624 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:14.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1742853890 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-21T06:27:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9dbc0b6700 msgr2=0x7f9dbc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9dbc0b6700 0x7f9dbc0b8c10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009230 tx=0x7f9db8031400 comp rx=0 tx=0).stop 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 msgr2=0x7f9dd019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 0x7f9dd019a470 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e9f0 tx=0x7f9dc000edf0 comp rx=0 tx=0).stop 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 shutdown_connections 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd0109d30 0x7f9dd019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9dd01093e0 0x7f9dd019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f9dbc0b6700 0x7f9dbc0b8c10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 --2- 172.21.15.31:0/1815649567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0103ab0 0x7f9dd019a470 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 >> 172.21.15.31:0/1815649567 conn(0x7f9dd00ff4d0 msgr2=0x7f9dd01018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 shutdown_connections 2024-09-21T06:27:14.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:12.993+0000 7f9dd5e84640 1 -- 172.21.15.31:0/1815649567 wait complete. 2024-09-21T06:27:14.532 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T06:27:14.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T06:27:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:13 smithi031 ceph-mon[31237]: pgmap v808: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 KiB/s wr, 25 op/s 2024-09-21T06:27:14.532 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:13 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1815649567' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:27:14.533 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:12 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/4183285826' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:12 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2988677048' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:12 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1190605271' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:13 smithi107 ceph-mon[34380]: pgmap v808: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 KiB/s wr, 25 op/s 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:13 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1815649567' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:12 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/4183285826' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:12 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2988677048' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:12 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1190605271' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:13 smithi110 ceph-mon[34550]: pgmap v808: 97 pgs: 40 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 36 active+clean; 4.3 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 KiB/s wr, 25 op/s 2024-09-21T06:27:14.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:13 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1815649567' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:27:14.794 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:15.266 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2944, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.003443019, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000039133, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002138353, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:15.267 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.003936066, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.001312022 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.268 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 219, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 199280, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.021065272, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.016451537, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001903818, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:15.269 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 217, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.023020245, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000106084 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 199280 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.270 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 290, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 285, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 857296, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 176064, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.051718767, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.023109845, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.035720813, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 284, 2024-09-21T06:27:15.271 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.022579587, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000079505 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 857296, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 176064 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-21T06:27:15.272 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:15.273 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:15.274 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:15.275 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 147920, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 27, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-21T06:27:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T06:27:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 292, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 816303, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 292, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 292, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 816303, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:15.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:15.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:15.279 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:27:15.875 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-21T06:27:15.875 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-21T06:27:15.875 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-21T06:27:15.875 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T06:27:15.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T06:27:16.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:15 smithi110 ceph-mon[34550]: pgmap v809: 97 pgs: 34 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 44 active+clean; 4.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 KiB/s wr, 25 op/s 2024-09-21T06:27:16.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:15 smithi107 ceph-mon[34380]: pgmap v809: 97 pgs: 34 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 44 active+clean; 4.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 KiB/s wr, 25 op/s 2024-09-21T06:27:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:16.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:15 smithi031 ceph-mon[31237]: pgmap v809: 97 pgs: 34 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 44 active+clean; 4.2 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 74 KiB/s wr, 25 op/s 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001397339, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000307443, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:16.609 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 297, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 292, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 859888, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 179760, 2024-09-21T06:27:16.610 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.052246814, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.024505266, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.036290980, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 291, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.014071228, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000048354 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 859888, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 179760 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 6, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 224, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3264, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 200256, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.022845186, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.016104229, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.004345386, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 223, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.021243123, 2024-09-21T06:27:16.613 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000095260 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 200256 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-21T06:27:16.614 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:16.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:16.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:16.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:16.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148704, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:16.619 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T06:27:16.620 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 299, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 817914, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 299, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 299, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 817914, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:16.621 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:27:17.226 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T06:27:17.226 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T06:27:17.226 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T06:27:17.226 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T06:27:17.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T06:27:17.484 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:17.859 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:17 smithi031 ceph-mon[31237]: pgmap v810: 97 pgs: 24 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 61 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-09-21T06:27:17.983 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001448006, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000309932, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 296, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 289, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 860784, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 177792, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.057875273, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.026959457, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.039576198, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 289, 2024-09-21T06:27:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.025788788, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000089234 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 860784, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 177792 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 220, 2024-09-21T06:27:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 199456, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.022105215, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.018214952, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001931444, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 218, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.013030687, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000059773 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 199456 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 147920, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 27, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T06:27:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 294, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 816563, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 294, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 294, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 816563, 2024-09-21T06:27:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T06:27:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:17.996 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:27:18.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:17 smithi110 ceph-mon[34550]: pgmap v810: 97 pgs: 24 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 61 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-09-21T06:27:18.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:17 smithi107 ceph-mon[34380]: pgmap v810: 97 pgs: 24 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 61 active+clean; 4.0 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-09-21T06:27:18.676 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-21T06:27:18.676 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-21T06:27:18.676 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-21T06:27:18.676 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T06:27:18.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T06:27:18.936 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:19.434 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-21T06:27:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 303, 2024-09-21T06:27:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 296, 2024-09-21T06:27:19.434 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 863376, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 181488, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.056037865, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.023763739, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.039814162, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 296, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.024234701, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000081873 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.435 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 863376, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 181488 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 224, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 200160, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:19.436 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.021509476, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.016977060, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.001973241, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 222, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.011855159, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000053401 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:19.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 200160 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.000464456, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:19.438 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000364638, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:19.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:19.440 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:19.441 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:19.442 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148704, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 29, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2024-09-21T06:27:19.443 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:19.444 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 301, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 818174, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 301, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 301, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 818174, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2024-09-21T06:27:19.445 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2024-09-21T06:27:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:19.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-21T06:27:19.446 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-21T06:27:20.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:19 smithi110 ceph-mon[34550]: pgmap v811: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 KiB/s wr, 7 op/s 2024-09-21T06:27:20.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:19 smithi107 ceph-mon[34380]: pgmap v811: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 KiB/s wr, 7 op/s 2024-09-21T06:27:20.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:19 smithi031 ceph-mon[31237]: pgmap v811: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 KiB/s wr, 7 op/s 2024-09-21T06:27:20.174 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T06:27:20.175 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T06:27:20.175 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T06:27:20.175 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T06:27:20.175 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T06:27:20.433 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:27:21.135 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T06:27:21.135 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:21.135 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 19702, 2024-09-21T06:27:21.135 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 19704, 2024-09-21T06:27:21.135 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 9813623, 2024-09-21T06:27:21.135 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 34554085, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 22, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 1.497611155, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.692410236, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.291642391, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.387122257, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 19702, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.145103698, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000108877 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:21.136 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 13, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000001732, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000133 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 18844, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 17972, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 10380296, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 35197512, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 27, 2024-09-21T06:27:21.137 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 6, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 1.464075155, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.640540337, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.369367764, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.383468237, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 17972, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.523105481, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000140390 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:21.138 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1386, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000298353, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000215 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1407072, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 388944 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 33969, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 40635, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 13306140, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 30330925, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 357, 2024-09-21T06:27:21.139 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 2.714941756, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 1.402343243, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 1.329019235, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.213242688, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 40633, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 7.405120500, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000182244 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 44886, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.011273551, 2024-09-21T06:27:21.140 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000251 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 8941312, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 17263584 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 19188, 2024-09-21T06:27:21.141 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 28.968844401, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001509737 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 926, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.037605598, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000040610 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 12414, 2024-09-21T06:27:21.142 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 12414, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 12414, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 411.686802255, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.033163106 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-09-21T06:27:21.143 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 1715, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 9, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 1, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 2001, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 572, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 1429, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 1183, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 1054, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 1055, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 53488, 2024-09-21T06:27:21.144 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 51906, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 469, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 34648, 2024-09-21T06:27:21.145 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 1026, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 1728069300, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 34648, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 1194, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 34, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 1160, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 33454, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:27:21.146 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 8833, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 6319, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 2, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 11568, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 2461, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 219, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 1351 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-09-21T06:27:21.147 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 517, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 270, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 247, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:27:21.148 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 34, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 10, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "uninline_started": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "uninline_succeeded": 0, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "uninline_write_failed": 0 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.149 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 10389, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 10095, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 10095, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 294, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 388, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 84, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 82, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 82, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 1, 2024-09-21T06:27:21.150 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 2, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 56670619, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 58776598, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 0, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5333, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 101.339562765, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.019002355 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 0 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-09-21T06:27:21.151 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 1821, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 2124, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 303, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 109, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 122, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 13, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 2001, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 3010, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 1009, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 1055, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 1566, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 511, 2024-09-21T06:27:21.152 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 143704, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 367931392 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 13743, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 12414, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 528, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1297, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 5.329779819, 2024-09-21T06:27:21.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004109313 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 8430, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 400.847040558, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.047550064 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 247, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.507852284, 2024-09-21T06:27:21.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002056082 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 907, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 1.089246326, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001200933 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-21T06:27:21.155 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000459712, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000229856 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.157 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-09-21T06:27:21.157 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.157 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.157 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 16, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003332261, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000208266 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 37, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.216206373, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005843415 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.158 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 16, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.184467373, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.011529210 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 526, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.228632423, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000434662 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 136, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.203016310, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001492766 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 479, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 2.096119336, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.004376032 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.256797285, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.128398642 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 10, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.058330076, 2024-09-21T06:27:21.161 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.005833007 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.065182995, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.021727665 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-09-21T06:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.163 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 9, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.022698048, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.002522005 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 297, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.577641076, 2024-09-21T06:27:21.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001944919 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 3, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 1, 2024-09-21T06:27:21.165 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 2, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 1475, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 737, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 940, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:21.166 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:21.167 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:21.168 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:21.169 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:21.169 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:21.169 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:21.169 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-21T06:27:21.170 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 945995, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 1408, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 704, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 8, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8280, 2024-09-21T06:27:21.171 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 183, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 6387650, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 98921, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-09-21T06:27:21.172 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 45224, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 62933721, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 31, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 45224, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 45224, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 428.743137441, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.009480433 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 45224, 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 52177 2024-09-21T06:27:21.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 45224, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 34828, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 10396, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1595, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 1554, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 146, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 5446, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 325, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-09-21T06:27:21.174 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 5, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 827, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 34682, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 3108, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-09-21T06:27:21.175 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 2, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-09-21T06:27:21.176 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 97, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 22, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 12, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 12, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 4162, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 0, 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 325 2024-09-21T06:27:21.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 1, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 24, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 11, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 284, 2024-09-21T06:27:21.178 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 270, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 72511, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 25700869, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 72511, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 72511, 2024-09-21T06:27:21.179 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 25700869, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-21T06:27:21.180 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 45224, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 62962135, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 45224, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 62962135, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:21.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 45224, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 45224, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 45224, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 45224, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.182 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 270, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 28414, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 270, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 113, 2024-09-21T06:27:21.183 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 28414, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x5620bfcff520": { 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 599, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 10389, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 54582294, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:21.184 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 10389, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 5333, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 54581695, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "trackedop": { 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 3 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:21.185 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T06:27:22.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:21 smithi110 ceph-mon[34550]: pgmap v812: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-09-21T06:27:22.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:21 smithi107 ceph-mon[34380]: pgmap v812: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-09-21T06:27:22.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:21 smithi031 ceph-mon[31237]: pgmap v812: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 KiB/s wr, 7 op/s 2024-09-21T06:27:22.138 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T06:27:22.138 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-09-21T06:27:22.138 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-09-21T06:27:22.138 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T06:27:22.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T06:27:22.393 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:27:22.869 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.001047028, 2024-09-21T06:27:22.870 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000196942, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:22.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 6, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 225, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3264, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 200432, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:22.872 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.016574301, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011417154, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.003289574, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 224, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.015576400, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000069537 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:22.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 200432 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 298, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 293, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 860160, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 180400, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:22.874 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.041777534, 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.019349623, 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.030419706, 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 292, 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.012127780, 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000041533 2024-09-21T06:27:22.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 860160, 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 180400 2024-09-21T06:27:22.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:22.877 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:22.878 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:22.879 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149735, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:22.880 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-09-21T06:27:22.881 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:22.882 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 300, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 818044, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 300, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 300, 2024-09-21T06:27:22.883 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 818044, 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:22.884 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T06:27:23.691 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-21T06:27:23.692 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-21T06:27:23.692 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-21T06:27:23.692 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T06:27:23.692 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T06:27:23.941 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:27:24.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:23 smithi110 ceph-mon[34550]: pgmap v813: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 43 KiB/s wr, 8 op/s 2024-09-21T06:27:24.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:23 smithi110 ceph-mon[34550]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T06:27:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:23 smithi107 ceph-mon[34380]: pgmap v813: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 43 KiB/s wr, 8 op/s 2024-09-21T06:27:24.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:23 smithi107 ceph-mon[34380]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T06:27:24.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:23 smithi031 ceph-mon[31237]: pgmap v813: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 43 KiB/s wr, 8 op/s 2024-09-21T06:27:24.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:23 smithi031 ceph-mon[31237]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T06:27:24.449 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T06:27:24.449 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 103, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 100, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 187712, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 60160, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.019640004, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.005789387, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.014869446, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:24.450 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 100, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.025448893, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000254488 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 187712, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 60160 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:24.451 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 75, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 160, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 93392, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.008939101, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.003712787, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.004584724, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 73, 2024-09-21T06:27:24.452 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003269106, 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000044782 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:27:24.453 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 93392 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003956656, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.003858945, 2024-09-21T06:27:24.454 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:24.455 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:24.456 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:24.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:24.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:24.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:24.457 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:24.458 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-21T06:27:24.459 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148823, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 27, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:24.460 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:24.461 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 100, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 170796, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 100, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 100, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 170796, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:24.462 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:24.463 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:24.463 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:24.463 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:24.463 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T06:27:25.154 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-09-21T06:27:25.154 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-09-21T06:27:25.154 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-09-21T06:27:25.154 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T06:27:25.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T06:27:25.400 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:27:25.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:25 smithi107 ceph-mon[34380]: pgmap v815: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 6 op/s 2024-09-21T06:27:25.967 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-21T06:27:25.967 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:25.967 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 301, 2024-09-21T06:27:25.967 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 296, 2024-09-21T06:27:25.967 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 860976, 2024-09-21T06:27:25.967 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 182320, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.045590783, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.021197755, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.031968005, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 295, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.013020855, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000044138 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:25.968 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 860976, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 182320 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 6, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 228, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3264, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 200960, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:25.969 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.017894995, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.012429928, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.003419548, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 227, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.023918452, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000105367 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:25.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 200960 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.000352049, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000283977, 2024-09-21T06:27:25.971 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:25.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:25.973 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:25.974 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:25.975 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149775, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 31, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:25.976 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-09-21T06:27:25.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 303, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 818434, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 303, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 303, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 818434, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-09-21T06:27:25.978 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-09-21T06:27:25.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:25.979 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-09-21T06:27:25.979 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-21T06:27:26.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:25 smithi110 ceph-mon[34550]: pgmap v815: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 6 op/s 2024-09-21T06:27:26.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:25 smithi031 ceph-mon[31237]: pgmap v815: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 KiB/s wr, 6 op/s 2024-09-21T06:27:26.711 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-21T06:27:26.711 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-21T06:27:26.711 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-21T06:27:26.712 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T06:27:26.712 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T06:27:26.972 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:27.442 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T06:27:27.442 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:27.442 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.001144463, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000259601, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:27.443 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 6, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 225, 2024-09-21T06:27:27.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 3264, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 200432, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.017045861, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.011955363, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.003090835, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 224, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.021788325, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000097269 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 200432 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 297, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 292, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 859824, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 180240, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:27.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.045623541, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.021485580, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.032865615, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 291, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.016109385, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000055358 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:27.447 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 859824, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 180240 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:27.448 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:27.449 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:27.450 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148721, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 30, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:27.451 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T06:27:27.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 299, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 817850, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 299, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 299, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 817850, 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:27.453 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:27.454 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:27.454 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:27.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:27.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:27.454 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T06:27:28.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:27 smithi110 ceph-mon[34550]: pgmap v816: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 45 KiB/s wr, 6 op/s 2024-09-21T06:27:28.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:27 smithi107 ceph-mon[34380]: pgmap v816: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 45 KiB/s wr, 6 op/s 2024-09-21T06:27:28.136 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:27 smithi031 ceph-mon[31237]: pgmap v816: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 45 KiB/s wr, 6 op/s 2024-09-21T06:27:28.269 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-09-21T06:27:28.269 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-09-21T06:27:28.269 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-09-21T06:27:28.269 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T06:27:28.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T06:27:28.520 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:29.006 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.001092157, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:29.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000249342, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:29.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 303, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 296, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 863376, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 181488, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.047311999, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.020672051, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.036440050, 2024-09-21T06:27:29.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 296, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.017489149, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000059084 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 863376, 2024-09-21T06:27:29.010 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 181488 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 224, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 320, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 200160, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.016227949, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.012533849, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.001622476, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:29.011 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 222, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.016738568, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000075398 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 200160 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T06:27:29.012 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:29.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:29.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:29.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:29.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:29.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:29.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148761, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 31, 2024-09-21T06:27:29.016 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:29.017 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 301, 2024-09-21T06:27:29.018 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 818174, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 301, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 301, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 818174, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:29.019 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T06:27:29.710 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-09-21T06:27:29.710 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-09-21T06:27:29.710 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-09-21T06:27:29.710 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T06:27:29.710 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T06:27:29.955 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:30.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:29 smithi110 ceph-mon[34550]: pgmap v817: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 4 op/s 2024-09-21T06:27:30.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:29 smithi107 ceph-mon[34380]: pgmap v817: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 4 op/s 2024-09-21T06:27:30.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:29 smithi031 ceph-mon[31237]: pgmap v817: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 33 KiB/s wr, 4 op/s 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 359, 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 359, 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 8562763, 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 113354, 2024-09-21T06:27:30.464 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 7, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.042171861, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.015335475, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.013992682, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.009707716, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 358, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.025815468, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000072110 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:30.465 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 673, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 668, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1003141, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 296927, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-09-21T06:27:30.466 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.090770170, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.039506495, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.044878636, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.013259375, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 667, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.048547030, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000072784 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.467 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 878704, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 182704 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 506, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 727, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 12608170, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 409790, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-09-21T06:27:30.468 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.069198953, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.027470218, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.021727881, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.015201840, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 725, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.037967722, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000052369 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.469 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 249200 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2054, 2024-09-21T06:27:30.470 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.066441779, 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000032347 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-09-21T06:27:30.471 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-09-21T06:27:30.472 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 796, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 413, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 383, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 207, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-09-21T06:27:30.473 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 2, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-09-21T06:27:30.474 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-09-21T06:27:30.475 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 1026, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 1728083382, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 6, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:27:30.476 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:27:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-09-21T06:27:30.478 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-09-21T06:27:30.479 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:27:30.480 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_started": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_succeeded": 0, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "uninline_write_failed": 0 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.481 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 2595, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-09-21T06:27:30.482 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 24, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 56670619, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 58800515, 2024-09-21T06:27:30.483 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 58776598, 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 2595 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-09-21T06:27:30.484 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 378, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 477, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 99, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 65, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 70, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 5, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 796, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 796, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 0, 2024-09-21T06:27:30.485 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 120212, 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 322019328 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-09-21T06:27:30.486 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.487 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.488 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.490 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.491 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-09-21T06:27:30.492 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.492 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.492 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.492 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-09-21T06:27:30.492 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.493 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.494 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-09-21T06:27:30.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.496 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-09-21T06:27:30.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.500 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 2, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 0, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 0, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2230, 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-09-21T06:27:30.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:30.502 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:30.503 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:30.504 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 237590, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 690, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8160, 2024-09-21T06:27:30.505 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 180, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 1828026, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 49717, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-09-21T06:27:30.506 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 1219, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 26, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 4, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 1219, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1219, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 14.477760359, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.011876751 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-09-21T06:27:30.507 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1219, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1220 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 1219, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 1219, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 698, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 518, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-09-21T06:27:30.508 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 2, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-09-21T06:27:30.509 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-09-21T06:27:30.510 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 98, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 12, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 8, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 8, 2024-09-21T06:27:30.511 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-09-21T06:27:30.512 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 1534, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 22016951, 2024-09-21T06:27:30.513 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 1534, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1534, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 22016951, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.514 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1219, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 20949003, 2024-09-21T06:27:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1219, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 20949003, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T06:27:30.516 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1219, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1219, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1219, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1219, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.517 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T06:27:30.518 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x564d1b00f790": { 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-09-21T06:27:30.519 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "trackedop": { 2024-09-21T06:27:30.520 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-09-21T06:27:30.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:30.521 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T06:27:31.028 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:30 smithi110 ceph-mon[34550]: pgmap v818: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-21T06:27:31.039 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:30 smithi107 ceph-mon[34380]: pgmap v818: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-21T06:27:31.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:30 smithi031 ceph-mon[31237]: pgmap v818: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-09-21T06:27:31.233 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T06:27:31.233 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:27:31.233 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:27:31.234 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T06:27:31.234 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T06:27:31.478 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 0, 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2, 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 0, 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:32.004 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.000994931, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000193089, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:32.005 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 303, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 298, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 861520, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 183600, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-09-21T06:27:32.006 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.047290867, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.021033799, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.036117410, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 297, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.014601118, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000049162 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:32.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 861520, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 183600 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 6, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 229, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 3264, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 201136, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.016451042, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.011617820, 2024-09-21T06:27:32.008 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.003030609, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 228, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.014390882, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000063117 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T06:27:32.009 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 201136 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:27:32.010 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-09-21T06:27:32.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-09-21T06:27:32.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148704, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 29, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-09-21T06:27:32.013 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 968, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-09-21T06:27:32.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 305, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 818694, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 305, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 305, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 818694, 2024-09-21T06:27:32.015 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-09-21T06:27:32.016 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-09-21T06:27:32.016 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-09-21T06:27:32.016 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-09-21T06:27:32.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:32.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-21T06:27:32.016 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-21T06:27:32.879 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T06:27:32.879 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T06:27:32.880 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T06:27:32.880 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T06:27:32.889 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T06:27:32.889 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:27:32.889 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:27:32.912 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-21T06:27:32.912 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:27:32.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:27:33.285 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:33.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:33 smithi031 ceph-mon[31237]: pgmap v819: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-09-21T06:27:33.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:33 smithi110 ceph-mon[34550]: pgmap v819: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-09-21T06:27:33.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:33 smithi107 ceph-mon[34380]: pgmap v819: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 -- 172.21.15.31:0/2509860157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 msgr2=0x7f9080110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 --2- 172.21.15.31:0/2509860157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080110320 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9078009ca0 tx=0x7f907802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 -- 172.21.15.31:0/2509860157 shutdown_connections 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 --2- 172.21.15.31:0/2509860157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080110320 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 --2- 172.21.15.31:0/2509860157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 0x7f908010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 --2- 172.21.15.31:0/2509860157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 0x7f9080104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 -- 172.21.15.31:0/2509860157 >> 172.21.15.31:0/2509860157 conn(0x7f9080079740 msgr2=0x7f9080079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 -- 172.21.15.31:0/2509860157 shutdown_connections 2024-09-21T06:27:33.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 -- 172.21.15.31:0/2509860157 wait complete. 2024-09-21T06:27:33.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.743+0000 7f9086568640 1 Processor -- start 2024-09-21T06:27:33.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 -- start start 2024-09-21T06:27:33.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 0x7f90801027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 0x7f9080102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9080113a90 con 0x7f908010de50 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9080113910 con 0x7f90801052d0 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9086568640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9080113c10 con 0x7f90801048a0 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52918/0 (socket says 172.21.15.31:52918) 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 -- 172.21.15.31:0/2706649412 learned_addr learned my addr 172.21.15.31:0/2706649412 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f907f7fe640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 0x7f9080102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f907ffff640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 0x7f90801027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 -- 172.21.15.31:0/2706649412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 msgr2=0x7f90801027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 0x7f90801027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 -- 172.21.15.31:0/2706649412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 msgr2=0x7f9080102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.744+0000 7f9084b05640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 0x7f9080102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f9084b05640 1 -- 172.21.15.31:0/2706649412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90801a8720 con 0x7f908010de50 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f907f7fe640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 0x7f9080102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f907ffff640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 0x7f90801027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f9084b05640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080103230 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9078005cc0 tx=0x7f907802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078003bf0 con 0x7f908010de50 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90801a89b0 con 0x7f908010de50 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9078003d90 con 0x7f908010de50 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907802fbc0 con 0x7f908010de50 2024-09-21T06:27:33.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.745+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90801a8f40 con 0x7f908010de50 2024-09-21T06:27:33.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.746+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908006c1a0 con 0x7f908010de50 2024-09-21T06:27:33.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.748+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f907802fd60 con 0x7f908010de50 2024-09-21T06:27:33.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.748+0000 7f907d7fa640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90680b6680 0x7f90680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:33.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.749+0000 7f907ffff640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90680b6680 0x7f90680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:33.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.749+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8499+0+0 (secure 0 0 0) 0x7f9078108d40 con 0x7f908010de50 2024-09-21T06:27:33.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.749+0000 7f907ffff640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90680b6680 0x7f90680b8b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9070005fd0 tx=0x7f9070005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:33.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.753+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f90780cf2a0 con 0x7f908010de50 2024-09-21T06:27:33.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.951+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9080103ab0 con 0x7f908010de50 2024-09-21T06:27:33.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.952+0000 7f907d7fa640 1 -- 172.21.15.31:0/2706649412 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v98) ==== 81+0+877 (secure 0 0 0) 0x7f9080103ab0 con 0x7f908010de50 2024-09-21T06:27:33.954 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4284062429 2024-09-22T06:03:05.499388+0000 2024-09-21T06:27:33.954 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/10892239 2024-09-22T06:03:05.499388+0000 2024-09-21T06:27:33.954 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3238988869 2024-09-22T06:01:35.161130+0000 2024-09-21T06:27:33.954 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2647070256 2024-09-22T06:01:35.161130+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1169957326 2024-09-22T06:03:05.499388+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/733625975 2024-09-22T06:01:35.161130+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1111343468 2024-09-22T06:02:01.461252+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2445761929 2024-09-22T06:02:01.461252+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/465044057 2024-09-22T06:01:35.161130+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3238988869 2024-09-22T06:01:35.161130+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/65624344 2024-09-22T06:03:05.499388+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/979439078 2024-09-22T06:02:01.461252+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/836492518 2024-09-22T06:02:01.461252+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/979439078 2024-09-22T06:02:01.461252+0000 2024-09-21T06:27:33.955 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/65624344 2024-09-22T06:03:05.499388+0000 2024-09-21T06:27:33.956 INFO:teuthology.orchestra.run.smithi031.stderr:listed 15 entries 2024-09-21T06:27:33.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90680b6680 msgr2=0x7f90680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90680b6680 0x7f90680b8b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9070005fd0 tx=0x7f9070005850 comp rx=0 tx=0).stop 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 msgr2=0x7f9080103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080103230 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9078005cc0 tx=0x7f907802f380 comp rx=0 tx=0).stop 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 shutdown_connections 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f90680b6680 0x7f90680b8b90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f908010de50 0x7f9080103230 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90801052d0 0x7f9080102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 --2- 172.21.15.31:0/2706649412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90801048a0 0x7f90801027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 >> 172.21.15.31:0/2706649412 conn(0x7f9080079740 msgr2=0x7f9080107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 shutdown_connections 2024-09-21T06:27:33.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:33.954+0000 7f9086568640 1 -- 172.21.15.31:0/2706649412 wait complete. 2024-09-21T06:27:34.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:34 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2706649412' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:27:34.668 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:27:34.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:27:34.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /tmp/cephfs 2024-09-21T06:27:34.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:34 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2706649412' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:27:34.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:34 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2706649412' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:27:34.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:27:34.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:27:34.938 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-21T06:27:34.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:27:34.993 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-21T06:27:35.026 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-21T06:27:35.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:27:35.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-21T06:27:35.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:27:35.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-21T06:27:35.186 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T06:27:35.202 INFO:tasks.cephadm:Teardown begin 2024-09-21T06:27:35.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:27:35.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:27:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:35.587 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:35 smithi031 ceph-mon[31237]: pgmap v820: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.4 KiB/s wr, 2 op/s 2024-09-21T06:27:35.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:35 smithi110 ceph-mon[34550]: pgmap v820: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.4 KiB/s wr, 2 op/s 2024-09-21T06:27:35.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:35 smithi107 ceph-mon[34380]: pgmap v820: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.4 KiB/s wr, 2 op/s 2024-09-21T06:27:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 -- 172.21.15.31:0/2474700239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 msgr2=0x7f66f01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2474700239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f01087d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009900 tx=0x7f66e402a410 comp rx=0 tx=0).stop 2024-09-21T06:27:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 -- 172.21.15.31:0/2474700239 shutdown_connections 2024-09-21T06:27:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2474700239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 0x7f66f010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2474700239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f01087d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2474700239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 0x7f66f0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 -- 172.21.15.31:0/2474700239 >> 172.21.15.31:0/2474700239 conn(0x7f66f0079740 msgr2=0x7f66f0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 -- 172.21.15.31:0/2474700239 shutdown_connections 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.976+0000 7f66f58f9640 1 -- 172.21.15.31:0/2474700239 wait complete. 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 Processor -- start 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 -- start start 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 0x7f66f019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 0x7f66f019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01129d0 con 0x7f66f01083c0 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0112850 con 0x7f66f0102a80 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66f58f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0112b50 con 0x7f66f0108d10 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66ee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66ee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f019b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52926/0 (socket says 172.21.15.31:52926) 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66ee7fc640 1 -- 172.21.15.31:0/2344280985 learned_addr learned my addr 172.21.15.31:0/2344280985 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66eeffd640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 0x7f66f019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.977+0000 7f66ef7fe640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 0x7f66f019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ee7fc640 1 -- 172.21.15.31:0/2344280985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 msgr2=0x7f66f019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ee7fc640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 0x7f66f019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ee7fc640 1 -- 172.21.15.31:0/2344280985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 msgr2=0x7f66f019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ee7fc640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 0x7f66f019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ee7fc640 1 -- 172.21.15.31:0/2344280985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f01a6530 con 0x7f66f01083c0 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ef7fe640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 0x7f66f019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66eeffd640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 0x7f66f019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66ee7fc640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f019b7e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f66e4005ab0 tx=0x7f66e4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e403f070 con 0x7f66f01083c0 2024-09-21T06:27:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f01a67c0 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66e4031b50 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4035030 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.978+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f01a6ca0 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.979+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f0103d00 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.982+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f66e4042070 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.982+0000 7f66f48f7640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66d80b66b0 0x7f66d80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.983+0000 7f66eeffd640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66d80b66b0 0x7f66d80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.983+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8499+0+0 (secure 0 0 0) 0x7f66e4108690 con 0x7f66f01083c0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.983+0000 7f66eeffd640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66d80b66b0 0x7f66d80b8bc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f66f006c6a0 tx=0x7f66e0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:35.986+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66e40cebf0 con 0x7f66f01083c0 2024-09-21T06:27:36.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.211+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66f0102e90 con 0x7f66f01083c0 2024-09-21T06:27:36.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.212+0000 7f66f48f7640 1 -- 172.21.15.31:0/2344280985 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f66e40d2fd0 con 0x7f66f01083c0 2024-09-21T06:27:36.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:27:36.214 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:27:36.214 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66d80b66b0 msgr2=0x7f66d80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66d80b66b0 0x7f66d80b8bc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f66f006c6a0 tx=0x7f66e0005f90 comp rx=0 tx=0).stop 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 msgr2=0x7f66f019b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f019b7e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f66e4005ab0 tx=0x7f66e4003b90 comp rx=0 tx=0).stop 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 shutdown_connections 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66f0108d10 0x7f66f019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f66d80b66b0 0x7f66d80b8bc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66f01083c0 0x7f66f019b7e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 --2- 172.21.15.31:0/2344280985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66f0102a80 0x7f66f019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 >> 172.21.15.31:0/2344280985 conn(0x7f66f0079740 msgr2=0x7f66f01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 shutdown_connections 2024-09-21T06:27:36.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:36.214+0000 7f66f58f9640 1 -- 172.21.15.31:0/2344280985 wait complete. 2024-09-21T06:27:36.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:36 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/2344280985' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:36.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:36 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/2344280985' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:36.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:36 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/2344280985' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:37.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:27:37.270 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:37.580 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:37 smithi031 ceph-mon[31237]: pgmap v821: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-21T06:27:37.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 -- 172.21.15.31:0/3910158926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae01052a0 msgr2=0x7f2ae010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 --2- 172.21.15.31:0/3910158926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae01052a0 0x7f2ae010d930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009900 tx=0x7f2ac802a410 comp rx=0 tx=0).stop 2024-09-21T06:27:37.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 -- 172.21.15.31:0/3910158926 shutdown_connections 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 --2- 172.21.15.31:0/3910158926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ae010de70 0x7f2ae0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 --2- 172.21.15.31:0/3910158926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae01052a0 0x7f2ae010d930 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 --2- 172.21.15.31:0/3910158926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01048c0 0x7f2ae0104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.758+0000 7f2ae507e640 1 -- 172.21.15.31:0/3910158926 >> 172.21.15.31:0/3910158926 conn(0x7f2ae0079760 msgr2=0x7f2ae0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.759+0000 7f2ae507e640 1 -- 172.21.15.31:0/3910158926 shutdown_connections 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.759+0000 7f2ae507e640 1 -- 172.21.15.31:0/3910158926 wait complete. 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.759+0000 7f2ae507e640 1 Processor -- start 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.759+0000 7f2ae507e640 1 -- start start 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ae507e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ae01048c0 0x7f2ae019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ae507e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01052a0 0x7f2ae019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ae507e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 0x7f2ae019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ae507e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae0113ab0 con 0x7f2ae010de70 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ae507e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae0113930 con 0x7f2ae01052a0 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ae507e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae0113c30 con 0x7f2ae01048c0 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2adf59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 0x7f2ae019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2adf59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 0x7f2ae019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52950/0 (socket says 172.21.15.31:52950) 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2adf59e640 1 -- 172.21.15.31:0/196696192 learned_addr learned my addr 172.21.15.31:0/196696192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2aded9d640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ae01048c0 0x7f2ae019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.760+0000 7f2ad7fff640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01052a0 0x7f2ae019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ad7fff640 1 -- 172.21.15.31:0/196696192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ae01048c0 msgr2=0x7f2ae019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ad7fff640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ae01048c0 0x7f2ae019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ad7fff640 1 -- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 msgr2=0x7f2ae019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:37.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ad7fff640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 0x7f2ae019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ad7fff640 1 -- 172.21.15.31:0/196696192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae01a0430 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2adf59e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 0x7f2ae019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ad7fff640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01052a0 0x7f2ae019aa60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802a930 tx=0x7f2ac8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac803f070 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae01a06c0 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac8003ca0 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8035070 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.761+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae01a6e80 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.762+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae006c190 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.764+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2ac803b4f0 con 0x7f2ae01052a0 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.765+0000 7f2adcd99640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2acc0b6680 0x7f2acc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:37.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.765+0000 7f2aded9d640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2acc0b6680 0x7f2acc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:37.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.765+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8499+0+0 (secure 0 0 0) 0x7f2ac8108160 con 0x7f2ae01052a0 2024-09-21T06:27:37.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.765+0000 7f2aded9d640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2acc0b6680 0x7f2acc0b8b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ae006c890 tx=0x7f2ad000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:37.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.768+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2ac80d2a40 con 0x7f2ae01052a0 2024-09-21T06:27:37.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:37 smithi110 ceph-mon[34550]: pgmap v821: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-21T06:27:37.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:37 smithi107 ceph-mon[34380]: pgmap v821: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-21T06:27:37.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.997+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ae01a7050 con 0x7f2ae01052a0 2024-09-21T06:27:38.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:37.998+0000 7f2adcd99640 1 -- 172.21.15.31:0/196696192 <== mon.2 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11850 (secure 0 0 0) 0x7f2ac80d7980 con 0x7f2ae01052a0 2024-09-21T06:27:38.000 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:27:38.000 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-21T06:21:27:966805+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14766,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3018651170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3018651170},{"type":"v1","addr":"172.21.15.31:6835","nonce":3018651170}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/594088756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":594088756},{"type":"v1","addr":"172.21.15.31:6837","nonce":594088756}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14829,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3970721911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":3970721911},{"type":"v1","addr":"172.21.15.31:6839","nonce":3970721911}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14835,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/4177463092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":4177463092},{"type":"v1","addr":"172.21.15.31:6841","nonce":4177463092}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":17493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2641406273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2641406273},{"type":"v1","addr":"172.21.15.107:6837","nonce":2641406273}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":24613,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/128096132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":128096132},{"type":"v1","addr":"172.21.15.110:6833","nonce":128096132}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/4010348472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":4010348472},{"type":"v1","addr":"172.21.15.107:6833","nonce":4010348472}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/1733260219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":1733260219},{"type":"v1","addr":"172.21.15.110:6835","nonce":1733260219}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/969332587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":969332587},{"type":"v1","addr":"172.21.15.110:6837","nonce":969332587}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24662,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/306946935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":306946935},{"type":"v1","addr":"172.21.15.107:6835","nonce":306946935}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:09:36.903399+0000","modified":"2024-09-21T06:21:27.966782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24719},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6839/1863570857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1863570857},{"type":"v1","addr":"172.21.15.110:6839","nonce":1863570857}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.107:6839/648775801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":648775801},{"type":"v1","addr":"172.21.15.107:6839","nonce":648775801}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24719,"qdb_cluster":[24719]},"id":1}]} 2024-09-21T06:27:38.000 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2acc0b6680 msgr2=0x7f2acc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2acc0b6680 0x7f2acc0b8b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ae006c890 tx=0x7f2ad000b390 comp rx=0 tx=0).stop 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01052a0 msgr2=0x7f2ae019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01052a0 0x7f2ae019aa60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2ac802a930 tx=0x7f2ac8004270 comp rx=0 tx=0).stop 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 shutdown_connections 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f2acc0b6680 0x7f2acc0b8b90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ae010de70 0x7f2ae019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae01052a0 0x7f2ae019aa60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 --2- 172.21.15.31:0/196696192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ae01048c0 0x7f2ae019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 >> 172.21.15.31:0/196696192 conn(0x7f2ae0079760 msgr2=0x7f2ae0107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 shutdown_connections 2024-09-21T06:27:38.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:38.000+0000 7f2ae507e640 1 -- 172.21.15.31:0/196696192 wait complete. 2024-09-21T06:27:38.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:38 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/196696192' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:38.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:27:38.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:38 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/196696192' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:38.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:38 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/196696192' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:27:38.941 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:39.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 -- 172.21.15.31:0/3774489432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb81083c0 msgr2=0x7f1eb81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/3774489432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb81083c0 0x7f1eb81087d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009900 tx=0x7f1eac02a410 comp rx=0 tx=0).stop 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 -- 172.21.15.31:0/3774489432 shutdown_connections 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/3774489432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb8108d10 0x7f1eb810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/3774489432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb81083c0 0x7f1eb81087d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/3774489432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb8102a80 0x7f1eb8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 -- 172.21.15.31:0/3774489432 >> 172.21.15.31:0/3774489432 conn(0x7f1eb8079740 msgr2=0x7f1eb8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 -- 172.21.15.31:0/3774489432 shutdown_connections 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.476+0000 7f1ebc82d640 1 -- 172.21.15.31:0/3774489432 wait complete. 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.477+0000 7f1ebc82d640 1 Processor -- start 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.477+0000 7f1ebc82d640 1 -- start start 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1ebc82d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb8102a80 0x7f1eb819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1ebc82d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb81083c0 0x7f1eb819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1ebc82d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 0x7f1eb819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1ebc82d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8112980 con 0x7f1eb8108d10 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1ebc82d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8112800 con 0x7f1eb81083c0 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1ebc82d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8112b00 con 0x7f1eb8102a80 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 0x7f1eb819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb659c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb8102a80 0x7f1eb819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 0x7f1eb819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33570/0 (socket says 172.21.15.31:33570) 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 -- 172.21.15.31:0/737156795 learned_addr learned my addr 172.21.15.31:0/737156795 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb5d9b640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb81083c0 0x7f1eb819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 -- 172.21.15.31:0/737156795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb8102a80 msgr2=0x7f1eb819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb8102a80 0x7f1eb819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 -- 172.21.15.31:0/737156795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb81083c0 msgr2=0x7f1eb819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb81083c0 0x7f1eb819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 -- 172.21.15.31:0/737156795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb81a0470 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb5d9b640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb81083c0 0x7f1eb819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.478+0000 7f1eb6d9d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 0x7f1eb819fd40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cad0 tx=0x7f1eb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.479+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000ae20 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.479+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb0005020 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.479+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb81a0760 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.479+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000f6b0 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.479+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb81a6ec0 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.480+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb8103d00 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.481+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1eb00051d0 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.482+0000 7f1ea77fe640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ea00b6680 0x7f1ea00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.482+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8499+0+0 (secure 0 0 0) 0x7f1eb00e7aa0 con 0x7f1eb8108d10 2024-09-21T06:27:39.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.482+0000 7f1eb659c640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ea00b6680 0x7f1ea00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.483+0000 7f1eb659c640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ea00b6680 0x7f1ea00b8b90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800bbe0 tx=0x7f1ea800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:39.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.485+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1eb0014030 con 0x7f1eb8108d10 2024-09-21T06:27:39.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:39 smithi031 ceph-mon[31237]: pgmap v822: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-21T06:27:39.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.677+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eb8102e90 con 0x7f1eb8108d10 2024-09-21T06:27:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.678+0000 7f1ea77fe640 1 -- 172.21.15.31:0/737156795 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23180 (secure 0 0 0) 0x7f1eb8102e90 con 0x7f1eb8108d10 2024-09-21T06:27:39.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-21T06:27:39.681 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":98,"fsid":"c04f7042-77de-11ef-baf5-efdc52797490","created":"2024-09-21T06:01:12.820557+0000","modified":"2024-09-21T06:27:23.362438+0000","last_up_change":"2024-09-21T06:08:48.349434+0000","last_in_change":"2024-09-21T06:08:24.740546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:05:05.978676+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:09:33.023556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:09:34.842253+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"80675501-db5a-4326-bf65-f570e2ef7dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6803","nonce":215542336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6805","nonce":215542336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6809","nonce":215542336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":215542336},{"type":"v1","addr":"172.21.15.31:6807","nonce":215542336}]},"public_addr":"172.21.15.31:6803/215542336","cluster_addr":"172.21.15.31:6805/215542336","heartbeat_back_addr":"172.21.15.31:6809/215542336","heartbeat_front_addr":"172.21.15.31:6807/215542336","state":["exists","up"]},{"osd":1,"uuid":"b0e6db75-1424-4686-b6d1-f889a7cf426c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6801","nonce":2850742213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6803","nonce":2850742213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6807","nonce":2850742213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2850742213},{"type":"v1","addr":"172.21.15.107:6805","nonce":2850742213}]},"public_addr":"172.21.15.107:6801/2850742213","cluster_addr":"172.21.15.107:6803/2850742213","heartbeat_back_addr":"172.21.15.107:6807/2850742213","heartbeat_front_addr":"172.21.15.107:6805/2850742213","state":["exists","up"]},{"osd":2,"uuid":"c2cfbd3a-7af0-4a1b-b9dd-d1367c9e9612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6801","nonce":3771685673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6803","nonce":3771685673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6807","nonce":3771685673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3771685673},{"type":"v1","addr":"172.21.15.110:6805","nonce":3771685673}]},"public_addr":"172.21.15.110:6801/3771685673","cluster_addr":"172.21.15.110:6803/3771685673","heartbeat_back_addr":"172.21.15.110:6807/3771685673","heartbeat_front_addr":"172.21.15.110:6805/3771685673","state":["exists","up"]},{"osd":3,"uuid":"24b23ec1-a44d-4d63-9a6e-fab54498c7db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6811","nonce":1116793462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6813","nonce":1116793462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6817","nonce":1116793462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1116793462},{"type":"v1","addr":"172.21.15.31:6815","nonce":1116793462}]},"public_addr":"172.21.15.31:6811/1116793462","cluster_addr":"172.21.15.31:6813/1116793462","heartbeat_back_addr":"172.21.15.31:6817/1116793462","heartbeat_front_addr":"172.21.15.31:6815/1116793462","state":["exists","up"]},{"osd":4,"uuid":"a5c6b1c9-2f76-4407-a87a-abcddb48fba3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6809","nonce":888650447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6811","nonce":888650447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6815","nonce":888650447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":888650447},{"type":"v1","addr":"172.21.15.107:6813","nonce":888650447}]},"public_addr":"172.21.15.107:6809/888650447","cluster_addr":"172.21.15.107:6811/888650447","heartbeat_back_addr":"172.21.15.107:6815/888650447","heartbeat_front_addr":"172.21.15.107:6813/888650447","state":["exists","up"]},{"osd":5,"uuid":"44342172-be5e-4e39-ad8e-0aed23c0efb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6809","nonce":3518155164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6811","nonce":3518155164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6815","nonce":3518155164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3518155164},{"type":"v1","addr":"172.21.15.110:6813","nonce":3518155164}]},"public_addr":"172.21.15.110:6809/3518155164","cluster_addr":"172.21.15.110:6811/3518155164","heartbeat_back_addr":"172.21.15.110:6815/3518155164","heartbeat_front_addr":"172.21.15.110:6813/3518155164","state":["exists","up"]},{"osd":6,"uuid":"f9f50053-37df-4668-b5a6-c186af35a2b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6819","nonce":3870004119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6821","nonce":3870004119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6825","nonce":3870004119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3870004119},{"type":"v1","addr":"172.21.15.31:6823","nonce":3870004119}]},"public_addr":"172.21.15.31:6819/3870004119","cluster_addr":"172.21.15.31:6821/3870004119","heartbeat_back_addr":"172.21.15.31:6825/3870004119","heartbeat_front_addr":"172.21.15.31:6823/3870004119","state":["exists","up"]},{"osd":7,"uuid":"26b1487e-7c8b-4705-8418-c1687458e181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6817","nonce":1767013823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6819","nonce":1767013823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6823","nonce":1767013823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1767013823},{"type":"v1","addr":"172.21.15.107:6821","nonce":1767013823}]},"public_addr":"172.21.15.107:6817/1767013823","cluster_addr":"172.21.15.107:6819/1767013823","heartbeat_back_addr":"172.21.15.107:6823/1767013823","heartbeat_front_addr":"172.21.15.107:6821/1767013823","state":["exists","up"]},{"osd":8,"uuid":"ee40c435-fe2e-4026-bfbc-2dfed559f5d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6817","nonce":4244140994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6819","nonce":4244140994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6823","nonce":4244140994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":4244140994},{"type":"v1","addr":"172.21.15.110:6821","nonce":4244140994}]},"public_addr":"172.21.15.110:6817/4244140994","cluster_addr":"172.21.15.110:6819/4244140994","heartbeat_back_addr":"172.21.15.110:6823/4244140994","heartbeat_front_addr":"172.21.15.110:6821/4244140994","state":["exists","up"]},{"osd":9,"uuid":"0d5a01fe-27f8-4e91-86b1-031461e41633","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6827","nonce":1883621951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6829","nonce":1883621951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6833","nonce":1883621951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1883621951},{"type":"v1","addr":"172.21.15.31:6831","nonce":1883621951}]},"public_addr":"172.21.15.31:6827/1883621951","cluster_addr":"172.21.15.31:6829/1883621951","heartbeat_back_addr":"172.21.15.31:6833/1883621951","heartbeat_front_addr":"172.21.15.31:6831/1883621951","state":["exists","up"]},{"osd":10,"uuid":"0c49045f-5d86-4a85-9c2b-22e12fb42351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6825","nonce":3727307895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6827","nonce":3727307895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6831","nonce":3727307895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3727307895},{"type":"v1","addr":"172.21.15.107:6829","nonce":3727307895}]},"public_addr":"172.21.15.107:6825/3727307895","cluster_addr":"172.21.15.107:6827/3727307895","heartbeat_back_addr":"172.21.15.107:6831/3727307895","heartbeat_front_addr":"172.21.15.107:6829/3727307895","state":["exists","up"]},{"osd":11,"uuid":"3922d4ae-a015-4a0e-92e8-2eb5daa7afe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6825","nonce":2652663870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6827","nonce":2652663870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6831","nonce":2652663870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":2652663870},{"type":"v1","addr":"172.21.15.110:6829","nonce":2652663870}]},"public_addr":"172.21.15.110:6825/2652663870","cluster_addr":"172.21.15.110:6827/2652663870","heartbeat_back_addr":"172.21.15.110:6831/2652663870","heartbeat_front_addr":"172.21.15.110:6829/2652663870","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:12.897179+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:04:37.640220+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:02.840323+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:29.083773+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:05:52.913613+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:17.860439+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:06:43.753250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:08.798072+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:32.009303+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:07:57.507228+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:21.575195+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:08:45.850125+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/4284062429":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/10892239":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/2647070256":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1169957326":"2024-09-22T06:03:05.499388+0000","172.21.15.31:0/733625975":"2024-09-22T06:01:35.161130+0000","172.21.15.31:0/1111343468":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/2445761929":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/465044057":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/3238988869":"2024-09-22T06:01:35.161130+0000","172.21.15.31:6801/65624344":"2024-09-22T06:03:05.499388+0000","172.21.15.31:6800/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:0/836492518":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6801/979439078":"2024-09-22T06:02:01.461252+0000","172.21.15.31:6800/65624344":"2024-09-22T06:03:05.499388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ea00b6680 msgr2=0x7f1ea00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ea00b6680 0x7f1ea00b8b90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800bbe0 tx=0x7f1ea800b390 comp rx=0 tx=0).stop 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 msgr2=0x7f1eb819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 0x7f1eb819fd40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000cad0 tx=0x7f1eb000cf90 comp rx=0 tx=0).stop 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 shutdown_connections 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1ea00b6680 0x7f1ea00b8b90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1eb8108d10 0x7f1eb819fd40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1eb81083c0 0x7f1eb819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 --2- 172.21.15.31:0/737156795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1eb8102a80 0x7f1eb819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:39.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 >> 172.21.15.31:0/737156795 conn(0x7f1eb8079740 msgr2=0x7f1eb806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:39.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 shutdown_connections 2024-09-21T06:27:39.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:39.681+0000 7f1ebc82d640 1 -- 172.21.15.31:0/737156795 wait complete. 2024-09-21T06:27:39.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:39 smithi110 ceph-mon[34550]: pgmap v822: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-21T06:27:39.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:39 smithi107 ceph-mon[34380]: pgmap v822: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-09-21T06:27:40.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T06:27:40.557 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:40.666 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:40 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/737156795' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:27:40.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:40 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/737156795' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:27:40.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:40 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/737156795' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:27:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.967+0000 7fec912ea640 1 -- 172.21.15.31:0/1296326241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 msgr2=0x7fec8c1125d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.967+0000 7fec912ea640 1 --2- 172.21.15.31:0/1296326241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1125d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec84009ca0 tx=0x7fec8402a950 comp rx=0 tx=0).stop 2024-09-21T06:27:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 -- 172.21.15.31:0/1296326241 shutdown_connections 2024-09-21T06:27:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 --2- 172.21.15.31:0/1296326241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1125d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 --2- 172.21.15.31:0/1296326241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c102f50 0x7fec8c10fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 --2- 172.21.15.31:0/1296326241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c102600 0x7fec8c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 -- 172.21.15.31:0/1296326241 >> 172.21.15.31:0/1296326241 conn(0x7fec8c079750 msgr2=0x7fec8c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 -- 172.21.15.31:0/1296326241 shutdown_connections 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 -- 172.21.15.31:0/1296326241 wait complete. 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.968+0000 7fec912ea640 1 Processor -- start 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 -- start start 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c102600 0x7fec8c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c102f50 0x7fec8c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c115ca0 con 0x7fec8c1101b0 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c115b20 con 0x7fec8c102f50 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec912ea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c115e20 con 0x7fec8c102600 2024-09-21T06:27:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec8b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec8b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33596/0 (socket says 172.21.15.31:33596) 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec8b7fe640 1 -- 172.21.15.31:0/1479665098 learned_addr learned my addr 172.21.15.31:0/1479665098 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec8affd640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c102600 0x7fec8c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.969+0000 7fec827fc640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c102f50 0x7fec8c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec8b7fe640 1 -- 172.21.15.31:0/1479665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c102600 msgr2=0x7fec8c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec8b7fe640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c102600 0x7fec8c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec8b7fe640 1 -- 172.21.15.31:0/1479665098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c102f50 msgr2=0x7fec8c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec8b7fe640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c102f50 0x7fec8c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec8b7fe640 1 -- 172.21.15.31:0/1479665098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c1a25e0 con 0x7fec8c1101b0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec8b7fe640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1a1eb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fec8402ae70 tx=0x7fec8402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84034040 con 0x7fec8c1101b0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec840308f0 con 0x7fec8c1101b0 2024-09-21T06:27:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec840047f0 con 0x7fec8c1101b0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c1a2870 con 0x7fec8c1101b0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.970+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c1a9030 con 0x7fec8c1101b0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.971+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8c06c280 con 0x7fec8c1101b0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.973+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fec84037020 con 0x7fec8c1101b0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.974+0000 7fec88ff9640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec780b6680 0x7fec780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.974+0000 7fec8affd640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec780b6680 0x7fec780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.974+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8499+0+0 (secure 0 0 0) 0x7fec84108700 con 0x7fec8c1101b0 2024-09-21T06:27:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.974+0000 7fec8affd640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec780b6680 0x7fec780b8b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00bbe0 tx=0x7fec7c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:40.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:40.978+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec84083710 con 0x7fec8c1101b0 2024-09-21T06:27:41.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.205+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fec8c102a10 con 0x7fec8c1101b0 2024-09-21T06:27:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.207+0000 7fec88ff9640 1 -- 172.21.15.31:0/1479665098 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7fec8c102a10 con 0x7fec8c1101b0 2024-09-21T06:27:41.210 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:27:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec780b6680 msgr2=0x7fec780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec780b6680 0x7fec780b8b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00bbe0 tx=0x7fec7c00b390 comp rx=0 tx=0).stop 2024-09-21T06:27:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 msgr2=0x7fec8c1a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1a1eb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fec8402ae70 tx=0x7fec8402f5d0 comp rx=0 tx=0).stop 2024-09-21T06:27:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 shutdown_connections 2024-09-21T06:27:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7fec780b6680 0x7fec780b8b90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec8c1101b0 0x7fec8c1a1eb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c102f50 0x7fec8c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 --2- 172.21.15.31:0/1479665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c102600 0x7fec8c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 >> 172.21.15.31:0/1479665098 conn(0x7fec8c079750 msgr2=0x7fec8c109de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 shutdown_connections 2024-09-21T06:27:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:41.210+0000 7fec912ea640 1 -- 172.21.15.31:0/1479665098 wait complete. 2024-09-21T06:27:41.225 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a[31233]: 2024-09-21T06:27:41.206+0000 7fe9c7fdb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: pgmap v823: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1479665098' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1479665098' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T06:27:41.505 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:41 smithi031 ceph-mon[31237]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: pgmap v823: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1479665098' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1479665098' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T06:27:41.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mon[34550]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T06:27:41.529 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i[64199]: ignoring --setuser ceph since I am not root 2024-09-21T06:27:41.529 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i[64199]: ignoring --setgroup ceph since I am not root 2024-09-21T06:27:41.529 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-mds[64203]: -- 172.21.15.110:0/2432253226 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56295d8e1c20 con 0x562961a89600 2024-09-21T06:27:41.529 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:27:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i[64199]: starting mds.i at 2024-09-21T06:27:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: pgmap v823: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 2.1 KiB/s wr, 2 op/s 2024-09-21T06:27:41.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1479665098' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1479665098' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mon[34380]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: ignoring --setuser ceph since I am not root 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: ignoring --setgroup ceph since I am not root 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-mds[63810]: -- 172.21.15.107:0/2697907472 <== mon.2 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558273ebd860 con 0x558278065180 2024-09-21T06:27:41.540 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:27:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: starting mds.b at 2024-09-21T06:27:41.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:27:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 -- 172.21.15.31:0/3459419174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 msgr2=0x7f752c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 --2- 172.21.15.31:0/3459419174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c103f70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f751c009900 tx=0x7f751c02a410 comp rx=0 tx=0).stop 2024-09-21T06:27:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 -- 172.21.15.31:0/3459419174 shutdown_connections 2024-09-21T06:27:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 --2- 172.21.15.31:0/3459419174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f752c109df0 0x7f752c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:42.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 --2- 172.21.15.31:0/3459419174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 0x7f752c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 --2- 172.21.15.31:0/3459419174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c103f70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 -- 172.21.15.31:0/3459419174 >> 172.21.15.31:0/3459419174 conn(0x7f752c07b850 msgr2=0x7f752c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 -- 172.21.15.31:0/3459419174 shutdown_connections 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.659+0000 7f75309de640 1 -- 172.21.15.31:0/3459419174 wait complete. 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 Processor -- start 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 -- start start 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 0x7f752c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f752c109df0 0x7f752c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f752c113b70 con 0x7f752c103b60 2024-09-21T06:27:42.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1139f0 con 0x7f752c1094a0 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f75309de640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f752c113cf0 con 0x7f752c109df0 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f752a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.660+0000 7f752a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33612/0 (socket says 172.21.15.31:33612) 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 -- 172.21.15.31:0/1121477674 learned_addr learned my addr 172.21.15.31:0/1121477674 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f7523fff640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 0x7f752c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752ad9d640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f752c109df0 0x7f752c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 -- 172.21.15.31:0/1121477674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f752c109df0 msgr2=0x7f752c19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f752c109df0 0x7f752c19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 -- 172.21.15.31:0/1121477674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 msgr2=0x7f752c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 0x7f752c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:42.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 -- 172.21.15.31:0/1121477674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c1a0530 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f7523fff640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 0x7f752c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f752a59c640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c19a5c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f751c006010 tx=0x7f751c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c03f030 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f751c003800 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c1a07c0 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c035070 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.661+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c1a6f80 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.663+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752c103ff0 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.665+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f751c0039b0 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.666+0000 7f75237fe640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75180b6680 0x7f75180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.666+0000 7f7523fff640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75180b6680 0x7f75180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.666+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8541+0+0 (secure 0 0 0) 0x7f751c1083a0 con 0x7f752c103b60 2024-09-21T06:27:42.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.667+0000 7f7523fff640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75180b6680 0x7f75180b8b90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f752c106450 tx=0x7f7514008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:42.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.669+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f751c0ce880 con 0x7f752c103b60 2024-09-21T06:27:42.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:42.897+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f752c02e640 con 0x7f752c103b60 2024-09-21T06:27:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.401+0000 7f75237fe640 1 -- 172.21.15.31:0/1121477674 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f751c0d2c60 con 0x7f752c103b60 2024-09-21T06:27:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75180b6680 msgr2=0x7f75180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75180b6680 0x7f75180b8b90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f752c106450 tx=0x7f7514008040 comp rx=0 tx=0).stop 2024-09-21T06:27:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 msgr2=0x7f752c19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c19a5c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f751c006010 tx=0x7f751c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:27:43.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 shutdown_connections 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f752c109df0 0x7f752c19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f752c1094a0 0x7f752c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f75180b6680 0x7f75180b8b90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 --2- 172.21.15.31:0/1121477674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f752c103b60 0x7f752c19a5c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 >> 172.21.15.31:0/1121477674 conn(0x7f752c07b850 msgr2=0x7f752c1075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 shutdown_connections 2024-09-21T06:27:43.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:43.404+0000 7f75309de640 1 -- 172.21.15.31:0/1121477674 wait complete. 2024-09-21T06:27:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: pgmap v825: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 2 op/s 2024-09-21T06:27:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.110:6838/2582138021,v1:172.21.15.110:6839/2582138021] up:boot 2024-09-21T06:27:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: mds.? [v2:172.21.15.107:6838/3314130834,v1:172.21.15.107:6839/3314130834] up:boot 2024-09-21T06:27:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T06:27:43.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:27:43.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:27:43.529 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:43 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1121477674' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: pgmap v825: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 2 op/s 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.110:6838/2582138021,v1:172.21.15.110:6839/2582138021] up:boot 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: mds.? [v2:172.21.15.107:6838/3314130834,v1:172.21.15.107:6839/3314130834] up:boot 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:27:43.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:43 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1121477674' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:27:43.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: pgmap v825: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 2 op/s 2024-09-21T06:27:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.110:6838/2582138021,v1:172.21.15.110:6839/2582138021] up:boot 2024-09-21T06:27:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: mds.? [v2:172.21.15.107:6838/3314130834,v1:172.21.15.107:6839/3314130834] up:boot 2024-09-21T06:27:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T06:27:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:27:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: from='mgr.14226 172.21.15.31:0/387202360' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:27:43.636 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:43 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1121477674' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:27:44.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:27:44.314 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:44.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:44 smithi031 ceph-mon[31237]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:27:44.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:44 smithi031 ceph-mon[31237]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:27:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:44 smithi031 ceph-mon[31237]: Cluster is now healthy 2024-09-21T06:27:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:44 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1121477674' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:27:44.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:44 smithi031 ceph-mon[31237]: fsmap 12 up:standby 2024-09-21T06:27:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:44 smithi110 ceph-mon[34550]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:27:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:44 smithi110 ceph-mon[34550]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:27:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:44 smithi110 ceph-mon[34550]: Cluster is now healthy 2024-09-21T06:27:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:44 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1121477674' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:27:44.528 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:44 smithi110 ceph-mon[34550]: fsmap 12 up:standby 2024-09-21T06:27:44.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:44 smithi107 ceph-mon[34380]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:27:44.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:44 smithi107 ceph-mon[34380]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:27:44.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:44 smithi107 ceph-mon[34380]: Cluster is now healthy 2024-09-21T06:27:44.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:44 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1121477674' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:27:44.539 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:44 smithi107 ceph-mon[34380]: fsmap 12 up:standby 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 -- 172.21.15.31:0/3966971764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 msgr2=0x7f342c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 --2- 172.21.15.31:0/3966971764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c110620 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3424009ca0 tx=0x7f342402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 -- 172.21.15.31:0/3966971764 shutdown_connections 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 --2- 172.21.15.31:0/3966971764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c110620 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 --2- 172.21.15.31:0/3966971764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c1094a0 0x7f342c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 --2- 172.21.15.31:0/3966971764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c103b60 0x7f342c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:44.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 -- 172.21.15.31:0/3966971764 >> 172.21.15.31:0/3966971764 conn(0x7f342c07b850 msgr2=0x7f342c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 -- 172.21.15.31:0/3966971764 shutdown_connections 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.831+0000 7f3430b0c640 1 -- 172.21.15.31:0/3966971764 wait complete. 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.832+0000 7f3430b0c640 1 Processor -- start 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.832+0000 7f3430b0c640 1 -- start start 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3430b0c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c103b60 0x7f342c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3430b0c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c1094a0 0x7f342c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3430b0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3430b0c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113ab0 con 0x7f342c109df0 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3430b0c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113930 con 0x7f342c103b60 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3430b0c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f342c113c30 con 0x7f342c1094a0 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:44.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c1a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33634/0 (socket says 172.21.15.31:33634) 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f3429d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c1094a0 0x7f342c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 -- 172.21.15.31:0/1100496041 learned_addr learned my addr 172.21.15.31:0/1100496041 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342a59c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c103b60 0x7f342c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 -- 172.21.15.31:0/1100496041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c1094a0 msgr2=0x7f342c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c1094a0 0x7f342c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 -- 172.21.15.31:0/1100496041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c103b60 msgr2=0x7f342c19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c103b60 0x7f342c19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 -- 172.21.15.31:0/1100496041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1a2570 con 0x7f342c109df0 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.833+0000 7f342ad9d640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c1a1f00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3424005cc0 tx=0x7f342402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.834+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424003cf0 con 0x7f342c109df0 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.834+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34240044f0 con 0x7f342c109df0 2024-09-21T06:27:44.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.834+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a2800 con 0x7f342c109df0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.834+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342402fc30 con 0x7f342c109df0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.834+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a9080 con 0x7f342c109df0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.835+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c103f70 con 0x7f342c109df0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.836+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f34240046a0 con 0x7f342c109df0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.837+0000 7f34237fe640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b6730 0x7f34180b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.837+0000 7f342a59c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b6730 0x7f34180b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.838+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8541+0+0 (secure 0 0 0) 0x7f3424108c70 con 0x7f342c109df0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.838+0000 7f342a59c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b6730 0x7f34180b8c40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f34140058c0 tx=0x7f341400a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:44.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:44.840+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f34240cf1b0 con 0x7f342c109df0 2024-09-21T06:27:45.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.032+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f342c19d7f0 con 0x7f342c109df0 2024-09-21T06:27:45.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.405+0000 7f34237fe640 1 -- 172.21.15.31:0/1100496041 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f34240ca070 con 0x7f342c109df0 2024-09-21T06:27:45.407 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:27:45.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.408+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b6730 msgr2=0x7f34180b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:45.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.408+0000 7f3430b0c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b6730 0x7f34180b8c40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f34140058c0 tx=0x7f341400a250 comp rx=0 tx=0).stop 2024-09-21T06:27:45.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.408+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 msgr2=0x7f342c1a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:45.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.408+0000 7f3430b0c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c1a1f00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3424005cc0 tx=0x7f342402f470 comp rx=0 tx=0).stop 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 shutdown_connections 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f34180b6730 0x7f34180b8c40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f342c109df0 0x7f342c1a1f00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f342c1094a0 0x7f342c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 --2- 172.21.15.31:0/1100496041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f342c103b60 0x7f342c19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 >> 172.21.15.31:0/1100496041 conn(0x7f342c07b850 msgr2=0x7f342c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:45.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 shutdown_connections 2024-09-21T06:27:45.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:45.409+0000 7f3430b0c640 1 -- 172.21.15.31:0/1100496041 wait complete. 2024-09-21T06:27:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:45 smithi110 ceph-mon[34550]: pgmap v826: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 2 op/s 2024-09-21T06:27:45.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:45 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1100496041' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:27:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:45 smithi107 ceph-mon[34380]: pgmap v826: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 2 op/s 2024-09-21T06:27:45.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:45 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1100496041' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:27:45.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:45 smithi031 ceph-mon[31237]: pgmap v826: 97 pgs: 97 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 77 KiB/s wr, 2 op/s 2024-09-21T06:27:45.798 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:45 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1100496041' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:27:46.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid c04f7042-77de-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:27:46.448 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:27:46.674 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:46 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1100496041' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:27:46.674 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:46 smithi031 ceph-mon[31237]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T06:27:46.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:46 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1100496041' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:27:46.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:46 smithi110 ceph-mon[34550]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T06:27:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:46 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1100496041' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:27:46.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:46 smithi107 ceph-mon[34380]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T06:27:46.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 -- 172.21.15.31:0/3271934875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c102ae0 msgr2=0x7f1c5c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:46.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 --2- 172.21.15.31:0/3271934875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c102ae0 0x7f1c5c102ef0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009900 tx=0x7f1c4c02a410 comp rx=0 tx=0).stop 2024-09-21T06:27:46.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 -- 172.21.15.31:0/3271934875 shutdown_connections 2024-09-21T06:27:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 --2- 172.21.15.31:0/3271934875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c5c108d20 0x7f1c5c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 --2- 172.21.15.31:0/3271934875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 --2- 172.21.15.31:0/3271934875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c102ae0 0x7f1c5c102ef0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 -- 172.21.15.31:0/3271934875 >> 172.21.15.31:0/3271934875 conn(0x7f1c5c0797b0 msgr2=0x7f1c5c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 -- 172.21.15.31:0/3271934875 shutdown_connections 2024-09-21T06:27:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.889+0000 7f1c629e2640 1 -- 172.21.15.31:0/3271934875 wait complete. 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.890+0000 7f1c629e2640 1 Processor -- start 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.890+0000 7f1c629e2640 1 -- start start 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c629e2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c5c102ae0 0x7f1c5c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c629e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c629e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c5c108d20 0x7f1c5c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c629e2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c113b00 con 0x7f1c5c1083d0 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c629e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c113980 con 0x7f1c5c108d20 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c629e2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c113c80 con 0x7f1c5c102ae0 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33646/0 (socket says 172.21.15.31:33646) 2024-09-21T06:27:46.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 -- 172.21.15.31:0/1963508995 learned_addr learned my addr 172.21.15.31:0/1963508995 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5bfff640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c5c102ae0 0x7f1c5c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c60f7f640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c5c108d20 0x7f1c5c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 -- 172.21.15.31:0/1963508995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c5c102ae0 msgr2=0x7f1c5c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c5c102ae0 0x7f1c5c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 -- 172.21.15.31:0/1963508995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c5c108d20 msgr2=0x7f1c5c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c5c108d20 0x7f1c5c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.891+0000 7f1c5b7fe640 1 -- 172.21.15.31:0/1963508995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c1a03e0 con 0x7f1c5c1083d0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.892+0000 7f1c5b7fe640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c19aa10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c970 tx=0x7f1c5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.892+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50004220 con 0x7f1c5c1083d0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.892+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c1a06d0 con 0x7f1c5c1083d0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.892+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c500043c0 con 0x7f1c5c1083d0 2024-09-21T06:27:46.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.892+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50004a00 con 0x7f1c5c1083d0 2024-09-21T06:27:46.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.892+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c1a6e30 con 0x7f1c5c1083d0 2024-09-21T06:27:46.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.893+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c102ef0 con 0x7f1c5c1083d0 2024-09-21T06:27:46.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.894+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1c500079b0 con 0x7f1c5c1083d0 2024-09-21T06:27:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.895+0000 7f1c597fa640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c440b6680 0x7f1c440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:27:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.895+0000 7f1c5bfff640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c440b6680 0x7f1c440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:27:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.895+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8016+0+0 (secure 0 0 0) 0x7f1c500e7560 con 0x7f1c5c1083d0 2024-09-21T06:27:46.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.896+0000 7f1c5bfff640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c440b6680 0x7f1c440b8b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c031440 tx=0x7f1c4c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:27:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:46.898+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c50014030 con 0x7f1c5c1083d0 2024-09-21T06:27:47.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.089+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1c5c19b5a0 con 0x7f1c5c1083d0 2024-09-21T06:27:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.420+0000 7f1c597fa640 1 -- 172.21.15.31:0/1963508995 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f1c5c19b5a0 con 0x7f1c5c1083d0 2024-09-21T06:27:47.422 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2024-09-21T06:27:47.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.424+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c440b6680 msgr2=0x7f1c440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:47.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.424+0000 7f1c629e2640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c440b6680 0x7f1c440b8b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c031440 tx=0x7f1c4c009560 comp rx=0 tx=0).stop 2024-09-21T06:27:47.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.424+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 msgr2=0x7f1c5c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:27:47.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.424+0000 7f1c629e2640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c19aa10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c970 tx=0x7f1c5000ce30 comp rx=0 tx=0).stop 2024-09-21T06:27:47.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 shutdown_connections 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c5c108d20 0x7f1c5c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:6800/2760018577,v1:172.21.15.31:6801/2760018577] conn(0x7f1c440b6680 0x7f1c440b8b90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c5c1083d0 0x7f1c5c19aa10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 --2- 172.21.15.31:0/1963508995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c5c102ae0 0x7f1c5c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 >> 172.21.15.31:0/1963508995 conn(0x7f1c5c0797b0 msgr2=0x7f1c5c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 shutdown_connections 2024-09-21T06:27:47.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-21T06:27:47.427+0000 7f1c629e2640 1 -- 172.21.15.31:0/1963508995 wait complete. 2024-09-21T06:27:47.690 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:47 smithi107 ceph-mon[34380]: pgmap v828: 65 pgs: 65 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:27:47.690 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:47 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1963508995' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:27:47.695 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:47 smithi110 ceph-mon[34550]: pgmap v828: 65 pgs: 65 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:27:47.695 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:47 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1963508995' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:27:47.783 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:47 smithi031 ceph-mon[31237]: pgmap v828: 65 pgs: 65 active+clean; 3.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:27:47.784 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:47 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1963508995' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:27:48.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:48.240 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:48.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:27:48.307 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T06:27:48.307 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:27:48.322 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:27:48.337 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:27:48.366 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T06:27:48.367 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T06:27:48.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:27:48.442 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:48 smithi031 systemd[1]: Stopping Ceph mon.a for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:48.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:48 smithi110 ceph-mon[34550]: from='client.? 172.21.15.31:0/1963508995' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:27:48.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:48 smithi110 ceph-mon[34550]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T06:27:48.788 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:48 smithi031 ceph-mon[31237]: from='client.? 172.21.15.31:0/1963508995' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:27:48.788 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:48 smithi031 ceph-mon[31237]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T06:27:48.789 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:48 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a[31233]: 2024-09-21T06:27:48.686+0000 7fe9cd7e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:48.789 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:48 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a[31233]: 2024-09-21T06:27:48.686+0000 7fe9cd7e6640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T06:27:48.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:48 smithi107 ceph-mon[34380]: from='client.? 172.21.15.31:0/1963508995' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:27:48.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:48 smithi107 ceph-mon[34380]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T06:27:49.135 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:48 smithi031 podman[112530]: 2024-09-21 06:27:48.83889906 +0000 UTC m=+0.271195829 container died 8fe9989c0afa1aefd1e7d292dd85c81673a3c3bd31bca6addb8dda6d7ef9a043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T06:27:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:49 smithi031 podman[112530]: 2024-09-21 06:27:49.395581913 +0000 UTC m=+0.827878668 container cleanup 8fe9989c0afa1aefd1e7d292dd85c81673a3c3bd31bca6addb8dda6d7ef9a043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T06:27:49.635 INFO:journalctl@ceph.mon.a.smithi031.stdout:Sep 21 06:27:49 smithi031 bash[112530]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:27:49.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T06:27:49.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:49.883 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T06:27:49.884 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T06:27:49.884 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.b 2024-09-21T06:27:50.202 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:49 smithi107 systemd[1]: Stopping Ceph mon.b for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:50.536 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:50 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-b[34376]: 2024-09-21T06:27:50.200+0000 7fd9599ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:50.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:50 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-b[34376]: 2024-09-21T06:27:50.200+0000 7fd9599ae640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:27:50.537 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:50 smithi107 podman[67776]: 2024-09-21 06:27:50.251351572 +0000 UTC m=+0.179130519 container died 651178a5eb5a841c0c20cbf5b0b0237321a8ccf46a6283b1caf52b72097f2fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-b, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T06:27:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:50 smithi107 podman[67776]: 2024-09-21 06:27:50.65927268 +0000 UTC m=+0.587051616 container cleanup 651178a5eb5a841c0c20cbf5b0b0237321a8ccf46a6283b1caf52b72097f2fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-b, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T06:27:50.789 INFO:journalctl@ceph.mon.b.smithi107.stdout:Sep 21 06:27:50 smithi107 bash[67776]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-b 2024-09-21T06:27:51.085 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T06:27:51.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:51.132 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T06:27:51.132 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T06:27:51.132 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.c 2024-09-21T06:27:51.452 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:51 smithi110 systemd[1]: Stopping Ceph mon.c for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:51.778 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c[34533]: 2024-09-21T06:27:51.450+0000 7f54bd3ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:51.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c[34533]: 2024-09-21T06:27:51.450+0000 7f54bd3ee640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:27:51.779 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:51 smithi110 podman[67991]: 2024-09-21 06:27:51.563178458 +0000 UTC m=+0.240937094 container died f53d66b96a12c65417f063af8c5a8948b01c87b8796ac9b43ffd0189bb18f15c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T06:27:52.203 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:51 smithi110 podman[67991]: 2024-09-21 06:27:51.967090952 +0000 UTC m=+0.644849588 container cleanup f53d66b96a12c65417f063af8c5a8948b01c87b8796ac9b43ffd0189bb18f15c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:27:52.203 INFO:journalctl@ceph.mon.c.smithi110.stdout:Sep 21 06:27:51 smithi110 bash[67991]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mon-c 2024-09-21T06:27:52.331 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T06:27:52.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:52.380 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T06:27:52.380 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T06:27:52.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:27:52.685 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:27:52 smithi031 systemd[1]: Stopping Ceph mgr.x for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:53.043 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:27:52 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x[31465]: 2024-09-21T06:27:52.683+0000 7f2567dab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:53.044 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:27:52 smithi031 podman[112654]: 2024-09-21 06:27:52.724434276 +0000 UTC m=+0.153662610 container died 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:27:53.385 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:27:53 smithi031 podman[112654]: 2024-09-21 06:27:53.250687444 +0000 UTC m=+0.679915767 container cleanup 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:27:53.385 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:27:53 smithi031 bash[112654]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x 2024-09-21T06:27:53.681 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Sep 21 06:27:53 smithi031 podman[112666]: 2024-09-21 06:27:53.419085798 +0000 UTC m=+0.697869109 container remove 283f39cb603a611faa2cbbf534c71c09215fff85d240d0c89a9af1897c63e7a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T06:27:53.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T06:27:53.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:53.722 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T06:27:53.722 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T06:27:53.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.y 2024-09-21T06:27:54.039 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 06:27:53 smithi107 systemd[1]: Stopping Ceph mgr.y for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:54.365 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 06:27:54 smithi107 podman[67902]: 2024-09-21 06:27:54.059378358 +0000 UTC m=+0.152471253 container died c8af4cd14750e3fa9e272a219176ba7f930009e562eafaff52dd573de7aae5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T06:27:54.707 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 06:27:54 smithi107 podman[67902]: 2024-09-21 06:27:54.438467426 +0000 UTC m=+0.531560321 container cleanup c8af4cd14750e3fa9e272a219176ba7f930009e562eafaff52dd573de7aae5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T06:27:54.708 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 21 06:27:54 smithi107 bash[67902]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-y 2024-09-21T06:27:54.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T06:27:54.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:54.893 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T06:27:54.893 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T06:27:54.894 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.z 2024-09-21T06:27:55.210 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:27:54 smithi110 systemd[1]: Stopping Ceph mgr.z for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:55.490 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:27:55 smithi110 podman[68115]: 2024-09-21 06:27:55.209778992 +0000 UTC m=+0.128763954 container died 7b6497934dcbf6abef57a30141aaff5158dd600836a11c870ed406e2d754a84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T06:27:55.778 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:27:55 smithi110 podman[68115]: 2024-09-21 06:27:55.622536415 +0000 UTC m=+0.541521384 container cleanup 7b6497934dcbf6abef57a30141aaff5158dd600836a11c870ed406e2d754a84d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:27:55.778 INFO:journalctl@ceph.mgr.z.smithi110.stdout:Sep 21 06:27:55 smithi110 bash[68115]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mgr-z 2024-09-21T06:27:55.997 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T06:27:56.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:27:56.031 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T06:27:56.032 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T06:27:56.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.0 2024-09-21T06:27:56.323 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:27:56 smithi031 systemd[1]: Stopping Ceph osd.0 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:27:56.591 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:27:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0[48759]: 2024-09-21T06:27:56.321+0000 7f9a82114640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:27:56.591 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:27:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0[48759]: 2024-09-21T06:27:56.321+0000 7f9a82114640 -1 osd.0 101 *** Got signal Terminated *** 2024-09-21T06:27:56.591 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:27:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0[48759]: 2024-09-21T06:27:56.321+0000 7f9a82114640 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:22.470 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:22.329+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:21.455457+0000) 2024-09-21T06:28:22.470 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:22.133+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:21.427579+0000) 2024-09-21T06:28:23.136 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:22 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:22.693+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:22.164978+0000) 2024-09-21T06:28:23.373 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:23.087+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:21.427579+0000) 2024-09-21T06:28:23.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:23 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:23.107+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:22.391215+0000) 2024-09-21T06:28:23.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:23.359+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:23.187445+0000) 2024-09-21T06:28:23.778 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:23.371+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:21.455457+0000) 2024-09-21T06:28:24.135 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:23.743+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:22.164978+0000) 2024-09-21T06:28:24.136 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:23 smithi031 podman[112773]: 2024-09-21 06:28:23.814999853 +0000 UTC m=+27.595960141 container died 64904715df8120f91d4870749de8e57de11bbf167ad6509e1b6c7afbd3f03e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T06:28:24.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:23.685+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:23.196425+0000) 2024-09-21T06:28:24.531 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:24 smithi031 podman[112773]: 2024-09-21 06:28:24.337451149 +0000 UTC m=+28.118411479 container cleanup 64904715df8120f91d4870749de8e57de11bbf167ad6509e1b6c7afbd3f03e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:28:24.532 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:24 smithi031 bash[112773]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0 2024-09-21T06:28:24.885 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:24 smithi031 podman[112801]: 2024-09-21 06:28:24.530696594 +0000 UTC m=+0.709067070 container remove 64904715df8120f91d4870749de8e57de11bbf167ad6509e1b6c7afbd3f03e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T06:28:25.218 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:24 smithi031 podman[112892]: 2024-09-21 06:28:24.86148698 +0000 UTC m=+0.016473327 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:25.219 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:24 smithi031 podman[112892]: 2024-09-21 06:28:24.982104607 +0000 UTC m=+0.137090986 container create 7da764ccaf3c81ead1491237a724893da07f6887df533bf5f2d5ff2093abe62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:28:25.219 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:25 smithi031 podman[112892]: 2024-09-21 06:28:25.217836023 +0000 UTC m=+0.372822368 container init 7da764ccaf3c81ead1491237a724893da07f6887df533bf5f2d5ff2093abe62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T06:28:25.635 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:25 smithi031 podman[112892]: 2024-09-21 06:28:25.221433966 +0000 UTC m=+0.376420300 container start 7da764ccaf3c81ead1491237a724893da07f6887df533bf5f2d5ff2093abe62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:28:25.636 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:25 smithi031 podman[112892]: 2024-09-21 06:28:25.301267982 +0000 UTC m=+0.456254318 container attach 7da764ccaf3c81ead1491237a724893da07f6887df533bf5f2d5ff2093abe62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T06:28:25.636 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 21 06:28:25 smithi031 podman[112892]: 2024-09-21 06:28:25.416725157 +0000 UTC m=+0.571711502 container died 7da764ccaf3c81ead1491237a724893da07f6887df533bf5f2d5ff2093abe62c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T06:28:26.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T06:28:26.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:26.201 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T06:28:26.202 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T06:28:26.202 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.1 2024-09-21T06:28:26.539 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:26 smithi107 systemd[1]: Stopping Ceph osd.1 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:28:27.039 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1[39074]: 2024-09-21T06:28:26.538+0000 7f20cb109640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:27.039 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1[39074]: 2024-09-21T06:28:26.538+0000 7f20cb109640 -1 osd.1 101 *** Got signal Terminated *** 2024-09-21T06:28:27.039 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1[39074]: 2024-09-21T06:28:26.538+0000 7f20cb109640 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:34.373 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:34 smithi107 podman[68038]: 2024-09-21 06:28:34.084583938 +0000 UTC m=+7.692980180 container died 30d955ba62db08498f35ebd6f1b9ccb67ce2dbb8b7c100488c179fb955bb94d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T06:28:34.789 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:34 smithi107 podman[68038]: 2024-09-21 06:28:34.371514962 +0000 UTC m=+7.979911202 container cleanup 30d955ba62db08498f35ebd6f1b9ccb67ce2dbb8b7c100488c179fb955bb94d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T06:28:34.789 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:34 smithi107 bash[68038]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1 2024-09-21T06:28:34.789 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:34 smithi107 podman[68055]: 2024-09-21 06:28:34.606146819 +0000 UTC m=+0.523443502 container remove 30d955ba62db08498f35ebd6f1b9ccb67ce2dbb8b7c100488c179fb955bb94d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T06:28:35.202 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:34 smithi107 podman[68146]: 2024-09-21 06:28:34.817426627 +0000 UTC m=+0.015619606 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:35.202 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:34 smithi107 podman[68146]: 2024-09-21 06:28:34.932215051 +0000 UTC m=+0.130408019 container create bcef57e51a11cd75345c609e63e0abeebd09da4f3dc77280bfd2e447697d3cd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-21T06:28:35.202 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:35 smithi107 podman[68146]: 2024-09-21 06:28:35.117538242 +0000 UTC m=+0.315731209 container init bcef57e51a11cd75345c609e63e0abeebd09da4f3dc77280bfd2e447697d3cd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T06:28:35.202 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:35 smithi107 podman[68146]: 2024-09-21 06:28:35.120855654 +0000 UTC m=+0.319048626 container start bcef57e51a11cd75345c609e63e0abeebd09da4f3dc77280bfd2e447697d3cd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T06:28:35.202 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:35 smithi107 podman[68146]: 2024-09-21 06:28:35.200833011 +0000 UTC m=+0.399025984 container attach bcef57e51a11cd75345c609e63e0abeebd09da4f3dc77280bfd2e447697d3cd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:28:35.539 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 21 06:28:35 smithi107 podman[68146]: 2024-09-21 06:28:35.307006056 +0000 UTC m=+0.505199034 container died bcef57e51a11cd75345c609e63e0abeebd09da4f3dc77280bfd2e447697d3cd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T06:28:35.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T06:28:35.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:35.931 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T06:28:35.931 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T06:28:35.931 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.2 2024-09-21T06:28:36.236 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:35 smithi110 systemd[1]: Stopping Ceph osd.2 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:28:36.528 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T06:28:36.233+0000 7fe646afd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:36.528 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T06:28:36.233+0000 7fe646afd640 -1 osd.2 101 *** Got signal Terminated *** 2024-09-21T06:28:36.528 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T06:28:36.233+0000 7fe646afd640 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:42.044 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:41 smithi110 podman[68256]: 2024-09-21 06:28:41.783780053 +0000 UTC m=+5.660688145 container died 81e0bc7f91d61cd111191987514045b266c9df5677fdb162850b1e74ec96d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T06:28:42.461 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:42 smithi110 podman[68256]: 2024-09-21 06:28:42.125758783 +0000 UTC m=+6.002666875 container cleanup 81e0bc7f91d61cd111191987514045b266c9df5677fdb162850b1e74ec96d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T06:28:42.461 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:42 smithi110 bash[68256]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2 2024-09-21T06:28:42.778 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:42 smithi110 podman[68271]: 2024-09-21 06:28:42.460552698 +0000 UTC m=+0.678996581 container remove 81e0bc7f91d61cd111191987514045b266c9df5677fdb162850b1e74ec96d629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:28:43.098 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:42 smithi110 podman[68361]: 2024-09-21 06:28:42.696937999 +0000 UTC m=+0.016623362 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:43.098 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:42 smithi110 podman[68361]: 2024-09-21 06:28:42.819746011 +0000 UTC m=+0.139431417 container create a7b70314b63a0fd4898adb35dbeec1caab44795c9638d9c96291a42a5b1fbdd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:28:43.098 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:43 smithi110 podman[68361]: 2024-09-21 06:28:43.013634443 +0000 UTC m=+0.333319804 container init a7b70314b63a0fd4898adb35dbeec1caab44795c9638d9c96291a42a5b1fbdd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T06:28:43.098 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:43 smithi110 podman[68361]: 2024-09-21 06:28:43.016889251 +0000 UTC m=+0.336574614 container start a7b70314b63a0fd4898adb35dbeec1caab44795c9638d9c96291a42a5b1fbdd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:28:43.098 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:43 smithi110 podman[68361]: 2024-09-21 06:28:43.09679353 +0000 UTC m=+0.416478904 container attach a7b70314b63a0fd4898adb35dbeec1caab44795c9638d9c96291a42a5b1fbdd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:28:43.487 INFO:journalctl@ceph.osd.2.smithi110.stdout:Sep 21 06:28:43 smithi110 podman[68361]: 2024-09-21 06:28:43.208694346 +0000 UTC m=+0.528379726 container died a7b70314b63a0fd4898adb35dbeec1caab44795c9638d9c96291a42a5b1fbdd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T06:28:43.901 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T06:28:43.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:43.939 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T06:28:43.940 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T06:28:43.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.3 2024-09-21T06:28:44.276 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:43 smithi031 systemd[1]: Stopping Ceph osd.3 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:28:44.635 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:44 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:44.272+0000 7f154b8c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:44.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:44 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:44.272+0000 7f154b8c7640 -1 osd.3 101 *** Got signal Terminated *** 2024-09-21T06:28:44.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:44 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:44.272+0000 7f154b8c7640 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:45.135 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:44 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:44.790+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:44.769010+0000) 2024-09-21T06:28:46.135 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:45 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:45.819+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:44.769010+0000) 2024-09-21T06:28:46.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:45 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:45.844+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:46.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:46 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:46.307+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:47.135 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:46 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:46.827+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:44.769010+0000) 2024-09-21T06:28:47.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:46 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:46.852+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:47.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:47.215+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:47.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:47.330+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:47.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:47.318+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:48.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:47.847+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:48.136 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:47 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:47.805+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:44.769010+0000) 2024-09-21T06:28:48.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:48.178+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:48.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:48.336+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:48.543 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:48 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:48.294+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:48.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:48 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:48.303+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:49.135 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:48 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3[55339]: 2024-09-21T06:28:48.836+0000 7f1547eb2640 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.864784+0000 front 2024-09-21T06:27:59.864937+0000 (oldest deadline 2024-09-21T06:28:44.769010+0000) 2024-09-21T06:28:49.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:48 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:48.881+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:49.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:49.140+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:49.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:49.329+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:49.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:49 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:49.327+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:49.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:49 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:49.311+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:50.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:49 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:49.874+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:50.136 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:49 smithi031 podman[113008]: 2024-09-21 06:28:49.917387788 +0000 UTC m=+5.765533043 container died bdc64bd44a6b2efe1d30cf89ba11b6348f13c79f88fff3c009f59ee1b9226910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T06:28:50.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:50.157+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:50.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:50.324+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:50.568 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:50 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:50.316+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:50.568 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:50 smithi031 podman[113008]: 2024-09-21 06:28:50.348906636 +0000 UTC m=+6.197051881 container cleanup bdc64bd44a6b2efe1d30cf89ba11b6348f13c79f88fff3c009f59ee1b9226910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T06:28:50.568 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:50 smithi031 bash[113008]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3 2024-09-21T06:28:50.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:50 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:50.328+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:50.858 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:50 smithi031 podman[113025]: 2024-09-21 06:28:50.567116507 +0000 UTC m=+0.645137944 container remove bdc64bd44a6b2efe1d30cf89ba11b6348f13c79f88fff3c009f59ee1b9226910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T06:28:51.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:50 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:50.856+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:51.135 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:50 smithi031 podman[113113]: 2024-09-21 06:28:50.95960702 +0000 UTC m=+0.103606546 container create a3e5309c492de52d615bd4396874c3bccaf6f0946135610091ecbdcf9d30b44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:28:51.136 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:50 smithi031 podman[113113]: 2024-09-21 06:28:50.872510726 +0000 UTC m=+0.016510195 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:51.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:51.166+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:51.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:51.320+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:51.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:51 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:51.325+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:51.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:51 smithi031 podman[113113]: 2024-09-21 06:28:51.212433707 +0000 UTC m=+0.356433181 container init a3e5309c492de52d615bd4396874c3bccaf6f0946135610091ecbdcf9d30b44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:28:51.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:51 smithi031 podman[113113]: 2024-09-21 06:28:51.216018042 +0000 UTC m=+0.360017491 container start a3e5309c492de52d615bd4396874c3bccaf6f0946135610091ecbdcf9d30b44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T06:28:51.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:51 smithi031 podman[113113]: 2024-09-21 06:28:51.295604315 +0000 UTC m=+0.439603782 container attach a3e5309c492de52d615bd4396874c3bccaf6f0946135610091ecbdcf9d30b44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T06:28:51.636 INFO:journalctl@ceph.osd.3.smithi031.stdout:Sep 21 06:28:51 smithi031 podman[113113]: 2024-09-21 06:28:51.413363716 +0000 UTC m=+0.557363173 container died a3e5309c492de52d615bd4396874c3bccaf6f0946135610091ecbdcf9d30b44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T06:28:51.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:51 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:51.342+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:52.000 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:51 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:51.904+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:52.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T06:28:52.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:52.077 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T06:28:52.077 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T06:28:52.077 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.4 2024-09-21T06:28:52.366 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:52 smithi107 systemd[1]: Stopping Ceph osd.4 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:28:52.429 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:52.295+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:52.132+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:52.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:52 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:52.312+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:52.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:52 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:52.364+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:52.789 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:52 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4[45525]: 2024-09-21T06:28:52.388+0000 7fc5237ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:28:52.789 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:52 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4[45525]: 2024-09-21T06:28:52.388+0000 7fc5237ac640 -1 osd.4 101 *** Got signal Terminated *** 2024-09-21T06:28:52.790 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:52 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4[45525]: 2024-09-21T06:28:52.388+0000 7fc5237ac640 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:28:53.352 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:52 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:52.950+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:53.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:53.158+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:53.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:53.267+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:53.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:53 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:53.351+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:53.691 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:53 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:53.414+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:54.277 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:54 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:54.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:54.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:53 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:53.922+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:54.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:54 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:54.275+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:54.703 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:54 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:54.397+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:54.885 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:54 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:54.387+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:55.289 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:28:55 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:28:55.124+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:28:55.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:54 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:54.935+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:55.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:55 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:55.232+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:55.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:55 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:55.147+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:55.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:55 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:55.147+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:28:55.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:55 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:55.370+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:55.885 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:55 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:55.390+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:55.886 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:55 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:55.390+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:28:56.289 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:28:56 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:28:56.120+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:28:56.359 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:55 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:55.903+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:56.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:56 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:56.206+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:56.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:56 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:56.182+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:56.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:56 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:56.182+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:28:56.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:56.357+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:56.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:56.357+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:28:56.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:56 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:56.325+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:56.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:56 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:56.325+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:28:57.373 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:56.932+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:57.374 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:56.932+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:28:57.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:57 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:57.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:57.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:57 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:57.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:28:57.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:57 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:57.209+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:57.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:57 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:57.209+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:28:57.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:28:57 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:28:57.084+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:28:57.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:57 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:57.286+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:57.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:57 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:57.286+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:28:57.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:57 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:57.371+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:57.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:57 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:57.371+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:28:58.157 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:57 smithi107 podman[68260]: 2024-09-21 06:28:57.888532852 +0000 UTC m=+5.609869814 container died 69cff832c6be3faef06e450aca1d25eb6c50fcd021522a1c3abc9a2b065c9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T06:28:58.157 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:28:58 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:28:58.131+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:28:58.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:57 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:57.954+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:58.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:57 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:57.954+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:28:58.523 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:58 smithi107 podman[68260]: 2024-09-21 06:28:58.263005981 +0000 UTC m=+5.984342930 container cleanup 69cff832c6be3faef06e450aca1d25eb6c50fcd021522a1c3abc9a2b065c9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T06:28:58.523 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:58 smithi107 bash[68260]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4 2024-09-21T06:28:58.524 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:58 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:58.279+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:58.524 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:58 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:58.279+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:28:58.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:58 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:58.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:58.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:58 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:58.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:28:58.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:58 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:58.165+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:58.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:58 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:58.165+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:28:58.789 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:58 smithi107 podman[68287]: 2024-09-21 06:28:58.522597623 +0000 UTC m=+0.633133989 container remove 69cff832c6be3faef06e450aca1d25eb6c50fcd021522a1c3abc9a2b065c9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T06:28:58.885 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:58 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:58.406+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:58.886 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:58 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:58.406+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:28:59.143 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:58 smithi107 podman[68376]: 2024-09-21 06:28:58.774768093 +0000 UTC m=+0.015935062 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:28:59.143 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:58 smithi107 podman[68376]: 2024-09-21 06:28:58.881741402 +0000 UTC m=+0.122908405 container create 7376854a4257cab8e3bc24abdf3fc388d3d649f8f4cdf80c49d7e50ec5a6e100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:28:59.368 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:58 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:58.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:28:59.369 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:58 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:58.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:28:59.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:28:59 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:28:59.307+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:28:59.528 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:59 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:59.172+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:28:59.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:59 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:28:59.172+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:28:59.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:59 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:59.130+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:28:59.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:28:59 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:28:59.130+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:28:59.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:28:59 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:28:59.177+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:28:59.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:59 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:59.231+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:28:59.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:28:59 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:28:59.231+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:28:59.540 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:59 smithi107 podman[68376]: 2024-09-21 06:28:59.142104018 +0000 UTC m=+0.383270973 container init 7376854a4257cab8e3bc24abdf3fc388d3d649f8f4cdf80c49d7e50ec5a6e100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T06:28:59.540 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:59 smithi107 podman[68376]: 2024-09-21 06:28:59.145415108 +0000 UTC m=+0.386582073 container start 7376854a4257cab8e3bc24abdf3fc388d3d649f8f4cdf80c49d7e50ec5a6e100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:28:59.540 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:59 smithi107 podman[68376]: 2024-09-21 06:28:59.217055899 +0000 UTC m=+0.458222858 container attach 7376854a4257cab8e3bc24abdf3fc388d3d649f8f4cdf80c49d7e50ec5a6e100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T06:28:59.540 INFO:journalctl@ceph.osd.4.smithi107.stdout:Sep 21 06:28:59 smithi107 podman[68376]: 2024-09-21 06:28:59.338095777 +0000 UTC m=+0.579262740 container died 7376854a4257cab8e3bc24abdf3fc388d3d649f8f4cdf80c49d7e50ec5a6e100 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T06:28:59.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:59 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:59.367+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:28:59.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:28:59 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:28:59.367+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:28:59.907 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T06:28:59.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:28:59.946 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T06:28:59.946 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T06:28:59.946 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.5 2024-09-21T06:29:00.247 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:00.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:00.247 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:00.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:00.247 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:28:59 smithi110 systemd[1]: Stopping Ceph osd.5 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:00.247 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:00.200+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:29:00.247 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:00.200+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:29:00.247 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:00.245+0000 7f0e58b94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:00.248 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:00.245+0000 7f0e58b94640 -1 osd.5 101 *** Got signal Terminated *** 2024-09-21T06:29:00.248 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:00.245+0000 7f0e58b94640 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:00.337 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:59 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:59.928+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:00.337 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:28:59 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:28:59.928+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:00.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:00 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:00.331+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:00.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:00 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:00.201+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:00.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:00 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:00.235+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:00.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:00 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:00.235+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:00.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:00.335+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:00.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:00.335+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:01.310 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:00.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:01.310 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:00 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:00.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:01.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:01 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:01.361+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:01.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:01 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:01.225+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:29:01.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:01 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:01.225+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:29:01.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:01 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:01.127+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:01.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:01 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:01.127+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:01.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:01 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:01.183+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:01.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:01 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:01.224+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:01.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:01 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:01.224+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:01.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:01 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:01.308+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:01.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:01 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:01.308+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:02.306 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:01 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:01.898+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:02.306 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:01 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:01.898+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:02.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:02 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:02.383+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:02.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:02 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:02.164+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:02.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:02 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:02.164+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:02.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:02 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:02.261+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:29:02.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:02 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:02.261+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:29:02.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:02 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:02.261+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:36.735171+0000 front 2024-09-21T06:28:36.735145+0000 (oldest deadline 2024-09-21T06:29:02.035434+0000) 2024-09-21T06:29:02.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:02 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:02.191+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:02.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:02 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:02.193+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:02.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:02 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:02.193+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:02.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:02 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:02.304+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:02.637 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:02 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:02.304+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:02.637 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:02 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:02.304+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:03.277 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:02 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:02.936+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:03.278 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:02 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:02.936+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:03.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:03.356+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:03.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:03.155+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:03.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:03.155+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:03.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:03.220+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:29:03.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:03.220+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:29:03.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:03 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:03.220+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:36.735171+0000 front 2024-09-21T06:28:36.735145+0000 (oldest deadline 2024-09-21T06:29:02.035434+0000) 2024-09-21T06:29:03.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:03 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:03.146+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:03.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:03 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:03.217+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:03.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:03 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:03.217+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:03.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:03 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:03.276+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:03.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:03 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:03.276+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:03.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:03 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:03.276+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:04.284 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:03 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:03.905+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:04.285 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:03 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:03.905+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:04.285 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:03 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:03.905+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:04.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:04.351+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:04.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:04.351+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:04.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:04.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:04.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:04.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:04.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:04.178+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:29:04.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:04.178+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:29:04.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:04 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:04.178+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:36.735171+0000 front 2024-09-21T06:28:36.735145+0000 (oldest deadline 2024-09-21T06:29:02.035434+0000) 2024-09-21T06:29:04.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:04 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:04.152+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:04.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:04 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:04.189+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:04.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:04 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:04.189+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:04.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:04.283+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:04.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:04.283+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:04.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:04.283+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:05.282 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:04.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:05.282 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:04.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:05.282 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:04 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:04.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:05.423 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:05 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:05.104+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:05.423 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:05 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:05.104+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:05.423 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:05 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:05.217+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:05.423 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:05 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:05.217+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:05.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:05.127+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:05.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:05.127+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:05.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:05.312+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:05.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:05.312+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:05.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:05.156+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:56.727700+0000 front 2024-09-21T06:27:56.727623+0000 (oldest deadline 2024-09-21T06:28:47.032118+0000) 2024-09-21T06:29:05.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:05.156+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.433587+0000 front 2024-09-21T06:28:30.433637+0000 (oldest deadline 2024-09-21T06:28:56.234224+0000) 2024-09-21T06:29:05.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:05 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5[45899]: 2024-09-21T06:29:05.156+0000 7f0e5517f640 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:36.735171+0000 front 2024-09-21T06:28:36.735145+0000 (oldest deadline 2024-09-21T06:29:02.035434+0000) 2024-09-21T06:29:05.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:05 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:05.280+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:05.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:05 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:05.280+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:05.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:05 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:05.280+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:06.130 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:05 smithi110 podman[68474]: 2024-09-21 06:29:05.864119571 +0000 UTC m=+5.720195889 container died d4e3c37b794dedadf03379c869b2d5ff8c3647eccef2c7f3ad20b0ec73cd1da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T06:29:06.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:05 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:05.875+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:06.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:05 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:05.875+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:06.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:05 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:05.875+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:06.245 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:06 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:06.097+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:06.245 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:06 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:06.097+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:06.498 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:06 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:06.338+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:06.498 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:06 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:06.338+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:06.498 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:06 smithi110 podman[68474]: 2024-09-21 06:29:06.23733263 +0000 UTC m=+6.093408937 container cleanup d4e3c37b794dedadf03379c869b2d5ff8c3647eccef2c7f3ad20b0ec73cd1da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T06:29:06.498 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:06 smithi110 bash[68474]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5 2024-09-21T06:29:06.499 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:06 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:06.128+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:06.499 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:06 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:06.128+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:06.499 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:06 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:06.128+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:06.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:06 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:06.243+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:06.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:06 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:06.243+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:06.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:06 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:06.243+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:06.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:06 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:06.240+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:06.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:06 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:06.240+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:06.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:06 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:06.240+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:06.779 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:06 smithi110 podman[68489]: 2024-09-21 06:29:06.496796513 +0000 UTC m=+0.629112505 container remove d4e3c37b794dedadf03379c869b2d5ff8c3647eccef2c7f3ad20b0ec73cd1da1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:29:07.127 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:06 smithi110 podman[68578]: 2024-09-21 06:29:06.839500244 +0000 UTC m=+0.108442496 container create d58304ce7212bd51a2237cdef7b6541094e6617781606132f87fce2005468cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T06:29:07.128 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:06 smithi110 podman[68578]: 2024-09-21 06:29:06.746900034 +0000 UTC m=+0.015842255 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:07.128 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:07 smithi110 podman[68578]: 2024-09-21 06:29:07.066878921 +0000 UTC m=+0.335821128 container init d58304ce7212bd51a2237cdef7b6541094e6617781606132f87fce2005468cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T06:29:07.128 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:07 smithi110 podman[68578]: 2024-09-21 06:29:07.070129765 +0000 UTC m=+0.339071972 container start d58304ce7212bd51a2237cdef7b6541094e6617781606132f87fce2005468cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T06:29:07.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:06 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:06.864+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:07.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:06 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:06.864+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:07.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:06 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:06.864+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:07.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:07 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:07.292+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:07.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:07 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:07.292+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:07.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:07 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:07.125+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:07.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:07 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:07.125+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:07.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:07 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:07.125+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:07.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:07 smithi110 podman[68578]: 2024-09-21 06:29:07.16892228 +0000 UTC m=+0.437864494 container attach d58304ce7212bd51a2237cdef7b6541094e6617781606132f87fce2005468cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T06:29:07.529 INFO:journalctl@ceph.osd.5.smithi110.stdout:Sep 21 06:29:07 smithi110 podman[68578]: 2024-09-21 06:29:07.260153984 +0000 UTC m=+0.529096194 container died d58304ce7212bd51a2237cdef7b6541094e6617781606132f87fce2005468cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-5-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T06:29:07.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:07 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:07.137+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:07.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:07 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:07.137+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:07.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:07 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:07.266+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:07.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:07 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:07.266+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:07.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:07 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:07.266+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:07.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:07.249+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:07.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:07.249+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:07.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:07.249+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:07.843 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T06:29:07.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:07.881 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T06:29:07.882 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T06:29:07.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.6 2024-09-21T06:29:08.136 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:07 smithi031 systemd[1]: Stopping Ceph osd.6 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:08.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:07.865+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:08.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:07.865+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:08.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:07.865+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:08.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:08 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:08.312+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:08.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:08 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:08.312+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:08.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:08 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:08.169+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:08.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:08 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:08.169+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:08.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:08 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:08.169+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:08.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:08 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:08.179+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:08.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:08 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:08.179+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:08.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:08 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:08.274+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:08.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:08 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:08.274+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:08.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:08 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:08.274+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:08.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:08.184+0000 7fce9cc95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:08.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:08.184+0000 7fce9cc95640 -1 osd.6 101 *** Got signal Terminated *** 2024-09-21T06:29:08.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:08.184+0000 7fce9cc95640 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:08.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:08.212+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:08.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:08.212+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:08.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:08.212+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:09.252 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:08.887+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:09.252 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:08.887+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:09.252 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:08 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:08.887+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:09.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:09 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:09.275+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:09.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:09 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:09.275+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:09.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:09 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:09.189+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:09.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:09 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:09.189+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:09.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:09 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:09.189+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:09.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:09 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:09.211+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:09.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:09 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:09.211+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:09.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:09 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:09.269+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:09.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:09 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:09.269+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:09.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:09 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:09.269+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:09.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:09 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:09.250+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:09.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:09 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:09.250+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:09.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:09 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:09.250+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:10.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:09 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:09.872+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:10.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:09 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:09.872+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:10.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:09 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:09.872+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:10.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:10 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:10.292+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:10.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:10 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:10.292+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:10.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:10 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:10.292+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:10.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:10 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:10.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:10.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:10 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:10.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:10.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:10 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:10.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:10.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:10.207+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:10.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:10.207+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:10.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:10.228+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:10.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:10.228+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:10.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:10 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:10.228+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:10.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:10 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:10.203+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:10.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:10 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:10.203+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:10.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:10 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:10.203+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:11.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:10 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:10.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:11.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:10 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:10.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:11.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:10 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:10.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:11.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:11.304+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:11.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:11.304+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:11.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:11.304+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:11.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:11.145+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:11.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:11.146+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:11.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:11 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:11.146+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:11.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:11 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:11.253+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:11.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:11 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:11.253+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:11.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:11 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:11.253+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:11.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:11 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:11.235+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:11.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:11 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:11.235+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:11.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:11 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:11.235+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:11.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:11.231+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:11.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:11.231+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:11.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:11.231+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:11.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:11.231+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.097287+0000 front 2024-09-21T06:28:49.097315+0000 (oldest deadline 2024-09-21T06:29:10.797429+0000) 2024-09-21T06:29:12.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:11.883+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:12.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:11.883+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:12.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:11.883+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:12.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:11.883+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:12.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:12 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:12.316+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:12.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:12 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:12.316+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:12.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:12 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:12.316+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:12.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:12 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:12.104+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:12.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:12 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:12.104+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:12.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:12 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:12.104+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:12.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:12.298+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:12.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:12.298+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:12.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:12.298+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:12.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:12.227+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:12.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:12.227+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:12.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:12.227+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:12.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:12 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:12.227+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:12.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:12.243+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:28:00.287308+0000 front 2024-09-21T06:28:00.287358+0000 (oldest deadline 2024-09-21T06:28:45.891431+0000) 2024-09-21T06:29:12.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:12.243+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.392666+0000 front 2024-09-21T06:28:29.392596+0000 (oldest deadline 2024-09-21T06:28:54.494370+0000) 2024-09-21T06:29:12.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:12.243+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.795656+0000 front 2024-09-21T06:28:39.795512+0000 (oldest deadline 2024-09-21T06:29:02.095559+0000) 2024-09-21T06:29:12.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6[61917]: 2024-09-21T06:29:12.243+0000 7fce99280640 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.097287+0000 front 2024-09-21T06:28:49.097315+0000 (oldest deadline 2024-09-21T06:29:10.797429+0000) 2024-09-21T06:29:13.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:13 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:13.070+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:13.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:13 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:13.070+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:13.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:13 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:13.070+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:13.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:12.927+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:13.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:12.927+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:13.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:12.927+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:13.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:12 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:12.927+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:13.253+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:13.253+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:13.253+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:13.224+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:13.224+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:13.224+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:13.542 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:13 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:13.224+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:13.778 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:13 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:13.352+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:13.778 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:13 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:13.352+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:13.778 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:13 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:13.352+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:14.135 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:13 smithi031 podman[113230]: 2024-09-21 06:29:13.82676557 +0000 UTC m=+5.744590852 container died 899043428def7b1f4b88c458514c9d95e5d463be0e59a2787985ce968ea5d879 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:29:14.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:13 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:13.933+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:14.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:13 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:13.933+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:14.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:13 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:13.933+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:14.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:13 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:13.933+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:14.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:14 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:14.321+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:14.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:14 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:14.321+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:14.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:14 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:14.321+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:14.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:14 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:14.094+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:14.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:14 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:14.094+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:14.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:14 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:14.094+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:14.216+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:14.216+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:14.216+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:14.248+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:14.248+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:14.248+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:14.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:14 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:14.248+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:14.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:14 smithi031 podman[113230]: 2024-09-21 06:29:14.427901983 +0000 UTC m=+6.345727292 container cleanup 899043428def7b1f4b88c458514c9d95e5d463be0e59a2787985ce968ea5d879 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T06:29:14.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:14 smithi031 bash[113230]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6 2024-09-21T06:29:14.970 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:14 smithi031 podman[113247]: 2024-09-21 06:29:14.679150239 +0000 UTC m=+0.847121651 container remove 899043428def7b1f4b88c458514c9d95e5d463be0e59a2787985ce968ea5d879 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T06:29:15.233 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:14 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:14.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:15.233 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:14 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:14.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:15.233 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:14 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:14.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:15.233 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:14 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:14.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:15.233 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:15 smithi031 podman[113333]: 2024-09-21 06:29:14.914213876 +0000 UTC m=+0.016308401 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:15.234 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:15 smithi031 podman[113333]: 2024-09-21 06:29:15.013264932 +0000 UTC m=+0.115359456 container create b8ee819c7b5a8832cd536db8ec3a17ee4094d2527e9bf720961b76ffddf681a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:29:15.234 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:15 smithi031 podman[113333]: 2024-09-21 06:29:15.232181525 +0000 UTC m=+0.334276050 container init b8ee819c7b5a8832cd536db8ec3a17ee4094d2527e9bf720961b76ffddf681a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T06:29:15.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:15 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:15.339+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:15.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:15 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:15.339+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:15.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:15 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:15.339+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:15.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:15 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:15.144+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:15.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:15 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:15.144+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:15.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:15 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:15.144+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:15.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:15.186+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:15.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:15.186+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:15.540 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:15.186+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:15.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:15.199+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:15.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:15.199+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:15.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:15.199+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:15.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:15 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:15.199+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:15.635 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:15 smithi031 podman[113333]: 2024-09-21 06:29:15.235781185 +0000 UTC m=+0.337875710 container start b8ee819c7b5a8832cd536db8ec3a17ee4094d2527e9bf720961b76ffddf681a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T06:29:15.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:15 smithi031 podman[113333]: 2024-09-21 06:29:15.332142073 +0000 UTC m=+0.434236607 container attach b8ee819c7b5a8832cd536db8ec3a17ee4094d2527e9bf720961b76ffddf681a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:29:15.636 INFO:journalctl@ceph.osd.6.smithi031.stdout:Sep 21 06:29:15 smithi031 podman[113333]: 2024-09-21 06:29:15.430184811 +0000 UTC m=+0.532279337 container died b8ee819c7b5a8832cd536db8ec3a17ee4094d2527e9bf720961b76ffddf681a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T06:29:16.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T06:29:16.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:16.149 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T06:29:16.149 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T06:29:16.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.7 2024-09-21T06:29:16.367 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:16.107+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:16.368 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:16.107+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:16.368 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:16.107+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:16.369 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:16.366+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:16.369 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:16.366+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:16.369 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:16 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:16.366+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:16.373 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:15 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:15.984+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:16.373 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:15 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:15.984+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:16.373 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:15 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:15.984+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:16.373 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:15 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:15.984+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:16.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:16.142+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:16.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:16.142+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:16.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:16.142+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:16.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 systemd[1]: Stopping Ceph osd.7 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:16.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:16.174+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:16.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:16.174+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:16.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:16.174+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:16.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:16.174+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:17.039 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:16.729+0000 7fd6aff30640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:17.040 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:16.729+0000 7fd6aff30640 -1 osd.7 101 *** Got signal Terminated *** 2024-09-21T06:29:17.040 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:16.729+0000 7fd6aff30640 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:17.380 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:17 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:17.099+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:17.381 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:17 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:17.099+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:17.381 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:17 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:17.099+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:17.381 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:16 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:16.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:17.381 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:16 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:16.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:17.381 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:16 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:16.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:17.382 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:16 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:16.967+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:17.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:17.157+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:17.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:17.157+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:17.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:17.157+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:17.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:17.131+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:17.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:17.131+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:17.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:17.131+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:17.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:17 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:17.131+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:17.778 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:17 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:17.378+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:17.778 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:17 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:17.378+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:17.779 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:17 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:17.378+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:18.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:17 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:17.954+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:18.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:17 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:17.954+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:18.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:17 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:17.954+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:18.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:17 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:17.954+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:18.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:18.338+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:18.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:18.338+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:18.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:18.338+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:18.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:18.141+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:18.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:18.141+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:18.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:18 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:18.141+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:18.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:18.135+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:18.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:18.135+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:18.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:18.135+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:18.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:18.135+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.027019+0000 front 2024-09-21T06:28:57.027047+0000 (oldest deadline 2024-09-21T06:29:18.127167+0000) 2024-09-21T06:29:18.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:18.154+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:18.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:18.154+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:18.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:18.154+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:18.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:18.154+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:19.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:18 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:18.952+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:19.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:18 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:18.952+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:19.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:18 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:18.952+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:19.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:18 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:18.952+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:19.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:19 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:19.333+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:19.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:19 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:19.333+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:19.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:19 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:19.333+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:19.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:19 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:19.159+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:19.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:19 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:19.159+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:19.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:19 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:19.159+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:19.163+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:19.163+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:19.163+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:19.163+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.027019+0000 front 2024-09-21T06:28:57.027047+0000 (oldest deadline 2024-09-21T06:29:18.127167+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:19.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:19.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:19.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:19.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:19.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:19 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:19.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:20.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:19 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:19.913+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:20.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:19 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:19.913+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:20.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:19 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:19.913+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:20.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:19 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:19.913+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:20.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:20.332+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:20.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:20.332+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:20.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:20.332+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:20.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:20.134+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:20.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:20.134+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:20.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:20.134+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:20.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:20 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:20.134+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:20.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:20.182+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:20.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:20.182+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:20.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:20.182+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:20.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:20.182+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.027019+0000 front 2024-09-21T06:28:57.027047+0000 (oldest deadline 2024-09-21T06:29:18.127167+0000) 2024-09-21T06:29:20.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:20.137+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:20.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:20.137+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:20.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:20.137+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:20.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:20.137+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:20.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:20 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:20.137+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:21.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:20 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:20.962+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:21.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:20 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:20.962+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:21.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:20 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:20.962+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:21.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:20 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:20.962+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:21.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:21.105+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:21.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:21.105+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:21.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:21.105+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:21.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:21.105+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:21.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:21.286+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:21.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:21.286+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:21.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:21.286+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:21.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:21 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:21.286+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:21.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:21.177+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.721946+0000 front 2024-09-21T06:28:29.721987+0000 (oldest deadline 2024-09-21T06:28:54.422111+0000) 2024-09-21T06:29:21.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:21.177+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:41.225627+0000 front 2024-09-21T06:28:41.225571+0000 (oldest deadline 2024-09-21T06:29:04.725305+0000) 2024-09-21T06:29:21.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:21.177+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:45.826206+0000 front 2024-09-21T06:28:45.826059+0000 (oldest deadline 2024-09-21T06:29:11.126168+0000) 2024-09-21T06:29:21.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7[51807]: 2024-09-21T06:29:21.177+0000 7fd6a8513640 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.027019+0000 front 2024-09-21T06:28:57.027047+0000 (oldest deadline 2024-09-21T06:29:18.127167+0000) 2024-09-21T06:29:21.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:21.121+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:21.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:21.121+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:21.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:21.121+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:21.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:21.121+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:21.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:21 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:21.121+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:22.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:21 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:21.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:22.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:21 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:21.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:22.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:21 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:21.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:22.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:21 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:21.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:22.429 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:22.261+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:22.261+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:22.261+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:22.261+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:22.121+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:22.121+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:22.121+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:22.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:22 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:22.121+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:22.538 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:22.112+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:22.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:22.112+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:22.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:22.112+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:22.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:22.112+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:22.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:22.112+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:22.539 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:22 smithi107 podman[68477]: 2024-09-21 06:29:22.283978737 +0000 UTC m=+5.941742833 container died 9c2a437c7c2e3d7eb6c995198ec9a188bd09f6b58b376f87c3253594f2843fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T06:29:22.789 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:22 smithi107 podman[68477]: 2024-09-21 06:29:22.611611683 +0000 UTC m=+6.269375769 container cleanup 9c2a437c7c2e3d7eb6c995198ec9a188bd09f6b58b376f87c3253594f2843fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:29:22.789 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:22 smithi107 bash[68477]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7 2024-09-21T06:29:23.101 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:22 smithi107 podman[68493]: 2024-09-21 06:29:22.796266733 +0000 UTC m=+0.513538150 container remove 9c2a437c7c2e3d7eb6c995198ec9a188bd09f6b58b376f87c3253594f2843fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T06:29:23.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:22 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:22.936+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:23.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:22 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:22.936+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:23.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:22 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:22.936+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:23.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:22 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:22.936+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:23.434 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:23 smithi107 podman[68580]: 2024-09-21 06:29:23.19728232 +0000 UTC m=+0.093682467 container create f6446d46ce6523966a6cd3827f36f5d3ddc091a509990e7d3885fbf26911767e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T06:29:23.434 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:23 smithi107 podman[68580]: 2024-09-21 06:29:23.119440448 +0000 UTC m=+0.015840597 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:23.434 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:23 smithi107 podman[68580]: 2024-09-21 06:29:23.433423615 +0000 UTC m=+0.329823767 container init f6446d46ce6523966a6cd3827f36f5d3ddc091a509990e7d3885fbf26911767e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T06:29:23.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:23 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:23.098+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:23.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:23 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:23.098+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:23.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:23 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:23.098+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:23.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:23 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:23.098+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:23.435 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:23 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:23.098+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:23.450 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:23.257+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:23.450 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:23.257+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:23.450 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:23.257+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:23.450 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:23.257+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:23.450 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:23.152+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:23.450 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:23.152+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:23.451 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:23.152+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:23.451 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:23 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:23.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:23.789 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:23 smithi107 podman[68580]: 2024-09-21 06:29:23.437107476 +0000 UTC m=+0.333507626 container start f6446d46ce6523966a6cd3827f36f5d3ddc091a509990e7d3885fbf26911767e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:29:23.789 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:23 smithi107 podman[68580]: 2024-09-21 06:29:23.541514644 +0000 UTC m=+0.437914795 container attach f6446d46ce6523966a6cd3827f36f5d3ddc091a509990e7d3885fbf26911767e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:29:23.789 INFO:journalctl@ceph.osd.7.smithi107.stdout:Sep 21 06:29:23 smithi107 podman[68580]: 2024-09-21 06:29:23.622328631 +0000 UTC m=+0.518728789 container died f6446d46ce6523966a6cd3827f36f5d3ddc091a509990e7d3885fbf26911767e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:29:24.170 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:24.125+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:24.170 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:24.125+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:24.170 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:24.125+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:24.170 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:24.125+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:24.171 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:24.125+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:24.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T06:29:24.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:24.227 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T06:29:24.227 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T06:29:24.227 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.8 2024-09-21T06:29:24.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:23.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:24.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:23.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:24.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:23.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:24.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:23 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:23.916+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:24.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:24.190+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:24.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:24.190+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:24.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:24.190+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:24.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:24.190+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:24.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.248+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:24.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.248+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:24.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.248+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:24.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.248+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:24.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 systemd[1]: Stopping Ceph osd.8 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:25.028 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.552+0000 7f1b3460d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:25.028 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.552+0000 7f1b3460d640 -1 osd.8 101 *** Got signal Terminated *** 2024-09-21T06:29:25.029 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:24 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:24.552+0000 7f1b3460d640 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:25.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:24 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:24.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:25.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:24 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:24.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:25.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:24 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:24.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:25.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:24 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:24.876+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:25.386 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:25 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:25.091+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:25.386 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:25 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:25.091+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:25.386 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:25 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:25.091+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:25.386 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:25 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:25.091+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:25.386 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:25 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:25.091+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:25.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:25.246+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:25.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:25.246+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:25.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:25.246+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:25.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:25.246+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:25.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:25.171+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:25.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:25.171+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:25.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:25.171+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:25.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:25 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:25.171+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:26.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:25 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:25.892+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:26.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:25 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:25.892+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:26.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:25 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:25.892+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:26.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:25 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:25.892+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:26.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:26.201+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:26.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:26.201+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:26.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:26.201+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:26.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:26.201+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:26.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:26.181+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:26.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:26.181+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:26.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:26.181+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:26.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:26 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:26.181+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:26.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:26.071+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:26.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:26.071+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:26.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:26.071+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:26.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:26.071+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:26.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:26 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:26.071+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:27.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:26 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:26.860+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:27.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:26 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:26.860+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:27.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:26 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:26.860+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:27.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:26 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:26.860+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:27.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:27.025+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:27.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:27.025+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:27.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:27.025+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:27.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:27.025+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:27.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:27 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:27.025+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:27.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:27.210+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:27.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:27.210+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:27.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:27.210+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:27.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:27.210+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:27.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:27.210+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:00.602907+0000 front 2024-09-21T06:29:00.602855+0000 (oldest deadline 2024-09-21T06:29:26.502785+0000) 2024-09-21T06:29:27.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:27.132+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:27.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:27.132+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:27.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:27.132+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:27.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:27.132+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:27.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:27 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:27.132+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:28.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:27 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:27.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:28.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:27 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:27.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:28.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:27 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:27.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:28.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:27 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:27.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:28.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:27 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:27.888+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:28.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:28.198+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:28.587 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:28.198+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:28.587 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:28.198+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:28.587 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:28.198+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:28.587 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:28.198+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:00.602907+0000 front 2024-09-21T06:29:00.602855+0000 (oldest deadline 2024-09-21T06:29:26.502785+0000) 2024-09-21T06:29:28.588 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:28.158+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:28.588 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:28.158+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:28.588 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:28.158+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:28.588 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:28.158+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:28.588 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:28.158+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:28.588 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:28 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:28.046+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:28.589 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:28 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:28.046+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:28.589 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:28 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:28.046+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:28.589 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:28 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:28.046+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:28.589 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:28 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:28.046+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:29.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:28 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:28.931+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:29.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:28 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:28.931+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:29.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:28 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:28.931+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:29.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:28 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:28.931+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:29.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:28 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:28.931+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:29.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:29.212+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:30.098356+0000 front 2024-09-21T06:28:30.098575+0000 (oldest deadline 2024-09-21T06:28:58.899098+0000) 2024-09-21T06:29:29.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:29.212+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:38.899591+0000 front 2024-09-21T06:28:38.899562+0000 (oldest deadline 2024-09-21T06:29:04.199771+0000) 2024-09-21T06:29:29.528 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:29.212+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:44.200417+0000 front 2024-09-21T06:28:44.200513+0000 (oldest deadline 2024-09-21T06:29:10.100375+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:29.212+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:56.502507+0000 front 2024-09-21T06:28:56.502428+0000 (oldest deadline 2024-09-21T06:29:20.602303+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8[52114]: 2024-09-21T06:29:29.212+0000 7f1b2cbf0640 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:00.602907+0000 front 2024-09-21T06:29:00.602855+0000 (oldest deadline 2024-09-21T06:29:26.502785+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:29.195+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:29.195+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:29.195+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:29.195+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:29.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:29 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:29.195+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:29.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:29 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:29.040+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:29.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:29 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:29.040+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:29.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:29 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:29.040+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:29.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:29 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:29.040+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:29.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:29 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:29.040+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:29.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:29 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:29.040+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:30.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:30 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:30.010+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:30.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:30 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:30.010+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:30.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:30 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:30.010+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:30.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:30 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:30.010+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:30.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:30 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:30.010+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:30.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:30 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:30.010+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:30.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:29.929+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:30.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:29.929+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:30.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:29.929+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:30.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:29.929+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:30.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:29 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:29.929+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:30.424 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:30 smithi110 podman[68691]: 2024-09-21 06:29:30.121902962 +0000 UTC m=+5.677342383 container died 00657abe235bf0448c0afab6bb826b9fe3c72f58ea0a0cdfdaeab8fa80d7f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:29:30.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:30 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:30.198+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:30.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:30 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:30.198+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:30.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:30 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:30.198+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:30.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:30 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:30.198+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:30.425 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:30 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:30.198+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:30.733 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:30 smithi110 podman[68691]: 2024-09-21 06:29:30.514222971 +0000 UTC m=+6.069662391 container cleanup 00657abe235bf0448c0afab6bb826b9fe3c72f58ea0a0cdfdaeab8fa80d7f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T06:29:30.733 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:30 smithi110 bash[68691]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8 2024-09-21T06:29:31.028 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:30 smithi110 podman[68707]: 2024-09-21 06:29:30.731980662 +0000 UTC m=+0.610570056 container remove 00657abe235bf0448c0afab6bb826b9fe3c72f58ea0a0cdfdaeab8fa80d7f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T06:29:31.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:30 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:30.882+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:31.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:30 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:30.882+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:31.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:30 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:30.882+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:31.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:30 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:30.882+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:31.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:30 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:30.882+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:31.193+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:31.193+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:31.193+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:31.193+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:31.193+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:31 smithi110 podman[68795]: 2024-09-21 06:29:30.942357713 +0000 UTC m=+0.015711156 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:31.353 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:31 smithi110 podman[68795]: 2024-09-21 06:29:31.049470101 +0000 UTC m=+0.122823534 container create 889b6b74daad712e13324416be2445dedb868f3bba8ac212243eef3121bdb918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T06:29:31.354 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:31 smithi110 podman[68795]: 2024-09-21 06:29:31.268561394 +0000 UTC m=+0.341914828 container init 889b6b74daad712e13324416be2445dedb868f3bba8ac212243eef3121bdb918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T06:29:31.354 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:31 smithi110 podman[68795]: 2024-09-21 06:29:31.271747046 +0000 UTC m=+0.345100488 container start 889b6b74daad712e13324416be2445dedb868f3bba8ac212243eef3121bdb918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T06:29:31.354 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:31 smithi110 podman[68795]: 2024-09-21 06:29:31.351996228 +0000 UTC m=+0.425349690 container attach 889b6b74daad712e13324416be2445dedb868f3bba8ac212243eef3121bdb918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T06:29:31.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:31 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:31.052+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:31.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:31 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:31.052+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:31.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:31 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:31.052+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:31.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:31 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:31.052+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:31.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:31 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:31.052+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:31.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:31 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:31.052+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:31.778 INFO:journalctl@ceph.osd.8.smithi110.stdout:Sep 21 06:29:31 smithi110 podman[68795]: 2024-09-21 06:29:31.46342338 +0000 UTC m=+0.536776818 container died 889b6b74daad712e13324416be2445dedb868f3bba8ac212243eef3121bdb918 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:29:32.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T06:29:32.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:32.130 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T06:29:32.130 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T06:29:32.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.9 2024-09-21T06:29:32.179 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:31 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:31.910+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:32.179 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:31 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:31.910+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:32.179 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:31 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:31.910+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:32.179 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:31 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:31.910+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:32.179 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:31 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:31.910+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:32.438 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 systemd[1]: Stopping Ceph osd.9 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:32.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:32 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:32.232+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:32.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:32 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:32.232+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:32.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:32 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:32.232+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:32.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:32 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:32.232+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:32.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:32 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:32.232+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:32.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:32 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:32.088+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:32.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:32 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:32.088+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:32.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:32 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:32.088+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:32.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:32 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:32.088+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:32.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:32 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:32.088+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:32.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:32 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:32.088+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:32.885 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.436+0000 7f5e04ebf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:32.886 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.436+0000 7f5e04ebf640 -1 osd.9 101 *** Got signal Terminated *** 2024-09-21T06:29:32.886 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.436+0000 7f5e04ebf640 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:33.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:33.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:33.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:33.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:33.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:32 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:32.915+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:33.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:33.214+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:33.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:33.214+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:33.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:33.214+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:33.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:33.214+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:33.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:33 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:33.214+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:33.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:33 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:33.133+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:33.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:33 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:33.133+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:33.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:33 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:33.133+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:33.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:33 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:33.133+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:33.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:33 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:33.133+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:33.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:33 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:33.133+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:34.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:33 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:33.894+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:34.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:33 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:33.894+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:34.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:33 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:33.894+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:34.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:33 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:33.894+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:34.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:33 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:33.894+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:34.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:34 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:34.191+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:34.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:34 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:34.191+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:34.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:34 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:34.191+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:34.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:34 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:34.191+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:34.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:34 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:34.191+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:34.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:34 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:34.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:34.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:34 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:34.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:34.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:34 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:34.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:34.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:34 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:34.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:34.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:34 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:34.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:34.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:34 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:34.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:35.289 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:35 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:35.138+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:35.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:35 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:35.138+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:35.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:35 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:35.138+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:35.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:35 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:35.138+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:35.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:35 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:35.138+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:35.290 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:35 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:35.138+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:35.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:34 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:34.903+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:35.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:34 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:34.903+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:35.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:34 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:34.903+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:35.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:34 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:34.903+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:35.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:34 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:34.903+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:35.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:35 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:35.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:35.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:35 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:35.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:35.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:35 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:35.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:35.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:35 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:35.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:35.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:35 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:35.167+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:36.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:35 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:35.862+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:36.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:35 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:35.862+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:36.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:35 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:35.862+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:36.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:35 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:35.862+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:36.136 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:35 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:35.862+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:36.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:36.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:36.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:36.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:36.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:36.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:36.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:36.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:36.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:36.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:36.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:36.153+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:36.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:36.163+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:36.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:36.163+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:36.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:36.163+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:36.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:36.163+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:36.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:36.163+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:36.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:36 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:36.163+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:37.385 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:36 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:36.899+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:57.296423+0000 front 2024-09-21T06:27:57.296476+0000 (oldest deadline 2024-09-21T06:28:45.100826+0000) 2024-09-21T06:29:37.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:36 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:36.899+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:31.002018+0000 front 2024-09-21T06:28:31.002056+0000 (oldest deadline 2024-09-21T06:28:56.902191+0000) 2024-09-21T06:29:37.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:36 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:36.899+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.203433+0000 front 2024-09-21T06:28:39.203464+0000 (oldest deadline 2024-09-21T06:29:03.903580+0000) 2024-09-21T06:29:37.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:36 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:36.899+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:49.204612+0000 front 2024-09-21T06:28:49.204581+0000 (oldest deadline 2024-09-21T06:29:11.504831+0000) 2024-09-21T06:29:37.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:36 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9[68120]: 2024-09-21T06:29:36.899+0000 7f5e014aa640 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:02.106166+0000 front 2024-09-21T06:29:02.106150+0000 (oldest deadline 2024-09-21T06:29:27.406207+0000) 2024-09-21T06:29:37.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:37 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:37.106+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:37.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:37 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:37.106+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:37.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:37 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:37.106+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:37.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:37 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:37.106+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:37.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:37 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:37.106+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:37.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:37 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:37.106+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:37.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:37 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:37.171+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:38.386 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:38 smithi031 podman[113447]: 2024-09-21 06:29:38.129206352 +0000 UTC m=+5.799021173 container died fcb70cc1450eed4b97d2229c4c74a850ebb66ac95b2f7c8ff1af042f016b50b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:29:38.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:38 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:38.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:38.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:38 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:38.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:38.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:38 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:38.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:38.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:38 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:38.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:38.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:38 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:38.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:38.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:38 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:38.092+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:38.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:38 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:38.203+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:38.747 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:38 smithi031 podman[113447]: 2024-09-21 06:29:38.570190602 +0000 UTC m=+6.240005477 container cleanup fcb70cc1450eed4b97d2229c4c74a850ebb66ac95b2f7c8ff1af042f016b50b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:29:38.748 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:38 smithi031 bash[113447]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9 2024-09-21T06:29:39.135 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:38 smithi031 podman[113462]: 2024-09-21 06:29:38.746460723 +0000 UTC m=+0.612258999 container remove fcb70cc1450eed4b97d2229c4c74a850ebb66ac95b2f7c8ff1af042f016b50b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T06:29:39.509 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:39 smithi031 podman[113552]: 2024-09-21 06:29:39.089312326 +0000 UTC m=+0.016225419 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:39.509 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:39 smithi031 podman[113552]: 2024-09-21 06:29:39.188967679 +0000 UTC m=+0.115880799 container create fc4a769ca32733c8ebb970706741d193fdb9607b7ade7dca0f6fa2fa6f9f92f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:29:39.509 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:39 smithi031 podman[113552]: 2024-09-21 06:29:39.432780337 +0000 UTC m=+0.359693423 container init fc4a769ca32733c8ebb970706741d193fdb9607b7ade7dca0f6fa2fa6f9f92f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T06:29:39.509 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:39 smithi031 podman[113552]: 2024-09-21 06:29:39.436191893 +0000 UTC m=+0.363104983 container start fc4a769ca32733c8ebb970706741d193fdb9607b7ade7dca0f6fa2fa6f9f92f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T06:29:39.509 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:39 smithi031 podman[113552]: 2024-09-21 06:29:39.507675126 +0000 UTC m=+0.434588208 container attach fc4a769ca32733c8ebb970706741d193fdb9607b7ade7dca0f6fa2fa6f9f92f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T06:29:39.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:39 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:39.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:39.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:39 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:39.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:39.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:39 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:39.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:39.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:39 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:39.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:39.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:39 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:39.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:39.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:39 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:39.118+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:39.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:39.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:39.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:39.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:39.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:39.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:39.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:39 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:39.225+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:39.885 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:39 smithi031 podman[113552]: 2024-09-21 06:29:39.629889701 +0000 UTC m=+0.556802790 container died fc4a769ca32733c8ebb970706741d193fdb9607b7ade7dca0f6fa2fa6f9f92f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:29:40.172 INFO:journalctl@ceph.osd.9.smithi031.stdout:Sep 21 06:29:40 smithi031 podman[113576]: 2024-09-21 06:29:40.143973629 +0000 UTC m=+0.508266918 container remove fc4a769ca32733c8ebb970706741d193fdb9607b7ade7dca0f6fa2fa6f9f92f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-9-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:29:40.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T06:29:40.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:40.220 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T06:29:40.221 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T06:29:40.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.10 2024-09-21T06:29:40.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:40 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:40.091+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:40.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:40 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:40.091+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:40.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:40 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:40.091+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:40.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:40 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:40.091+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:40.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:40 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:40.091+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:40.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:40 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:40.091+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.222+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:40.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 systemd[1]: Stopping Ceph osd.10 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:41.039 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.562+0000 7fe71744a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:41.039 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.562+0000 7fe71744a640 -1 osd.10 101 *** Got signal Terminated *** 2024-09-21T06:29:41.039 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:40 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:40.562+0000 7fe71744a640 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:41.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:41.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:41.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:41.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:41.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:41.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:41.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:41.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:41.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:41.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:41.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:41.135+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:41.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:41.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:41.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:41.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:41.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:41.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:41.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:41 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:41.176+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:42.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:42 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:42.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:42.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:42 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:42.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:42.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:42 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:42.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:42.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:42 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:42.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:42.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:42 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:42.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:42.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:42 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:42.112+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:42.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:42 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:42.191+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:43.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:43.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:43.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:43.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:43.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:43.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:43.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:43 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:43.149+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:43.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:43 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:43.172+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:44.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:44.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:44.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:44.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:44.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:44.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:44.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:44 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:44.123+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:44.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:44 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:44.141+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:45.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:45.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:45.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:45.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:45.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:45.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:45.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:45 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:45.110+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:45.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:59.491061+0000 front 2024-09-21T06:27:59.491015+0000 (oldest deadline 2024-09-21T06:28:47.796383+0000) 2024-09-21T06:29:45.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:28.297134+0000 front 2024-09-21T06:28:28.297116+0000 (oldest deadline 2024-09-21T06:28:55.897700+0000) 2024-09-21T06:29:45.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:40.499395+0000 front 2024-09-21T06:28:40.499206+0000 (oldest deadline 2024-09-21T06:29:05.699674+0000) 2024-09-21T06:29:45.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2024-09-21T06:28:48.902208+0000 front 2024-09-21T06:28:48.902285+0000 (oldest deadline 2024-09-21T06:29:11.802183+0000) 2024-09-21T06:29:45.539 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:57.103258+0000 front 2024-09-21T06:28:57.103219+0000 (oldest deadline 2024-09-21T06:29:19.303762+0000) 2024-09-21T06:29:45.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:03.305294+0000 front 2024-09-21T06:29:03.305314+0000 (oldest deadline 2024-09-21T06:29:28.605131+0000) 2024-09-21T06:29:45.540 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:45 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10[58465]: 2024-09-21T06:29:45.105+0000 7fe70fa2d640 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:11.506221+0000 front 2024-09-21T06:29:11.506239+0000 (oldest deadline 2024-09-21T06:29:36.806170+0000) 2024-09-21T06:29:46.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:46.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:46.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:46.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:46.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:46.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:46.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:46 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:46.101+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:46.537 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:46 smithi107 podman[68696]: 2024-09-21 06:29:46.274112679 +0000 UTC m=+5.864143362 container died a7b08048005b6ed6a9758112306e31e664e2294fd4ab8e31a10adcabc5eb8cc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T06:29:46.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:46 smithi107 podman[68696]: 2024-09-21 06:29:46.619545602 +0000 UTC m=+6.209576294 container cleanup a7b08048005b6ed6a9758112306e31e664e2294fd4ab8e31a10adcabc5eb8cc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T06:29:46.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:46 smithi107 bash[68696]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10 2024-09-21T06:29:47.236 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:46 smithi107 podman[68711]: 2024-09-21 06:29:46.829066031 +0000 UTC m=+0.549170652 container remove a7b08048005b6ed6a9758112306e31e664e2294fd4ab8e31a10adcabc5eb8cc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T06:29:47.240 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:47 smithi107 podman[68796]: 2024-09-21 06:29:47.138268469 +0000 UTC m=+0.016087090 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:47.240 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:47 smithi107 podman[68796]: 2024-09-21 06:29:47.237974908 +0000 UTC m=+0.115793564 container create ffe332e9549da19c6da435957a83d3ae16ee237555e2f4a8f383780690115229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T06:29:47.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:47.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:47.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:47.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:47.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:47.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:47.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:47 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:47.066+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:47.532 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:47 smithi107 podman[68796]: 2024-09-21 06:29:47.465264517 +0000 UTC m=+0.343083128 container init ffe332e9549da19c6da435957a83d3ae16ee237555e2f4a8f383780690115229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:29:47.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:47 smithi107 podman[68796]: 2024-09-21 06:29:47.468568145 +0000 UTC m=+0.346386759 container start ffe332e9549da19c6da435957a83d3ae16ee237555e2f4a8f383780690115229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:29:47.533 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:47 smithi107 podman[68796]: 2024-09-21 06:29:47.53183798 +0000 UTC m=+0.409656607 container attach ffe332e9549da19c6da435957a83d3ae16ee237555e2f4a8f383780690115229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:29:47.789 INFO:journalctl@ceph.osd.10.smithi107.stdout:Sep 21 06:29:47 smithi107 podman[68796]: 2024-09-21 06:29:47.664311637 +0000 UTC m=+0.542130264 container died ffe332e9549da19c6da435957a83d3ae16ee237555e2f4a8f383780690115229 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T06:29:48.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T06:29:48.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:48.299 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T06:29:48.299 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T06:29:48.299 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.11 2024-09-21T06:29:48.325 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:48.326 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:48.326 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:48.326 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:48.326 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:48.326 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:48.326 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.053+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:48.617 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 systemd[1]: Stopping Ceph osd.11 for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:49.028 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.615+0000 7f3779d9d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:49.028 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.615+0000 7f3779d9d640 -1 osd.11 101 *** Got signal Terminated *** 2024-09-21T06:29:49.028 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:48 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:48.615+0000 7f3779d9d640 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:29:49.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:49.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:49.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:49.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:49.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:49.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:49.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:49 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:49.085+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:50.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:50.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:50.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:50.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:50.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:50.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:50.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:50 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:50.045+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:51.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:51.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:51.528 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:51.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:51.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:51.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:51.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:51.529 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:51 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:51.081+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-09-21T06:29:25.670257+0000 front 2024-09-21T06:29:25.670294+0000 (oldest deadline 2024-09-21T06:29:50.370297+0000) 2024-09-21T06:29:52.429 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:52.430 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:52 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:52.114+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-09-21T06:29:25.670257+0000 front 2024-09-21T06:29:25.670294+0000 (oldest deadline 2024-09-21T06:29:50.370297+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-21T06:27:55.555242+0000 front 2024-09-21T06:27:55.555355+0000 (oldest deadline 2024-09-21T06:28:46.260843+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6804 osd.1 since back 2024-09-21T06:28:29.062454+0000 front 2024-09-21T06:28:29.062562+0000 (oldest deadline 2024-09-21T06:28:54.362563+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-09-21T06:28:39.063523+0000 front 2024-09-21T06:28:39.063660+0000 (oldest deadline 2024-09-21T06:29:06.064129+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6812 osd.4 since back 2024-09-21T06:28:53.566372+0000 front 2024-09-21T06:28:53.566271+0000 (oldest deadline 2024-09-21T06:29:19.466061+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-09-21T06:29:04.767134+0000 front 2024-09-21T06:29:04.767087+0000 (oldest deadline 2024-09-21T06:29:26.367607+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2024-09-21T06:29:09.868789+0000 front 2024-09-21T06:29:09.868771+0000 (oldest deadline 2024-09-21T06:29:35.768505+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.107:6820 osd.7 since back 2024-09-21T06:29:19.869508+0000 front 2024-09-21T06:29:19.869647+0000 (oldest deadline 2024-09-21T06:29:42.769429+0000) 2024-09-21T06:29:53.448 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:53 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11[58798]: 2024-09-21T06:29:53.087+0000 7f3772380640 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-09-21T06:29:25.670257+0000 front 2024-09-21T06:29:25.670294+0000 (oldest deadline 2024-09-21T06:29:50.370297+0000) 2024-09-21T06:29:54.445 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:54 smithi110 podman[68898]: 2024-09-21 06:29:54.13917309 +0000 UTC m=+5.644091321 container died 4c1c853bc3ec3a792532c226fbf99b06a5f56960c5d07b22bcc7536416000aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:29:54.707 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:54 smithi110 podman[68898]: 2024-09-21 06:29:54.543288731 +0000 UTC m=+6.048206969 container cleanup 4c1c853bc3ec3a792532c226fbf99b06a5f56960c5d07b22bcc7536416000aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:29:54.707 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:54 smithi110 bash[68898]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11 2024-09-21T06:29:55.028 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:54 smithi110 podman[68923]: 2024-09-21 06:29:54.7948133 +0000 UTC m=+0.649075792 container remove 4c1c853bc3ec3a792532c226fbf99b06a5f56960c5d07b22bcc7536416000aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T06:29:55.365 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:55 smithi110 podman[69017]: 2024-09-21 06:29:55.004524431 +0000 UTC m=+0.016337376 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:29:55.365 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:55 smithi110 podman[69017]: 2024-09-21 06:29:55.120570895 +0000 UTC m=+0.132383833 container create c06ae047e38795d83fd527a9ee23aafaf467c987efc60b6a6a796edfa4fda80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T06:29:55.365 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:55 smithi110 podman[69017]: 2024-09-21 06:29:55.364641228 +0000 UTC m=+0.376454167 container init c06ae047e38795d83fd527a9ee23aafaf467c987efc60b6a6a796edfa4fda80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:29:55.778 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:55 smithi110 podman[69017]: 2024-09-21 06:29:55.367913626 +0000 UTC m=+0.379726570 container start c06ae047e38795d83fd527a9ee23aafaf467c987efc60b6a6a796edfa4fda80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T06:29:55.778 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:55 smithi110 podman[69017]: 2024-09-21 06:29:55.447983589 +0000 UTC m=+0.459796549 container attach c06ae047e38795d83fd527a9ee23aafaf467c987efc60b6a6a796edfa4fda80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:29:55.779 INFO:journalctl@ceph.osd.11.smithi110.stdout:Sep 21 06:29:55 smithi110 podman[69017]: 2024-09-21 06:29:55.551542281 +0000 UTC m=+0.563355231 container died c06ae047e38795d83fd527a9ee23aafaf467c987efc60b6a6a796edfa4fda80b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T06:29:56.193 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T06:29:56.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:29:56.274 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T06:29:56.274 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T06:29:56.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.a 2024-09-21T06:29:56.588 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:29:56 smithi031 systemd[1]: Stopping Ceph mds.a for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:29:56.864 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:29:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a[74478]: 2024-09-21T06:29:56.585+0000 7f3517753640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:29:56.864 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:29:56 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a[74478]: 2024-09-21T06:29:56.585+0000 7f3517753640 -1 mds.a *** got signal Terminated *** 2024-09-21T06:30:00.698 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:30:00 smithi031 podman[113655]: 2024-09-21 06:30:00.413662511 +0000 UTC m=+3.933586022 container died 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:30:01.135 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:30:00 smithi031 podman[113655]: 2024-09-21 06:30:00.928056212 +0000 UTC m=+4.447979712 container cleanup 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:30:01.136 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:30:00 smithi031 bash[113655]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a 2024-09-21T06:30:01.406 INFO:journalctl@ceph.mds.a.smithi031.stdout:Sep 21 06:30:01 smithi031 podman[113680]: 2024-09-21 06:30:01.154272177 +0000 UTC m=+0.737062514 container remove 22ee396dc0fc0e3e05eb68afdbe9c68d51270dd344e9748631be0f8e8d949842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T06:30:01.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T06:30:01.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:01.457 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T06:30:01.457 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T06:30:01.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.d 2024-09-21T06:30:01.792 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:30:01 smithi031 systemd[1]: Stopping Ceph mds.d for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:02.135 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:30:01 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-d[73445]: 2024-09-21T06:30:01.790+0000 7fe201c71640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:02.136 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:30:01 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-d[73445]: 2024-09-21T06:30:01.790+0000 7fe201c71640 -1 mds.d *** got signal Terminated *** 2024-09-21T06:30:06.074 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:30:05 smithi031 podman[113790]: 2024-09-21 06:30:05.800413321 +0000 UTC m=+4.112217708 container died 241ef4fdd1f86d3faada4577a46e5fcec42aeb205ab137a5db4658b9c48ce3f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:30:06.385 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:30:06 smithi031 podman[113790]: 2024-09-21 06:30:06.264629283 +0000 UTC m=+4.576433658 container cleanup 241ef4fdd1f86d3faada4577a46e5fcec42aeb205ab137a5db4658b9c48ce3f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T06:30:06.385 INFO:journalctl@ceph.mds.d.smithi031.stdout:Sep 21 06:30:06 smithi031 bash[113790]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-d 2024-09-21T06:30:06.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T06:30:06.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:06.741 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T06:30:06.741 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T06:30:06.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.g 2024-09-21T06:30:07.078 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:30:06 smithi031 systemd[1]: Stopping Ceph mds.g for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:07.354 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:30:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-g[74174]: 2024-09-21T06:30:07.076+0000 7fb20ab6f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:07.355 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:30:07 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-g[74174]: 2024-09-21T06:30:07.076+0000 7fb20ab6f640 -1 mds.g *** got signal Terminated *** 2024-09-21T06:30:10.337 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:30:09 smithi031 podman[113908]: 2024-09-21 06:30:09.953288786 +0000 UTC m=+2.983426329 container died ee5a195a692392819841ce4980f7a735ad2f249010f32b876fcc2ab0be0aede8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-g, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T06:30:10.635 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:30:10 smithi031 podman[113908]: 2024-09-21 06:30:10.502012165 +0000 UTC m=+3.532149705 container cleanup ee5a195a692392819841ce4980f7a735ad2f249010f32b876fcc2ab0be0aede8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T06:30:10.635 INFO:journalctl@ceph.mds.g.smithi031.stdout:Sep 21 06:30:10 smithi031 bash[113908]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-g 2024-09-21T06:30:10.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T06:30:10.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:10.988 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T06:30:10.988 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T06:30:10.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.j 2024-09-21T06:30:11.382 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:11 smithi031 systemd[1]: Stopping Ceph mds.j for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:11.635 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j[73064]: 2024-09-21T06:30:11.380+0000 7f2d54cb3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:11.636 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:11 smithi031 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j[73064]: 2024-09-21T06:30:11.380+0000 7f2d54cb3640 -1 mds.j *** got signal Terminated *** 2024-09-21T06:30:15.356 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:14 smithi031 podman[114029]: 2024-09-21 06:30:14.983739076 +0000 UTC m=+3.772875388 container died 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:30:15.356 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:15 smithi031 podman[114029]: 2024-09-21 06:30:15.355381767 +0000 UTC m=+4.144518067 container cleanup 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:30:15.635 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:15 smithi031 bash[114029]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j 2024-09-21T06:30:15.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T06:30:15.979 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:15 smithi031 podman[114041]: 2024-09-21 06:30:15.723133471 +0000 UTC m=+0.739276241 container remove 0e0af7b12f8c2eb15f322871394d9c6ce25088f06326f4e5a00af369a15400b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T06:30:15.979 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:15 smithi031 systemd[1]: ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.j.service: Deactivated successfully. 2024-09-21T06:30:15.979 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:15 smithi031 systemd[1]: Stopped Ceph mds.j for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:30:15.979 INFO:journalctl@ceph.mds.j.smithi031.stdout:Sep 21 06:30:15 smithi031 systemd[1]: ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.j.service: Consumed 1.135s CPU time. 2024-09-21T06:30:15.989 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:15.990 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T06:30:15.990 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T06:30:15.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.b 2024-09-21T06:30:16.303 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:16 smithi107 systemd[1]: Stopping Ceph mds.b for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:16.789 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: 2024-09-21T06:30:16.301+0000 7f16384c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:16.789 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:16 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b[63806]: 2024-09-21T06:30:16.301+0000 7f16384c7640 -1 mds.b *** got signal Terminated *** 2024-09-21T06:30:17.570 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:17 smithi107 podman[68911]: 2024-09-21 06:30:17.303509901 +0000 UTC m=+1.110508053 container died 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:30:17.844 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:17 smithi107 podman[68911]: 2024-09-21 06:30:17.650478717 +0000 UTC m=+1.457476868 container cleanup 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T06:30:17.844 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:17 smithi107 bash[68911]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b 2024-09-21T06:30:18.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T06:30:18.095 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:17 smithi107 podman[68924]: 2024-09-21 06:30:17.843139489 +0000 UTC m=+0.535147991 container remove 824fe551eecd95febedc79be7cb58f6b83c288ab86bb2173b49c9dc349abc7d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T06:30:18.095 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:18 smithi107 systemd[1]: ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.b.service: Deactivated successfully. 2024-09-21T06:30:18.095 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:18 smithi107 systemd[1]: Stopped Ceph mds.b for c04f7042-77de-11ef-baf5-efdc52797490. 2024-09-21T06:30:18.095 INFO:journalctl@ceph.mds.b.smithi107.stdout:Sep 21 06:30:18 smithi107 systemd[1]: ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.b.service: Consumed 1min 8.088s CPU time. 2024-09-21T06:30:18.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:18.108 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T06:30:18.108 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T06:30:18.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.e 2024-09-21T06:30:18.422 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:30:18 smithi107 systemd[1]: Stopping Ceph mds.e for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:18.789 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:30:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e[63258]: 2024-09-21T06:30:18.421+0000 7fc194855640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:18.789 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:30:18 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e[63258]: 2024-09-21T06:30:18.421+0000 7fc194855640 -1 mds.e *** got signal Terminated *** 2024-09-21T06:30:21.414 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:30:21 smithi107 podman[69032]: 2024-09-21 06:30:21.124550127 +0000 UTC m=+2.799943307 container died d00d1338772c0ec2a6c65ff021c0e09658847a677bde4556c2545a7e8692363f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:30:21.747 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:30:21 smithi107 podman[69032]: 2024-09-21 06:30:21.50390819 +0000 UTC m=+3.179301361 container cleanup d00d1338772c0ec2a6c65ff021c0e09658847a677bde4556c2545a7e8692363f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T06:30:21.747 INFO:journalctl@ceph.mds.e.smithi107.stdout:Sep 21 06:30:21 smithi107 bash[69032]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-e 2024-09-21T06:30:21.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T06:30:21.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:21.915 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T06:30:21.916 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T06:30:21.916 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.h 2024-09-21T06:30:22.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:21 smithi107 systemd[1]: Stopping Ceph mds.h for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:22.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h[63527]: 2024-09-21T06:30:22.224+0000 7f3d5c8a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:22.289 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:22 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h[63527]: 2024-09-21T06:30:22.224+0000 7f3d5c8a7640 -1 mds.h *** got signal Terminated *** 2024-09-21T06:30:23.475 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:23 smithi107 podman[69154]: 2024-09-21 06:30:23.215111963 +0000 UTC m=+1.081285200 container died c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:30:23.475 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:23 smithi107 podman[69154]: 2024-09-21 06:30:23.474022046 +0000 UTC m=+1.340195293 container cleanup c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:30:23.789 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:23 smithi107 bash[69154]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h 2024-09-21T06:30:23.789 INFO:journalctl@ceph.mds.h.smithi107.stdout:Sep 21 06:30:23 smithi107 podman[69166]: 2024-09-21 06:30:23.691736018 +0000 UTC m=+0.472050522 container remove c57195ac8e99a4a1fcd1d83419d9a56eaee397d8b7fa090f987032a949527759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:30:23.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T06:30:23.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:23.861 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T06:30:23.861 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T06:30:23.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.k 2024-09-21T06:30:24.162 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:23 smithi107 systemd[1]: Stopping Ceph mds.k for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:24.539 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k[62971]: 2024-09-21T06:30:24.160+0000 7fea0e1da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:24.539 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:24 smithi107 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k[62971]: 2024-09-21T06:30:24.160+0000 7fea0e1da640 -1 mds.k *** got signal Terminated *** 2024-09-21T06:30:28.039 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:27 smithi107 podman[69273]: 2024-09-21 06:30:27.776775661 +0000 UTC m=+3.700209657 container died f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T06:30:28.407 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:28 smithi107 podman[69273]: 2024-09-21 06:30:28.076774991 +0000 UTC m=+4.000208986 container cleanup f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T06:30:28.407 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:28 smithi107 bash[69273]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k 2024-09-21T06:30:28.407 INFO:journalctl@ceph.mds.k.smithi107.stdout:Sep 21 06:30:28 smithi107 podman[69285]: 2024-09-21 06:30:28.294278707 +0000 UTC m=+0.516081371 container remove f75eddb83e30c196af3c5fd9820f6b7bcde1f3f9a197f992d37661ee65f91768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-k, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T06:30:28.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T06:30:28.461 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:28.462 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T06:30:28.462 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T06:30:28.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.c 2024-09-21T06:30:28.762 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:28 smithi110 systemd[1]: Stopping Ceph mds.c for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:29.028 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-c[63915]: 2024-09-21T06:30:28.760+0000 7f636d6e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:29.028 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:28 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-c[63915]: 2024-09-21T06:30:28.760+0000 7f636d6e3640 -1 mds.c *** got signal Terminated *** 2024-09-21T06:30:29.972 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:29 smithi110 podman[69121]: 2024-09-21 06:30:29.844358804 +0000 UTC m=+1.187961371 container died 5a0a47c244dedafb9874305724d393934c6eceabb6b76c034f7f60bc7252d544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T06:30:30.669 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:30 smithi110 podman[69121]: 2024-09-21 06:30:30.469046895 +0000 UTC m=+1.812649458 container cleanup 5a0a47c244dedafb9874305724d393934c6eceabb6b76c034f7f60bc7252d544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T06:30:30.670 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:30 smithi110 bash[69121]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-c 2024-09-21T06:30:30.925 INFO:journalctl@ceph.mds.c.smithi110.stdout:Sep 21 06:30:30 smithi110 podman[69134]: 2024-09-21 06:30:30.668386643 +0000 UTC m=+0.826895819 container remove 5a0a47c244dedafb9874305724d393934c6eceabb6b76c034f7f60bc7252d544 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T06:30:30.939 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T06:30:30.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:30.976 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T06:30:30.976 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T06:30:30.976 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.f 2024-09-21T06:30:31.278 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:30:31 smithi110 systemd[1]: Stopping Ceph mds.f for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:31.778 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:30:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f[63642]: 2024-09-21T06:30:31.365+0000 7ff8f7463640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:31.778 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:30:31 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f[63642]: 2024-09-21T06:30:31.365+0000 7ff8f7463640 -1 mds.f *** got signal Terminated *** 2024-09-21T06:30:35.659 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:30:35 smithi110 podman[69252]: 2024-09-21 06:30:35.380206575 +0000 UTC m=+4.188913113 container died 4c4fe3b56612acdd2fd7e02e0af2e9fac7e595c3ab12f1e5c5ab66de5659bd15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:30:36.021 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:30:35 smithi110 podman[69252]: 2024-09-21 06:30:35.75689042 +0000 UTC m=+4.565596948 container cleanup 4c4fe3b56612acdd2fd7e02e0af2e9fac7e595c3ab12f1e5c5ab66de5659bd15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:30:36.021 INFO:journalctl@ceph.mds.f.smithi110.stdout:Sep 21 06:30:35 smithi110 bash[69252]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-f 2024-09-21T06:30:36.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T06:30:36.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:36.218 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T06:30:36.218 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T06:30:36.218 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.i 2024-09-21T06:30:36.545 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:36 smithi110 systemd[1]: Stopping Ceph mds.i for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:37.028 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i[64199]: 2024-09-21T06:30:36.543+0000 7f979ad16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:37.028 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:36 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i[64199]: 2024-09-21T06:30:36.543+0000 7f979ad16640 -1 mds.i *** got signal Terminated *** 2024-09-21T06:30:40.778 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:40 smithi110 podman[69373]: 2024-09-21 06:30:40.559527477 +0000 UTC m=+4.128388521 container died 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:30:41.072 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:40 smithi110 podman[69373]: 2024-09-21 06:30:40.883072367 +0000 UTC m=+4.451933396 container cleanup 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:30:41.073 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:40 smithi110 bash[69373]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i 2024-09-21T06:30:41.447 INFO:journalctl@ceph.mds.i.smithi110.stdout:Sep 21 06:30:41 smithi110 podman[69384]: 2024-09-21 06:30:41.17919942 +0000 UTC m=+0.616377372 container remove 4da8b98930b7400dd432b537daa2669a221e1aeae035c7a2d4ffabd61fb280d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T06:30:41.464 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T06:30:41.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:41.497 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T06:30:41.497 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T06:30:41.497 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.l 2024-09-21T06:30:41.778 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:30:41 smithi110 systemd[1]: Stopping Ceph mds.l for c04f7042-77de-11ef-baf5-efdc52797490... 2024-09-21T06:30:42.278 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:30:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l[63372]: 2024-09-21T06:30:41.806+0000 7f8ac7f0e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:30:42.278 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:30:41 smithi110 ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l[63372]: 2024-09-21T06:30:41.806+0000 7f8ac7f0e640 -1 mds.l *** got signal Terminated *** 2024-09-21T06:30:45.683 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:30:45 smithi110 podman[69495]: 2024-09-21 06:30:45.380041631 +0000 UTC m=+3.667845347 container died 9ed8dca21fd2bb50753f79490ac4f38e54e818f27c48e9b54aa39c05ce1018ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T06:30:45.975 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:30:45 smithi110 podman[69495]: 2024-09-21 06:30:45.781804287 +0000 UTC m=+4.069608010 container cleanup 9ed8dca21fd2bb50753f79490ac4f38e54e818f27c48e9b54aa39c05ce1018ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T06:30:45.975 INFO:journalctl@ceph.mds.l.smithi110.stdout:Sep 21 06:30:45 smithi110 bash[69495]: ceph-c04f7042-77de-11ef-baf5-efdc52797490-mds-l 2024-09-21T06:30:46.101 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c04f7042-77de-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T06:30:46.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:30:46.133 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T06:30:46.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c04f7042-77de-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:30:46.347 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:30:48.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c04f7042-77de-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:30:49.151 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:30:51.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c04f7042-77de-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:30:51.842 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:30:54.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:30:54.308 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:30:54.340 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:30:54.367 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T06:30:54.369 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi031/crash 2024-09-21T06:30:54.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:30:54.410 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:30:54.410 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:30:54.412 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi107/crash 2024-09-21T06:30:54.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:30:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:30:54.455 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:30:54.457 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi110/crash 2024-09-21T06:30:54.457 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:30:54.496 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/c04f7042-77de-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:30:54.496 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:30:54.497 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T06:30:54.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T06:30:54.537 INFO:tasks.cephadm:Compressing logs... 2024-09-21T06:30:54.538 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:30:54.580 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:30:54.583 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:30:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T06:30:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:: No such file or directory 2024-09-21T06:30:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T06:30:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:30:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:30:54.612 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.a.log: 83.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.6.log: 90.1% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:30:54.613 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.i.log: /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T06:30:54.617 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip 80.7% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr: -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log: 90.3% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr: 86.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T06:30:54.618 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T06:30:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log: 86.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:30:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T06:30:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.d.log: 95.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi031.stderr: 95.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-09-21T06:30:54.624 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log: 90.2% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log: 80.7% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.y.log: 95.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T06:30:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T06:30:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.3.log: 95.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T06:30:54.629 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.8.log: 90.8% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:30:54.629 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T06:30:54.629 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.z.log: 94.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T06:30:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log: 86.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:30:54.630 INFO:teuthology.orchestra.run.smithi107.stderr: 91.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:30:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T06:30:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T06:30:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.b.log: 92.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T06:30:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:30:54.631 INFO:teuthology.orchestra.run.smithi110.stderr: 92.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T06:30:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T06:30:54.631 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:30:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/cephadm.log: 95.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T06:30:54.633 INFO:teuthology.orchestra.run.smithi031.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:30:54.639 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.f.log: 95.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T06:30:54.639 INFO:teuthology.orchestra.run.smithi110.stderr: 94.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T06:30:54.640 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.e.log: 95.0%/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T06:30:54.640 INFO:teuthology.orchestra.run.smithi107.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:30:54.641 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:30:54.911 INFO:teuthology.orchestra.run.smithi110.stderr: 93.4% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T06:30:55.069 INFO:teuthology.orchestra.run.smithi031.stderr: 90.4% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T06:30:55.206 INFO:teuthology.orchestra.run.smithi110.stderr: 91.6% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T06:30:55.234 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T06:30:55.317 INFO:teuthology.orchestra.run.smithi107.stderr: 94.2% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T06:30:55.789 INFO:teuthology.orchestra.run.smithi031.stderr: 90.2% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T06:31:05.873 INFO:teuthology.orchestra.run.smithi031.stderr: 94.1% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T06:31:08.177 INFO:teuthology.orchestra.run.smithi110.stderr: 94.1% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T06:31:11.326 INFO:teuthology.orchestra.run.smithi110.stderr: 94.1% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T06:31:12.888 INFO:teuthology.orchestra.run.smithi107.stderr: 93.8% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T06:31:13.313 INFO:teuthology.orchestra.run.smithi107.stderr: 94.2% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T06:31:13.463 INFO:teuthology.orchestra.run.smithi110.stderr: 93.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T06:31:14.622 INFO:teuthology.orchestra.run.smithi031.stderr: 94.1% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T06:31:15.153 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T06:31:15.375 INFO:teuthology.orchestra.run.smithi110.stderr: 94.4% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T06:31:15.378 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-21T06:31:15.378 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m20.781s 2024-09-21T06:31:15.378 INFO:teuthology.orchestra.run.smithi110.stderr:user 1m8.121s 2024-09-21T06:31:15.378 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m2.629s 2024-09-21T06:31:16.407 INFO:teuthology.orchestra.run.smithi107.stderr: 93.9% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T06:31:19.357 INFO:teuthology.orchestra.run.smithi031.stderr: 94.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T06:31:20.889 INFO:teuthology.orchestra.run.smithi031.stderr: 94.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T06:31:20.890 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-21T06:31:20.890 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m26.297s 2024-09-21T06:31:20.890 INFO:teuthology.orchestra.run.smithi031.stderr:user 1m20.974s 2024-09-21T06:31:20.891 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m2.883s 2024-09-21T06:31:29.686 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/c04f7042-77de-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T06:31:29.688 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-21T06:31:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m35.090s 2024-09-21T06:31:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:user 1m51.737s 2024-09-21T06:31:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m3.809s 2024-09-21T06:31:29.688 INFO:tasks.cephadm:Archiving logs... 2024-09-21T06:31:29.689 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi031/log 2024-09-21T06:31:29.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:31:40.257 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi107/log 2024-09-21T06:31:40.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:31:53.891 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi110/log 2024-09-21T06:31:53.891 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:32:03.009 INFO:tasks.cephadm:Removing cluster... 2024-09-21T06:32:03.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c04f7042-77de-11ef-baf5-efdc52797490 --force 2024-09-21T06:32:03.224 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:32:03.594 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c04f7042-77de-11ef-baf5-efdc52797490 --force 2024-09-21T06:32:03.798 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:32:04.187 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c04f7042-77de-11ef-baf5-efdc52797490 --force 2024-09-21T06:32:04.391 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: c04f7042-77de-11ef-baf5-efdc52797490 2024-09-21T06:32:04.729 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T06:32:04.729 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:32:04.748 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:32:04.764 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:32:04.780 INFO:tasks.cephadm:Teardown complete 2024-09-21T06:32:04.780 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T06:32:04.792 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T06:32:04.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:32:04.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:32:04.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:32:04.916 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T06:32:04.926 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T06:32:04.926 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:32:04.929 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:32:04.932 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:32:04.944 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:32:04.944 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:32:04.946 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:32:04.948 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:32:04.948 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-21T06:32:04.948 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -26us[-8891ns] +/- 50ms 2024-09-21T06:32:04.948 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +287us[ +287us] +/- 77ms 2024-09-21T06:32:04.948 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +14us[ +14us] +/- 81ms 2024-09-21T06:32:04.948 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:32:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:32:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-21T06:32:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -96us[ -45us] +/- 50ms 2024-09-21T06:32:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 +261us[ +311us] +/- 77ms 2024-09-21T06:32:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -16us[ +35us] +/- 81ms 2024-09-21T06:32:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:32:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:32:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-21T06:32:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -71us[ -42us] +/- 50ms 2024-09-21T06:32:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +298us[ +298us] +/- 77ms 2024-09-21T06:32:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -3536ns[-3536ns] +/- 81ms 2024-09-21T06:32:04.952 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:32:04.952 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T06:32:04.963 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T06:32:04.963 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T06:32:04.974 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:32:05.080 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:32:05.145 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:32:05.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:32:05.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:32:05.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:32:05.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:32:05.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:32:05.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:32:05.296 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T06:32:05.312 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T06:32:05.342 INFO:teuthology.task.internal:Duration was 2267.744331 seconds 2024-09-21T06:32:05.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T06:32:05.356 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T06:32:05.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:32:05.359 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:32:05.361 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:32:05.398 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:32:05.399 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:32:05.401 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:32:05.823 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T06:32:05.823 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-21T06:32:05.824 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:32:05.847 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-21T06:32:05.847 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:32:05.943 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-21T06:32:05.943 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:32:05.969 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T06:32:05.969 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:32:05.971 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:32:05.973 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:32:06.035 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T06:32:06.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:32:06.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:32:06.041 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:32:07.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T06:32:07.412 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T06:32:07.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:32:07.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:32:07.472 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:32:07.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T06:32:07.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:32:07.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:32:07.524 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:32:07.542 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-21T06:32:07.548 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-21T06:32:07.576 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-21T06:32:07.640 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:32:07.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:32:07.656 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:32:07.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:32:07.698 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:32:07.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:32:07.715 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T06:32:07.729 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T06:32:07.731 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi031 2024-09-21T06:32:07.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:32:07.792 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi107 2024-09-21T06:32:07.792 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:32:07.831 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913520/remote/smithi110 2024-09-21T06:32:07.832 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:32:07.870 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T06:32:07.870 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:32:07.873 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:32:07.875 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:32:07.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T06:32:07.936 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T06:32:07.936 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T06:32:07.951 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T06:32:07.951 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:32:07.953 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:32:07.956 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:32:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:32 /home/ubuntu/cephtest 2024-09-21T06:32:07.969 INFO:teuthology.orchestra.run.smithi107.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:32 /home/ubuntu/cephtest 2024-09-21T06:32:07.979 INFO:teuthology.orchestra.run.smithi110.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:32 /home/ubuntu/cephtest 2024-09-21T06:32:07.980 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T06:32:07.991 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T06:32:08.023 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 2267.744330883026 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T06:32:08.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:32:08.147 INFO:teuthology.run:pass